META'S QUANTIZED SMALL MODELS

Summary from the TLDR AI newsletter

Meta has quantized its 1B and 3B language models to be 2-4x faster at minimal loss of quality while being 50%+ smaller. This was enabled by its quantization-aware training setup, which allows for models to adjust to the small precision.