APPLE'S 20B ON-DEVICE MODEL USES JUST 2GB OF MEMORY

Summary from the TLDR AI newsletter

Apple's 20-billion-parameter on-device model keeps most of its weights in flash storage and activates only 1–4 billion parameters per prompt. Testing on an M3 Max Mac showed roughly 2GB of memory usage and significantly lower energy consumption than comparable open models, but weaker math performance.