FLASH-MOE

Summary from the TLDR AI newsletter

Flash-MoE is an inference engine that can run Qwen3.5-397B-A17B (a 397 billion parameter Mixture-of-Experts model) on a MacBook Pro with 48GB RAM at 4.4+ tokens/second. It streams the entire 209 GB model from SSD through a custom Metal compute pipeline without Python or frameworks. Details about the architecture are available in the repository.