META RELEASES EXECUTORCH FRAMEWORK FOR LLM ON EDGE DEVICES

Summary from the TLDR AI newsletter

Meta's ExecuTorch Framework is a post-training quantization toolkit that enables running Llama models on various iPhone and Galaxy devices. It is able to get as much as 11 tokens per second on mobile phones running 7B sized language models.