RUNNING LLMS ON PERSONAL DEVICES

Summary from the TLDR AI newsletter

This paper introduces INT-FP-QSim, an open-source simulator that tests how well large language models and vision transformers can work at different levels of numerical precision.