LOCAL LLMS ON MOBILE ARE A GIMMICK - FOR NOW

Summary from the TLDR Web Dev newsletter

Current limitations for running LLMs locally on mobile devices include large model sizes, the need for specific model formats and inference frameworks, and performance trade-offs between speed and quality, leading to device heating and battery drain. Frameworks like MLC try to simplify the process by compiling and optimizing models for specific devices. While on-device AI is probably a big part of the future, it's a gimmick in its current state.