RUNNING LLMS IN THE BROWSER WITH THREE.JS

Summary from the TLDR newsletter

Three-LLM runs GPT-2, SmolLM2, Qwen, and Phi models locally in the browser by turning their inference graphs into Three.js TSL compute shaders on WebGPU. The project was created to see how far the general computer capabilities of Three.js and WebGPU can be pushed, and to see what today's small language models can do when they run in a browser. Three.js is known as a rendering library, but its WebGPU renderer also exposes storage buffers, compute dispatches, workgroup memory, atomics, and GPU readback. Large language models put all of those capabilities to work as they need hundreds of ordered compute dispatches, large matrix-vector products, reductions, persistent attention or recurrent state, and a tight loop between the GPU and JavaScript as each new token appears.