AIRLLM

Summary from the TLDR DevOps newsletter

AirLLM allows 70B large language models to run on a single 4GB GPU without quantization. The tool can run Kimi K3 (2.8T) on under 4GB of VRAM and DeepSeek-V3 (671B) on about 12GB. It works by keeping only one layer on the GPU at a time. AirLLM also supports training huge models on small VRAM, with Qwen3.8-Flash-Next (125B) training under 6GB.