JAMESOB'S GUIDE TO RUNNING SOTA LLMS LOCALLY

Summary from the TLDR AI newsletter

This guide discusses the hardware used to run SOTA AI models locally and how to run speech-to-text (STT) locally. It also provides a ready-to-run configuration for running models within Docker containers. A budget of $2,000 can get you a setup that runs Qwen and good STT. $40,000 can get you a machine that can run almost-Opus-level models.