SPEED IS BECOMING MORE IMPORTANT THAN INTELLIGENCE FOR AI MODELS

Summary from the TLDR DevOps newsletter

Once models are capable enough for everyday coding, research, and analysis, inference speed can matter more than marginal gains in intelligence, with roughly 100–200 output tokens per second feeling fast enough for interactive work. Further model acceleration eventually produces diminishing returns because tool calls, databases, local hardware, and human review become the dominant bottlenecks, shifting competition toward serving speed and price.