BUILDING AN INFERENCE STACK THAT IMPROVES ITSELF

Summary from the TLDR AI newsletter

Decagon's AutoResearch agents diagnose inference bottlenecks, test optimizations against production traffic replays, and preserve findings across experiments. The company reports throughput gains up to 2.5x within endpoint latency requirements through quantization, scheduling, engine patches, and speculative decoding, with production changes requiring human approval.