TLDR AI · DEEP DIVES & ANALYSIS

DFLASH AND SPEC V2 DECODING

lmsys.org14 min read

Summary from the TLDR AI newsletter

Detailing the latest generation of speculative decoding with DFlash and SGLang's Spec V2 engine. Benchmarks showed substantial throughput gains over baseline inference and native MTP speculation.