TLDR AI · DEEP DIVES & ANALYSIS
SPECULATIVE DECODING IN LLMS
Summary from the TLDR AI newsletter
Perplexity applies speculative decoding to speed up its Sonar models, using lightweight draft models to propose multiple tokens verified by larger LLMs.
TLDR AI · DEEP DIVES & ANALYSIS
Perplexity applies speculative decoding to speed up its Sonar models, using lightweight draft models to propose multiple tokens verified by larger LLMs.