EXACT LOOKAHEAD DECODING REDUCES INFERENCE LATENCY 2.3X

Summary from the TLDR AI newsletter

There are many ways to speed up autoregressive language model inference. One way that people are excited about is using a draft model. This requires two models but it can be slower. However, by generating related ngrams from the same model, you can alleviate the need for a draft model and speed up generation linearly.