META'S SET BLOCK DECODING

Summary from the TLDR AI newsletter

Set Block Decoding enables parallel sampling of multiple future tokens by combining next-token and masked token prediction. This reduces decoding passes up to 5x without sacrificing accuracy.