TEXT DIFFUSION MODELS ARE FASTER AT WRITING CODE

Summary from the TLDR AI newsletter

Diffusion language models generate code at a faster rate than large language models. Increased structure tends to correlate with reduced entropy, which leads to higher confidence token predictions, which directly means more tokens decoded in parallel per step. Tests suggest that it really is the structuredness of the output, not memorization, that matters.