MASKED DIFFUSION MODELS FOR LANGUAGE MODELING

Summary from the TLDR AI newsletter

Researchers have demonstrated that masked discrete diffusion models can rival autoregressive methods in language modeling by applying an effective training recipe and introducing a simplified Rao-Blackwellized objective.