ENERGY-BASED TRANSFORMERS ARE SCALABLE LEARNERS AND THINKERS

Summary from the TLDR AI newsletter

Energy-Based Transformers replace direct prediction with learned verification functions that assign compatibility scores between inputs and candidate outputs. The architecture is first to out-scale standard Transformers as it allows models to dynamically allocate computation and self-verify predictions without external supervision, achieving up to 35% higher scaling rates.