LLM-SQUARED

Summary from the TLDR AI newsletter

Sakana AI has employed an evolutionary strategy to find a preference optimization scheme that outperforms DPO. It used a language model to suggest code and then trained models based on that code. After 100 or so generations, it had several proposed variants with extremely high performance.