EMBERS OF AUTOREGRESSION FOR O1

Summary from the TLDR AI newsletter

Large language models have limitations due to next token prediction. However, OpenAI's o1 model was trained with a new objective over reasoning traces. Despite this change, o1 still shows signs of limitations due to next token prediction.