GOAT: SMALL MODELS BEAT GPT4 ON ARITHMETIC

Summary from the TLDR AI newsletter

To be fair, calculators also beat GPT4 on arithmetic. The interesting discovery here is that the llama tokenizer (which splits numbers into individual digits) provides benefits for arithmetic even over other similarly powerful base models fine-tuned in the same task.