TEST TIME TRAINING

Summary from the TLDR AI newsletter

One of the most tantalizing phrases in model development is 'new scaling axis'. Every time the industry has found a new scaling axis, it has unlocked a large boost in model effectiveness. The idea of test-time training is an appealing one as it unlocks a new scaling axis. New research has discovered some exciting techniques, but the continual learning problem has still yet to be solved.