GENERATIVE VERIFIERS: REWARD MODELING AS NEXT-TOKEN PREDICTION

Summary from the TLDR AI newsletter

Reward models are generally trained as discriminative classifiers. This work from DeepMind uses a language model's logits for yes/no as the reward signal. It found that by allowing a model to use CoT and ensembling, researchers improved performance by 16%.