ALIGN A LANGUAGE MODEL WITHOUT TRAINING

Summary from the TLDR AI newsletter

Alignment helps make language models more helpful and harmless. It can sometimes hurt performance, but in general is a positive. Alignment is expensive and requires substantial alignment data. However, if you allow the model to rewind during generation after evaluating its own output, it can improve alignment performance on the frozen model by up to 81%.