HOW FEEDBACK AFFECTS LLMS?

Summary from the TLDR AI newsletter

When training large language models to understand and match human values, do we use scores (rating) or comparisons (ranking) as feedback? This study found that the choice matters a lot. For instance, models trained with rankings are preferred when evaluated by comparison but not always by scoring. This shows that how we choose to give feedback has big implications for how well the model works in real-life situations.