FINE-TUNING OPEN LLM JUDGES TO OUTPERFORM GPT-5.2

Summary from the TLDR AI newsletter

Open-source models like GPT-OSS 120B and Qwen3 235B are fine-tuned using Direct Preference Optimization (DPO) to potentially outperform GPT-5.2 on human preference tasks. RewardBench 2 is used for evaluation, highlighting areas like Math and Safety where these models excel. Cost-efficient, these open models also offer transparency, enabling better alignment with specific use cases while significantly reducing reliance on expensive, closed-source alternatives.