TLDR AI · MISCELLANEOUS
HOW BENCHMAXXED IS GPT-OSS-120B?
Summary from the TLDR AI newsletter
An analysis of currently available cutting-edge open source models found that gpt-oss-120b scored the worst on LiveBench, a private benchmark that uses secret questions. While the model looked good on release day, a bunch of critical reports have since been released. This isn't conclusive evidence that OpenAI is gaming benchmarks, but it brings up the question of whether it is worth creating meta-benchmarks that try to measure overfitting in more robust ways.