LLM DECONTAMINAOR

Summary from the TLDR AI newsletter

When new models come out with astounding capabilities, it is natural to question whether they actually trained on the validation data or on benchmarks. Decontamination is the process of removing such testing data from the input. The LMSYS group found that if you rephrase test data in a way that passes decontamination but still contains information about the benchmark, you can fake your way to amazing performance.