ARE WE IN A GPT-4-STYLE LEAP THAT EVALS CAN'T SEE?

Summary from the TLDR AI newsletter

Chat is a terrible way to evaluate models. GPT-4 was so significantly better at answering questions that it was an obvious step forward. The industry needs to add more types of benchmarking. Models are currently being evaluated like students in an exam, but that's not representative of how the world works.