FINDING GPT-4'S MISTAKES WITH GPT-4

Summary from the TLDR Web Dev newsletter

OpenAI developed CriticGPT, a model based on GPT-4, to identify errors in ChatGPT's code output, and found it improves human trainers' ability to spot mistakes during the reinforcement learning process. CriticGPT was trained using RLHF on ChatGPT answers containing errors. It can generate longer, more comprehensive critiques by utilizing additional test-time search against the critique reward model.