OUTSIDE EXPERTS PICK UP THE SLACK ON TESTING OPENAI'S NEWEST MODEL RELEASE

Summary from the TLDR InfoSec newsletter

AI red teaming company SplxAI chose to publish its own testing after OpenAI failed to publish a safety report for GPT-4.1 as it did for 4.0. SplxAI used the same prompts from its 4.0 testing to simulate a financial advisor and evaluate its safety in 11 categories across 1,000 simulated tests. GPT-4.1 performed 3x worse than GPT-4.0. Following OpenAI's guidance that 4.1 may require more explicit prompting, the SplxAI team developed a 1,400-word prompt over 5 hours of iterating that performed better than their initial prompt.