CONSTITUTIONAL CLASSIFIERS: DEFENDING AGAINST UNIVERSAL JAILBREAKS

Summary from the TLDR AI newsletter

A new paper from the Anthropic Safeguards Research Team describes a method that defends AI models against universal jailbreaks. A prototype version of the method was robust to thousands of hours of human red teaming for universal jailbreaks, albeit with high over-refusal rates and compute overhead. An updated version achieved similar robustness on synthetic evaluations and did so with a 0.38% increase in refusal rates and moderate additional compute costs.