SLEEPER AGENTS: TRAINING DECEPTIVE LLMS THAT PERSIST THROUGH SAFETY TRAINING

Summary from the TLDR AI newsletter

Anthropic released some really interesting research that purposefully poisons a language model with a sleeper phrase that elicits some behavior. It found that it was unable to “align” this sort of model with the powerful system used for its production models. Meaning, once the model had been poisoned, bad behavior could not be removed with current tools.