ANTHROPIC RESEARCHERS WEAR DOWN AI ETHICS WITH REPEATED QUESTIONS

Summary from the TLDR InfoSec newsletter

Researchers at Anthropic have devised a new technique to convince AI models to answer questions that they aren’t meant to. The method involves exploiting larger context windows to prime the LLM with a series of questions (that it won’t answer due to filters) and answers as if the LLM answered them. Over time, the LLM will get “confused” and think it answered the questions and will answer inappropriate questions.