A SMALL NUMBER OF SAMPLES CAN POISON LLMS OF ANY SIZE

Summary from the TLDR InfoSec newsletter

Anthropic has found that large language models (LLMs) like Claude can be compromised by injecting as few as 250 malicious files into their training data, regardless of the model's size or the total dataset volume. Attackers can create a “backdoor” in the model triggered by specific phrases, causing the model to behave unpredictably or output gibberish. This finding overturns the traditional belief that an attacker needs control of a percentage of training data: even a tiny, fixed number of documents might be enough.