HOW WE ESTIMATE THE RISK FROM PROMPT INJECTION ATTACKS ON AI SYSTEMS

Summary from the TLDR InfoSec newsletter

Google DeepMind's AI security Team has developed an evaluation framework to test an AI system's susceptibility to indirect prompt injection attacks. The framework employs a hypothetical scenario where an AI agent receives a fictitious email conversation. It uses three prompt injection methods: actor-critic, where a model generates and refines injections based on success probability; beam search; and Tree of Attacks with Pruning, which aims to make the AI violate safety protocols.