PRACTICAL LLM SECURITY ADVICE FROM THE NVIDIA AI RED TEAM

Summary from the TLDR InfoSec newsletter

The NVIDIA AI Red Team has observed three common vulnerabilities in the implementation of AI systems. The first vulnerability involves directly executing LLM-generated code, which could lead to remote code execution in the case of direct or indirect prompt injection. The second vulnerability is related to insufficient access controls in RAG data sources, which could allow a user to read data they are not privileged to read or write data to the data store. The third vulnerability is related to active content rendering of LLM outputs, which could lead to information leakage via images or other network requests.