HOW WE BUILT AN APPSEC AI THAT SECURITY RESEARCHERS AGREE WITH 96% OF THE TIME

Summary from the TLDR InfoSec newsletter

Semgrep built Semgrep Assistant's auto triage function to automatically classify a finding as a true or false positive by sending a minimal amount of code to an LLM. To benchmark the model, Semgrep's research team labeled 2,000 findings and then the system was evaluated against it. The original Assistant system was able to score a 90% success rate, but through optimizations around models, prompting, and context the team managed to increase the true positive rate to 96%.