HOW LLM-BASED PENETRATION TESTING HARNESSES WORK

Summary from the TLDR InfoSec newsletter

Utku Sen tested three open-source LLM penetration-testing harnesses - Cairn, Artex, and Strix - against a deliberately vulnerable e-commerce site containing DOM XSS, unpaid purchases, IDOR, and account takeover. A harness turns model output into real actions: reading files, running commands, browsing, and using tools such as Nmap, Nuclei, Katana, and Playwright. Cairn pursued single goals through planner and worker agents. Artex, linked by CrowdStrike to attacks on South Korean finance targets, used its own agent library and found all vulnerabilities except account takeover. Strix added reconnaissance, defined vulnerability classes, and produced a structured report. Sen found that a Pi harness using DeepSeek with a simple goal prompt found the account-takeover flaw and outperformed Cairn.