Doc-foundry – AI writes Security detection rules, then proves they work

DOC-foundry turns a supplied threat - such as a vulnerability, incident report, or ATT&CK technique - into a first-version Sigma detection rule using specialized AI roles: threat analyst, detection engineer, telemetry analyst, and red-team adversary. Its proof pipeline has Blue draft and revise the rule while Red tests it against two fixed, documented attacks. A deterministic engine, rather than the agents, evaluates the rule against attack examples, benign look-alikes, and adversarial mutations; it calculates false-positive and false-negative counts, assigns a coverage label (real-coverage, brittle, or broken), and produces a reproducibility hash. The deliverables are the rule, a threat report showing how it changed, and a proof bundle. “Real-coverage” means the rule catches the defined corpus and both fixed attacks without firing on the benign baseline - not that it defeats every future evasion.

A separate review pipeline focuses on stakeholder feedback, not proof: Red identifies evasion and blind spots, Blue revises, and a SOC lead assesses alert volume, triage, severity, and tuning before the final rule is packaged with its review history. A Joomla vulnerability example shows the competing pressures: Red urged broader detection to catch alternate shells and evasion, while the SOC lead flagged benign alerts and operational noise. The final revision tightened parent-process requirements, excluded normal interpreter behavior, added triage details, and lowered severity. This chain does not execute rules or calculate coverage, so its output is reviewed, not validated. The project also offers a live web interface and offline fixtures; live authoring requires a supported model API key.