How to Validate an AI-Generated Vulnerability

AI-assisted discovery produces volume. Research produces a validated path. This series — False Positive, Reachability, Exploitability, Patch Verification — is how the laboratory keeps the two apart.

An unverified model report is not a finding. We will not post “the model found CVE-shaped text” as work.

The object of this note is not a CWE. It is a candidate: a model- or scanner-emitted claim that some function is unsafe. Most candidates are wrong — wrong sink, dead code, already patched, or not reachable from untrusted input.

Any codebase you are allowed to analyze: your lab targets, open-source trees under a research policy, and vendor programs that permit review. Do not run speculative analysis against systems you do not have authorization to test.

The candidate names a function. The surface question is whether untrusted input can still reach it in a build you can run. If the answer is unknown, the candidate stays in the notebook.

A valid finding still needs a failed assumption in human language. If the model only said “possible injection,” you do not have a root cause.

This is the validation pipeline for the rest of the series.

Models over-report RCE. We only assign that impact after a sink that executes is both present and reachable. Until then the candidate is “unvalidated execution claim.”

If a vendor already fixed the function, the candidate is historical. Confirm on the fixed tag. Patch verification is part of this series (#04), not an afterthought.

Detect bad research practice: publishing model output, skipping reachability, or copying a CWE from the prompt. Internal metric: candidates discarded versus notes that survive review.

For teams using AI in review: require a human-written invariant and a lab note before any ticket is marked confirmed. For this journal: no article ships without the twelve-section spine.

AI + vulnerability research is a brand only if validation is stricter than the generator. The next notes in this series will take one stage each: false positive, reachability, exploitability, patch verification.

GitHub Security Lab notes on verification — discovery is not complete without it.

CISA KEV — a reminder that real-world exploitation, not model confidence, sets priority.

Art of Vector Security disclosure policy — /about/responsible-disclosure.