Where AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden Vulnerabilities

How AI Vulnerability Discovery Harnesses Silently Miss Vulnerabilities

AI vulnerability discovery harnesses can miss real, reachable vulnerabilities because of how the surrounding software is built, and how potentially, this can also be used by an attacker to hide vulnerabilities in plain sight.

This study began while hardening two harnesses I help maintain, Knostic's own OpenAnt, and raptor. We then conducted an exploratory survey by feeding known vulnerabilities to harnesses, analyzed the root causes of the misses, and used the data to run a cross-sectional survey of 10 other harnesses.

The paper aims to share these findings with the community and, hopefully, spark a broader discussion of the engineering challenges and threats involved with harness engineering, in cyber security, and beyond.

A common pattern is that descriptive rules become prescriptive. Assumptions about typical file extensions, function names, entry points, or model responses become filters that determine what gets analyzed. This is similar to work such as from Thomas Ptacek and Timothy Newsham's, " Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection", all the way back in 1998.

Link to paper:
https://arxiv.org/abs/2610.29379

The 13 ways vulnerabilities disappear

1. Unrecognized file types: Executable code is excluded because its extension is missing or unrecognized.
2. Argument count mismatch: Valid calls with optional or variable arguments lose their call-graph edges.
3. Built-in name conflicts: User-defined functions are mistaken for built-ins and pruned as unreachable.
4. Unrecognized entry points: Framework-invoked handlers are classified as unreachable.
5. Overridden findings: Old fixed or false-positive statuses override new findings.
6. Scan limit cutoffs: File-count, priority, or resource limits silently exclude code.
7. Verification filtering: Valid findings are discarded without corroboration or because model responses fail formatting checks.
8. Suppression comments: Markers such as # nosemgrep suppress findings without disclosure.
9. Inactive rules: Documented security queries or analyzers are never executed.
10. Stale caching: Old clean verdicts are reused after relevant changes.
11. Keyword-based skipping: Text matching excludes files such as auth/login.py because their paths contain ignored keywords.
12. Unparsed code blocks: Macro bodies or code excluded from compilation never become analysis units.
13. Diff blindspots: Pull-request scans miss deleted security checks and unchanged code made vulnerable by changes elsewhere.

What the experiments showed

In OpenAnt, renaming a PHP function from header_x() to header() caused its call graph to mistake the user-defined App\header() for PHP's built-in. PHP still executed the vulnerable function, but the missing call-graph edge caused OpenAnt to prune the reachable command injection.

Another experiment placed a reachable Python remote-code-execution vulnerability in an extensionless file under tests/, with an @generated comment. All six harnesses excluded the file before the model saw its code.

The harnesses did not all fail for the same reason. Four excluded the missing extension, one excluded the test directory, and another excluded the generated-code marker.

Why these failures happen

Many come from engineering choices, and errors, used to select code, construct call graphs, and control analysis costs.

This relates to Thomas Ptacek's and Timothy Newsham's 1998 paper, " Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection", on intrusion detection evasion: detectors can miss activity when they interpret input differently from the protected system. Here, the application executes code that the scanner considers absent or unreachable.

Other failures occur after detection, when verification, caching, or merging removes a valid finding. These relate to the fail-safe default: incomplete or unsuccessful analysis should not be reported as clean.

A common pattern is that descriptive rules become prescriptive. Assumptions about typical file extensions, function names, entry points, or model responses become filters that determine what gets analyzed.

Bottom line

Considering false negatives should be an engineering requirement for harnesses. These should preserve unresolved code, disclose exclusions, distinguish incomplete analysis from clean results, and test coverage using behavior-preserving code changes.

These findings likely apply to other LLM harnesses, although we studied only security harnesses.

For security teams: A clean report means no finding was produced. Examine what was actually analyzed.

For harness developers: Test how ordinary engineering choices affect coverage, especially when the target code can influence these choices.

Disclosure: While these aren't security vulnerabilities requiring coordinated disclosure, maintainers were informed before publication. OpenAnt and raptor (which I help maintain) are fully patched. Rivian also informed us they fixed the findings in their own system.