VulnFanatic-NG

LLM-assisted vulnerability research for Binary Ninja.

VulnFanatic-NG adds a sidebar panel that scans the current binary and asks an LLM — a locally hosted OpenAI-compatible model by default, or Anthropic Claude, Google Gemini, or Azure OpenAI (see LLM backends) — to judge whether suspicious code is actually vulnerable. It works primarily from Binary Ninja's decompiler output (HLIL), falling back to assembly when needed, and reports only confirmed issues with clickable references back to the code.

---

How it works

A scan runs in up to three phases (Phase 3 is opt-in and online-only):

Phase 1 — dangerous function calls

Finds call sites of dangerous functions defined in rules/phase1_rules.json — strcpy, memcpy, sprintf/format strings, system, alloca, scanf, command/exec APIs, weak RNG, the free/delete family (use-after-free / double-free), reads of untrusted input into fixed buffers (recv/read/fread/ReadFile), SQL injection (sqlite3_exec/mysql_query/PQexec), disabled TLS certificate verification (SSL_CTX_set_verify/curl), SSRF, and improper privilege management (setuid/setresgid), the memset/bzero family, and comparisons with an attacker-controlled length (memcmp/strncmp → authentication bypass), across C/C++, Win32, and (best-effort) Rust FFI. Coverage includes fortified _chk (FORTIFY) and Annex-K _s variants. Bounded formatted-output functions (snprintf and variants) have their own default-safe rule so a correct size argument is not reported as an overflow. Call sites are found three ways: direct calls to the named symbols; calls routed through forwarding thunks / PLT stubs (the real callers are recovered, so an import reached only through a stub is not missed); and — unless vulnfanatic.scanIndirectCalls is off — indirect calls dispatched through a function pointer or vtable that Binary Ninja resolved to a dangerous function. For each call site it builds an interprocedural, decompiler-centric context, budgeted to a token limit (default 100k):

• the call expression and its arguments,
• the declared prototype of the called function (from Binary Ninja's type info, else a built-in table) so the model maps arguments to parameters correctly — fortified __*_chk and bounds-checked *_s variants take extra leading arguments, shifting the format/size/destination position,
• the type and byte size of each call argument (buffer capacities), derived from the argument's HLIL expression type so that a struct field like s->buf resolves to the field's real array size rather than the pointer size of s; struct definitions in the types section also carry per-field byte sizes,
• the concrete value / range of each argument resolved by Binary Ninja's constant-propagation and value-set analysis (e.g. a length proven constant 0x40 or bounded to [0, 0xff]), which the model uses as ground truth when comparing a size against a buffer capacity instead of guessing,
• the stack frame layout of the calling function (variable offsets and byte sizes) when it contains a fixed-size buffer, so a stack overflow can be judged against the adjacent variables and the saved return address (vulnfanatic.includeStackLayout),
• the path constraints (the if/loop/switch conditions guarding the call),
• an argument data-flow summary — where each call argument is defined and used within the function,
• parameter resolution across callers — when a dangerous argument is a parameter of the calling function, the context reports what every caller actually passes for it (e.g. "all callers pass a string literal"), so a format/ size parameter that is always constant isn't mistaken for attacker-controlled,
• the full decompiled body of the calling function,
• data-type definitions (struct/union/enum) for the types referenced across the call chain and the argument variables, so the model knows real buffer/field sizes and integer widths,
• the decompiled bodies of functions that produce or consume the call's argument variables (traced via HLIL def/use), which is what makes use-after-free / double-free and tainted-size reasoning possible,
• call paths from entry points / exported functions down to the call,
• the decompiled body of every function along those call paths (nearest the dangerous call first), each annotated with the call site and the conditions guarding the next hop,
• the bodies of other functions those path functions call (e.g. for MAIN→ABCD→strcpy, also the functions MAIN and ABCD call elsewhere), since they may hold the bounds/validation checks that gate the dangerous value (vulnfanatic.includeCallPathSiblings, filled while budget allows), and
• tainted-source hints (input functions like recv/read/getenv called in the same function).

That context plus a rule-specific prompt is sent to the model, which returns a structured verdict. Non-issues are dropped. The prompts are tuned for a strong local code model (e.g. Qwen2.5-Coder) and instruct it to analyze the entire flow and emit JSON only.

Reasoning, scratchpad, and confidence

The model is told to favor recall — report plausible, security-relevant issues and express uncertainty through a Confidence rather than dropping anything it can't fully prove. It shows its work in a scratchpad that quotes the verbatim code snippets it relied on (the input source, each guard, the size/length, the relevant type, and the sink), which is stored on the finding so you can audit the reasoning.

Each finding carries a Confidence (high/medium/low): high = the whole chain is shown in the context; medium = likely, with a link or two inferred; low = a lead worth manual review. This is the headline metric (the model's severity estimate is a secondary field). Set vulnfanatic.minConfidence to drop anything below a threshold.

Tuning precision vs. recall

By default VulnFanatic-NG favors recall (catching real issues). If you get too many false positives, tighten with any of:

• vulnfanatic.validationPass (default off) — runs a second LLM pass that double-checks each flagged issue against the same context (verifying the scratchpad snippets and re-tracing the flow) and can correct the verdict or confidence. Doubles LLM calls for flagged candidates.
  • Separate validator model (recommended when you use the validation pass). Set vulnfanatic.validatorModel (plus validatorProvider / validatorBaseUrl / validatorApiKey) to run the second pass on a different model. A second opinion is far more useful from an independent model — it shares fewer blind spots and is much less likely to rubber-stamp the first verdict (models tend to prefer their own answers). A good pattern is a cascade: a fast model as the analyst (broad recall) and your strongest model as the validator, which only runs on flagged candidates. Leave the validator model blank to validate with the analyst model. The validator should be at least as capable as the analyst — a weaker one mostly adds false rejections. Everything except provider/base URL/key/ model is inherited from the analyst connection settings; a blank validator key reuses the analyst key; and if the validator endpoint is unreachable the first verdict is kept (the finding is never lost to a validator outage).
• vulnfanatic.minConfidence (default low) — raise to medium/high to report only stronger findings.
• vulnfanatic.skipConstantArgCalls (default off) — skip overflow-class call sites whose arguments are all compile-time constants.