vulnhuntr

Description

Vulnhuntr leverages the power of LLMs to automatically create and analyze entire code call chains starting from remote user input and ending at server output for detection of complex, multi-step, security-bypassing vulnerabilities that go far beyond what traditional static code analysis tools are capable of performing. See all the details including the Vulnhuntr output for all the 0-days here: Protect AI Vulnhuntr Blog

Vulnerabilities Found

[!TIP] Found a vulnerability using Vulnhuntr? Submit a report to huntr.com to get $$ and submit a PR to add it to the list below!

[!NOTE] This table is just a sample of the vulnerabilities found so far. We will unredact as responsible disclosure periods end.

Repository	Stars	Vulnerabilities
gpt_academic	67k	LFI, XSS
ComfyUI	66k	XSS
Langflow	46k	RCE, IDOR
FastChat	37k	SSRF
Ragflow	31k	RCE
LLaVA	21k	SSRF
gpt-researcher	17k	AFO
Letta	14k	AFO

Limitations

• Only Python codebases are supported.
• Can only identify the following vulnerability classes:
  • Local file include (LFI)
  • Arbitrary file overwrite (AFO)
  • Remote code execution (RCE)
  • Cross site scripting (XSS)
  • SQL Injection (SQLI)
  • Server side request forgery (SSRF)
  • Insecure Direct Object Reference (IDOR)

Installation

[!IMPORTANT] Vulnhuntr strictly requires Python 3.10 because of a number of bugs in Jedi which it uses to parse Python code. It will not work reliably if installed with any other versions of Python.

We recommend using pipx or Docker to easily install and run Vulnhuntr.

Using Docker:

docker build -t vulnhuntr https://github.com/protectai/vulnhuntr.git#main

Using pipx:

pipx install git+https://github.com/protectai/vulnhuntr.git --python python3.10

Alternatively you can install directly from source using poetry:

git clone https://github.com/protectai/vulnhuntr
cd vulnhuntr && poetry install

Usage

This tool is designed to analyze a GitHub repository for potential remotely exploitable vulnerabilities. The tool requires an API key and the local path to a GitHub repository. You may also optionally specify a custom endpoint for the LLM service.

[!CAUTION] Always set spending limits or closely monitor costs with the LLM provider you use. This tool has the potential to rack up hefty bills as it tries to fit as much code in the LLMs context window as possible.

[!TIP] We recommend using Claude for the LLM. Through testing we have had better results with it over GPT.

Command Line Interface

usage: vulnhuntr [-h] -r ROOT [-a ANALYZE] [-l {claude,gpt,ollama}] [-v]

Analyze a GitHub project for vulnerabilities. Export your ANTHROPIC_API_KEY/OPENAI_API_KEY before running.

options:
  -h, --help            show this help message and exit
  -r ROOT, --root ROOT  Path to the root directory of the project
  -a ANALYZE, --analyze ANALYZE
                        Specific path or file within the project to analyze
  -l {claude,gpt,ollama}, --llm {claude,gpt,ollama}
                        LLM client to use (default: claude)
  -v, --verbosity       Increase output verbosity (-v for INFO, -vv for DEBUG)

Examples

From a pipx install, analyze the entire repository using Claude:

export ANTHROPIC_API_KEY="sk-1234"
vulnhuntr -r /path/to/target/repo/

[!TIP] We recommend giving Vulnhuntr specific files that handle remote user input and scan them individually.

From a pipx install, analyze the /path/to/target/repo/server.py file using GPT-4o. Can also specify a subdirectory instead of a file:

export OPENAI_API_KEY="sk-1234"
vulnhuntr -r /path/to/target/repo/ -a server.py -l gpt 

From a docker installation, run using Claude and a custom endpoint to analyze /local/path/to/target/repo/repo-subfolder/target-file.py:

docker run --rm -e ANTHROPIC_API_KEY=sk-1234 -e ANTHROPIC_BASE_URL=https://localhost:1234/api -v /local/path/to/target/repo:/repo vulnhuntr:latest -r /repo -a repo-subfolder/target-file.py

Experimental

Ollama is included as an option, however we haven't had success with the open source models structuring their output correctly.

export OLLAMA_BASE_URL=http://localhost:11434/api/generate
export OLLAMA_MODEL=llama3.2
vulnhuntr -r /path/to/target/repo/ -a server.py -l ollama

Logic Flow

• LLM summarizes the README and includes this in the system prompt
• LLM does initial analysis on an entire file and reports any potential vulnerabilities
• Vulnhuntr then gives the LLM a vulnerability-specific prompt for secondary analysis
• Each time the LLM analyzes the code, it requests additional context functions/classes/variables from other files in the project
• It continues doing this until the entire call chain from user input to server processing is complete then gives a final analysis
• The final analysis consists of it's reasoning, a proof of concept exploit, and a confidence score

[!TIP] Generally confidence scores < 7 mean there's unlikely a vulnerability. Confidence scores of 7 mean it should be investigated, and confidence scores of 8+ mean it is very likely to be a valid vulnerability.

Output

The tool generates a detailed report of the vulnerabilities found in the analyzed files. The report includes:

• Initial assessment results for each file.
• Secondary assessment results with context functions and class references.
• Confidence scores for vulnerabilities found.
• Logs of the analysis process.
• PoC exploit

Below is an example of a Vulnhuntr report describing a 0-day remote code execution vulnerability in Ragflow (now fixed):