exfil-scan

Scan LLM outputs and AI-generated content for data exfiltration signals before they reach users, logs, or downstream systems.

1) Why exfil-scan exists

Modern LLM applications routinely process sensitive prompts, internal documents, API responses, and user data. That creates a new security boundary: the output layer.

The EchoLeak family of attacks (including CVE-2025-32711) demonstrated how hidden or obfuscated payloads can be embedded in generated text and used to exfiltrate data through tools, links, or parsing chains.

Traditional scanners focus on files, dependencies, and runtime traffic. They do not inspect generated model output in a purpose-built way.

exfil-scan provides a "virus scanner for LLM responses":

• inspects plain text, JSON, markdown, and logs
• identifies six classes of suspicious exfiltration patterns
• reports findings with severity and location
• integrates cleanly into CI/CD and local developer workflows

2) What it detects

3) Feature highlights

• Three sensitivity profiles: low, medium, high
• YAML configuration override support
• Input from stdin, single file, or directory
• Output in text, json, markdown, or html
• Stable exit codes for automation
• Clean Python API for embedding into other tools

4) Installation

Install from source

git clone https://github.com/your-org/exfil-scan.git
cd exfil-scan
pip install -e .

Standard pip install (when published)

pip install exfil-scan

Development install

pip install -e ".[dev]"
pytest

5) Quick start

Scan piped LLM output

echo 'api_key: sk-ABCDEFGHIJKLMNOPQRSTUVWXYZ123456' | exfil-scan

Scan a single file

exfil-scan --input output.txt

Scan a directory recursively

exfil-scan --directory ./model-logs --recursive

6) CLI options reference

Notes:

• Use either --input or --directory, not both.
• If no path is provided, the tool reads from stdin.
• If findings exist, the process exits with code 1.

7) Usage examples by output format

Plain text output

exfil-scan --input suspicious.txt --format text

Expected style:

exfil-scan report
=================
Scanned targets: 1
Total findings: 2

1. [HIGH] metadata_leaks/known_token_format:openai_key
   Location: suspicious.txt:1:10
   Description: Matched known credential/token format (openai_key).

JSON output

exfil-scan --input suspicious.txt --format json

Expected shape:

{
  "finding_count": 2,
  "scanned_targets": ["suspicious.txt"],
  "findings": [
    {
      "category": "metadata_leaks",
      "rule": "known_token_format:openai_key",
      "severity": "HIGH",
      "description": "Matched known credential/token format (openai_key).",
      "location": "suspicious.txt:1:10",
      "snippet": "..."
    }
  ]
}

HTML output

exfil-scan --directory ./outputs --recursive --format html --output report.html

Expected behavior:

• Generates a standalone HTML page with findings table
• Escapes content for safe rendering
• Includes severity, category, rule, location, description

Markdown output

exfil-scan --input suspicious.txt --format markdown --output report.md

Expected style:

# exfil-scan Report

- Scanned targets: 1
- Total findings: 2

8) Configuration and sensitivity

The repository ships with default-config.yaml. You can pass your own file with --config.

Sensitivity tuning:

• low: fewer false positives, larger thresholds
• medium: balanced defaults
• high: catches more subtle patterns

CLI sensitivity override:

exfil-scan --input out.txt --sensitivity high

9) Custom patterns and policy tuning

You can add your own token formats and metadata keys:

sensitivity: medium
rules:
  metadata_keys:
    - api_key
    - db_password
    - internal_token
  token_patterns:
    custom_jwt: "\\beyJ[A-Za-z0-9_\\-]{10,}\\.[A-Za-z0-9_\\-]{10,}\\.[A-Za-z0-9_\\-]{10,}\\b"
    corp_secret: "\\bcorp_[A-Za-z0-9]{24,}\\b"

Then run:

exfil-scan --input response.log --config ./my-config.yaml

10) Architecture overview

Module responsibilities:

• exfil_scan/cli.py: argument parsing, input selection, output rendering, exit handling
• exfil_scan/config.py: defaults, YAML loading, recursive merge, profile normalization
• exfil_scan/scanner.py: six detection engines, finding dataclasses, report formatting
• tests/test_scan.py: detection and output-format regression tests

Execution flow:

1. CLI parses arguments.
2. Config loads defaults + optional overrides.
3. Scanner reads text from stdin/file/directory.
4. Each enabled category runs independently.
5. Findings are severity-sorted and rendered.
6. Exit code communicates pass/fail/error for automation.

11) Exit codes

This makes CI integration straightforward:

exfil-scan --directory ./artifacts --recursive --format json --output exfil-report.json

If any detection is found, your job can fail fast based on exit code 1.

12) Testing

Run tests:

pytest

Coverage includes:

• all six detection categories
• report formatting in all four output modes
• rule-level assertions for stable behavior

13) Security notes

exfil-scan is a heuristic scanner. It is designed to reduce risk and surface suspicious output, not prove content safety.

Recommended deployment pattern:

1. run scanner in CI on test prompts and fixture outputs
2. run scanner in production guardrails on generated responses
3. route findings to analyst triage and telemetry
4. continuously tune config thresholds based on false-positive review

14) License

This project is licensed under the MIT License. See LICENSE for full terms.

15) Contributing

Contributions are welcome. When opening a PR, include:

• clear reproduction steps
• expected and actual behavior
• tests for new detection logic
• documentation updates when introducing new rules