valtik-markdown-exfil-tester

Black-box tester for indirect prompt injection leading to markdown / HTML exfiltration in LLM-backed chatbots. This is the vulnerability class that hit Microsoft 365 Copilot (CVE-2025-32711, aka EchoLeak), ChatGPT (OpenAI Feb 2026 patch), and Salesforce (ForcedLeak). The pattern:

1. Attacker plants markdown or HTML in content the LLM will read (docs, support tickets, PRs, ingested web pages, RAG sources).
2. The LLM emits markdown like ![](https://assets.kitploit.com/production/public/readmes/placeholders/f0fc86cfe65f76d40e15aaec61704ec8220a56dc89d4be03c46f67cb31b9fa8c.svg) in its response.
3. The chatbot frontend renders the markdown, the browser fetches the attacker URL, and secrets leak via URL parameter on every render.

Existing tools (Garak, Augustus, Promptfoo) are great at text-layer injection testing but do not close the render-side oracle — "did your frontend actually fetch the attacker URL?" That's the real vulnerability. This tool fills that gap.

npx valtik-markdown-exfil-tester https://your-chatbot.example.com

Authorization required

Do not run this against chatbots you do not own or have explicit written permission to test. Running this tool against a third-party LLM chatbot without authorization is very likely a violation of the service's terms of use and may be illegal in your jurisdiction (CFAA in the US, Computer Misuse Act in the UK, similar laws in the EU).

Legitimate use cases:

• Pentesting a chatbot your organization owns.
• Testing a chatbot in the scope of an authorized engagement (written SoW).
• Testing a target on a bug bounty program whose policy explicitly permits this kind of test (read the policy first — many LLM programs restrict heavy prompt-injection fuzzing).
• Research against your own locally-hosted chatbot stack.

Valtik Studios LLC disclaims all liability for unauthorized use.

How it works

1. Launches a headless browser (Playwright / chromium).
2. Spins up a local HTTP sink listener bound to 127.0.0.1 on a random high port. Every incoming request is logged with timestamp, path, query, headers, Referer, IP, User-Agent.
3. Delivers a rotating payload set to the chatbot. Each payload is designed to coax the LLM into emitting a markdown image, reference link, raw ``, <svg>, CSS background, <link rel=prefetch>, etc. that points at the local sink.
4. Renders the chatbot's response in the headless browser with a permissive markdown-to-HTML pass (mimicking a vulnerable frontend).
5. Correlates: did the sink receive a fetch tagged with this payload's id? Did the browser try and fail (CSP / sanitizer)? Did the LLM emit the URL in raw text but the frontend ignore it? Did the LLM refuse?
6. Classifies:
   • confirmed — sink received a hit. Critical. Exfil works.
   • browser-tried — browser issued the fetch but it failed (CSP, referrer policy, sanitizer). Medium. Defense held but brittle — do not rely on CSP alone.
   • llm-emitted — LLM reply contained the sink URL but the frontend did not render it as a fetch. Low.
   • refused — LLM reply did not contain the payload. Defense held.
   • inconclusive — no LLM reply.

Install

Run without installing:

npx valtik-markdown-exfil-tester https://chat.example.com

Or install globally:

npm install -g valtik-markdown-exfil-tester
npx playwright install chromium
markdown-exfil-tester https://chat.example.com

Requires Node 20+.

Usage

Usage: valtik-markdown-exfil-tester <chatbot-url> [options]

Arguments:
  chatbot-url        URL of the chatbot endpoint or page

Options:
  --mode <type>      direct | indirect-doc        (default: direct)
  --endpoint <url>   API endpoint if different from chatbot-url
  --sink-port <n>    Local sink port              (default: random high port)
  --payloads <path>  Override payload library JSON
  --browser <bin>    playwright | none            (default: auto-detect)
  --auth <cookie>    Auth cookie for the chatbot, if needed
  --timeout <s>      Per-payload wait             (default 30)
  --max-payloads <n> Stop after N payloads        (default: all)
  --rate-limit-ms <n> Delay between payloads     (default 3000)
  --out-dir <dir>    Working dir for indirect-doc (default: /tmp/...)
  --user-agent <ua>
  --json             Machine-readable output
  --fail-on <level>  Exit non-zero on severity >= level (critical|high|medium|low)
  -v, --version
  -h, --help

Modes

• direct (v0.1) — POSTs the payload straight to the chatbot endpoint. Tests direct prompt injection. Weaker-signal than indirect, but fast.
• indirect-doc (v0.1) — writes each payload as a standalone markdown document to --out-dir. You upload each file to the chatbot's knowledge base / RAG source / support-ticket system / wiki, then query the chatbot. The tool waits --timeout seconds per payload for a sink hit tagged with that payload's id.
• indirect-pr (v0.2, not yet implemented) — automates GitHub PR body delivery against Copilot-style PR reviewers.
• attach (v0.2, not yet implemented) — uploads the file via the chatbot's upload flow.

Examples

# Fast test with first 10 payloads
npx valtik-markdown-exfil-tester https://chat.example.com --max-payloads 10

# JSON output for CI
npx valtik-markdown-exfil-tester https://chat.example.com --json | jq .findings

# Fail the CI job on any high-severity finding
npx valtik-markdown-exfil-tester https://chat.example.com --fail-on high

# Indirect-doc mode: generate docs, upload them manually, then re-run
npx valtik-markdown-exfil-tester https://chat.example.com \
  --mode indirect-doc --out-dir ./payload-docs --timeout 60

Payload library

Ships with 30+ templates covering:

Technique	Example id	Notes
Markdown image	md-image-basic	Classic  exfil
Markdown image	md-image-alt-prompt	Alt text masquerades as instruction
Markdown image	md-image-empty-alt	Empty alt, slips past alt heuristics
Reference-style	md-ref-image	[x][id] + trailing [id]: url
Raw HTML	raw-img-tag	`` in markdown
Raw HTML	raw-img-srcset	`` sanitizer bypass
SVG	svg-image-href	<svg><image href>
SVG	svg-image-xlink	Legacy xlink:href
Iframe / Object	iframe-src, object-data	Usually blocked, still tested
CSS	style-tag-bg, style-attr-bg	url() in <style>
CSS	style-list-image	list-style-image trick
CSS	style-border-image	border-image niche vector
<link>	link-prefetch, link-preload, link-dns-prefetch	Rarely sanitized
<meta>	meta-refresh	Whole-page redirect
<script>	script-location, script-fetch	CSP baseline test
Obfuscation	base64-subdomain	Blocklist bypass
Obfuscation	homoglyph-url	Cyrillic lookalike path
Obfuscation	punycode-hint	Punycode-ish path segment
Redirect chain	open-redirect-chain	Assumes target-origin allow-list
Anchor	mailto-body, anchor-ping	Pure anchor
Media	video-poster, audio-src, source-src	<video controls>/<audio>/<picture>
Form	input-formaction	<input type=image>
Social	prompt-wrapped-image, prompt-wrapped-citation	Coaxing language around the payload