
Taxonomy-guided adaptive red teaming framework that sweeps prompt-injection techniques and refines failed attempts to find diverse attacks against tool-using LLM agents.
Code for the paper
AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents
AutoPI is an adaptive red teamer for tool-using LLM agents. It is built to find many different ways to break an agent, not just to break it often. For each scenario, an attacker LLM writes an indirect prompt injection for every sub-technique of a prompt-injection taxonomy, then reworks the ones that fail using feedback from an LLM judge.
Working injections are rare and are conjectured to lie in regions, one per technique. AutoPI searches in three phases, which a strategy can skip, resize, repeat or reorder:
Install AutoPI with the bundled AgentDojo. Python 3.11 is recommended.
git clone https://github.com/stevendamianakis/autopi
cd autopi
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e . -e benchmarks/agentdojo
Point the target and the generator at an OpenAI-compatible server, such as vLLM on port 8000, and run the Full strategy on all 949 AgentDojo scenarios:
autopi run --benchmark agentdojo \
--target vllm:8000:MODEL_ID \
--refiner vllm:8000:MODEL_ID \
--strategy configs/strategies/full.yaml \
--logdir runs/agentdojo-full
Add --dry-run to check the settings and list the scenarios without calling a model. Each run writes
its resolved settings and every attempt of every scenario to the --logdir folder. In the code, the
generator is called the refiner.
Name each model once in a copy of configs/models.example.yaml, then pass the names. API keys are read
from environment variables and never stored in the file.
cp configs/models.example.yaml configs/models.local.yaml
autopi run --benchmark agentdojo --models-config configs/models.local.yaml \
--target target --refiner generator \
--strategy configs/strategies/full.yaml --logdir runs/agentdojo-full
Providers are openai-compatible (any URL), openai, anthropic, google and ollama, and short
specs such as anthropic:MODEL_ID work without a file. Model configuration covers
decoding settings, thinking modes and the recovery of text that a server-side tool parser removed.
pip install -e benchmarks/ipi_arena
# the 8 browser scenarios also need:
pip install -e 'benchmarks/ipi_arena[browser]' && python -m playwright install chromium
autopi run --benchmark ipiarena --models-config configs/models.local.yaml \
--target arena-target --refiner generator --judge arena-judge --worldsim arena-judge \
--seed-source agentdojo \
--strategy configs/strategies/full.yaml --logdir runs/ipiarena-full
IPI Arena has 41 scenarios: 18 tool use, 15 coding and 8 browser. A scenario counts as broken only when
every one of its criteria passes, and --ipi-ignore-concealment selects an action-only metric instead.
The target uses native tool calls, and browser scenarios need an image-capable target. The benchmark
ships no seed, so use --seed-source agentdojo, a --seed-file, or a strategy without a seed block.
--mode 3, the default and the paper's setting, shows the generator the user task and the injection
goal. Modes 1 and 2 write goal-blind templates, and mode 2 also sees the user task.--no-judge, --no-context and --no-legitimize switch off one of the three adaptive mechanisms.--seed-file uses another seed, and --no-seed opens with the sweep.--taxonomy-path sweeps another taxonomy with the same file layout.--no-early-stop keeps going after the first success, for coverage runs.--shard and --num-shards split a run across workers, and --resume continues an interrupted one.autopi run --help lists every option.
A strategy is an ordered list of blocks, an attempt budget per scenario and a stop rule. The three files
in configs/strategies/ are the paper's AutoPI-Full, AutoPI-Seed-Exploit and AutoPI-L1. They are
examples, not limits:
| File | Blocks | Attempts | Goal |
|---|---|---|---|
full.yaml | seed, L1 sweep, 3 variants of each failed line | 529 | coverage |
seed-exploit.yaml | seed, 600 variants of the seed | 601 | ASR with a strong seed |
l1.yaml | seed, L1 sweep | 133 | low cost |
# configs/strategies/full.yaml
version: 1
budget: 529
stop: first_success
blocks:
- kind: seed
- kind: sweep
id: exploration
level: 1
order: priority
- kind: sweep_exploit
source: exploration
reps: 3