Skip to content
KitploitKITPLOIT
ToolsExploitsBlog
Log in
Submit
ToolsExploitsBlog
Submit

Hacking, PenTest, and Cybersecurity Tools for Your Security Arsenal!

Kitploit is a directory of hacking, cybersecurity, and pentesting tools. Discover the latest project updates to find vulnerabilities, analyze systems, automate testing, and strengthen your security.

FeedsContactPrivacy© 2026 Kitploit

Tool Directory

Categories

View all categories
Loading categories
autopi — Taxonomy-guided adaptive red teaming framework that sweeps prompt-injection techniques and refines failed attempts to find diverse attacks against tool-using LLM agents. | Kitploit
Tools/GitHubGitHub/stevendamianakis/autopi
Penetration Testing FrameworksVulnerability AnalysisExploitationMachine LearningPapers & ResearchRed TeamingAI SecurityAdversarial Attack
GitHubstevendamianakis/autopi

autopi

Taxonomy-guided adaptive red teaming framework that sweeps prompt-injection techniques and refines failed attempts to find diverse attacks against tool-using LLM agents.

11 day agoNot yet reviewed
View Repository

Most Popular

View all →

Discover the most used tools by our community.

  • 1

    OsintHunter

    letox4/osinthunter

    🎯 The ultimate OSINT toolkit: investigation graph, confidence scoring, alias families, map, timeline and reports. 49 modules, 400+ sites, no API keys required.

    960
  • 2

    ghostlock-app

    yukonga/ghostlock-app

    GhostLock One-Tap Execution App (CVE-2026-43499)

    554
  • 3

    redforge

    cybersecurity91/redforge

    Browser-side red team toolkit with 102 offline tools for recon, payload generation, credential attacks, crypto, encoding and engagement reporting.

    349
  • 4

    looking-glass

    anonghosty/looking-glass

    CSIRT platform that ingests Shadowserver reports into MongoDB and replays them as interactive maps, analysis, findings, advisories and executive dashboards.

    343
  • 5

    ARTEX

    baarai/artex

    LLM multi-agent autonomous penetration testing platform with planner/worker agents, asset and exploration graphs, MITM traffic recording, and vulnerability retesting.

    323
  • 6

    CyberStrikeAI

    ed1s0nz/cyberstrikeai

    The system of action for AI-native cybersecurity—where intent becomes governed execution, evidence becomes operational memory, and every operation improves the next.

    295

Explore all tools

Browse our collection of tools

View all tools →
Share

AutoPI logo

AutoPI

Code for the paper
AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents

Website Quick start Cite

AutoPI is an adaptive red teamer for tool-using LLM agents. It is built to find many different ways to break an agent, not just to break it often. For each scenario, an attacker LLM writes an indirect prompt injection for every sub-technique of a prompt-injection taxonomy, then reworks the ones that fail using feedback from an LLM judge.

The AutoPI architecture. The engine holds three phases: Initialize with an optional fixed seed, Explore by sweeping a taxonomy, and Exploit by refining failed injections. It sends a prompt to the generator LLM, which writes one injection. The adapter places the injection and runs the target agent once in the benchmark's environment, whose checker reports the outcome. The transcript of a failed attempt goes to the judge LLM, whose diagnosis returns to the engine.

  • Coverage-aware. A sweep tries every sub-technique of the taxonomy, so the successes show which constructions a target falls for, not only how often it falls.
  • Black-box and training-free. AutoPI only sends inputs and reads the transcript and the outcome. There is nothing to train before the first attempt.
  • Steerable. A search strategy is a short YAML file of four building blocks in any order and number.
  • Any benchmark, any model. One engine runs on AgentDojo and IPI Arena through small adapters, and the target, the generator and the judge are plain LLM endpoints.

How it works

A sketch of the attack space of one scenario. Coloured regions hold the injections that work, one per technique. Dashed Explore hops sweep the taxonomy and land near every region. Short Exploit steps refine what failed and close in on a success. A strong seed starts beside a region, a weak seed far from all of them.

Working injections are rare and are conjectured to lie in regions, one per technique. AutoPI searches in three phases, which a strategy can skip, resize, repeat or reorder:

  1. Initialize with an optional fixed seed injection, tried with no generator call.
  2. Explore with a taxonomy sweep. The generator writes one short injection line for every sub-technique.
  3. Exploit through local refinement. A line that fails is reworked into variants of the same construction, guided by the judge's diagnosis of why it failed.

Quick start

Install AutoPI with the bundled AgentDojo. Python 3.11 is recommended.

git clone https://github.com/stevendamianakis/autopi
cd autopi
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e . -e benchmarks/agentdojo

Point the target and the generator at an OpenAI-compatible server, such as vLLM on port 8000, and run the Full strategy on all 949 AgentDojo scenarios:

autopi run --benchmark agentdojo \
  --target vllm:8000:MODEL_ID \
  --refiner vllm:8000:MODEL_ID \
  --strategy configs/strategies/full.yaml \
  --logdir runs/agentdojo-full

Add --dry-run to check the settings and list the scenarios without calling a model. Each run writes its resolved settings and every attempt of every scenario to the --logdir folder. In the code, the generator is called the refiner.

Hosted models and per-role settings

Name each model once in a copy of configs/models.example.yaml, then pass the names. API keys are read from environment variables and never stored in the file.

cp configs/models.example.yaml configs/models.local.yaml
autopi run --benchmark agentdojo --models-config configs/models.local.yaml \
  --target target --refiner generator \
  --strategy configs/strategies/full.yaml --logdir runs/agentdojo-full

Providers are openai-compatible (any URL), openai, anthropic, google and ollama, and short specs such as anthropic:MODEL_ID work without a file. Model configuration covers decoding settings, thinking modes and the recovery of text that a server-side tool parser removed.

IPI Arena
pip install -e benchmarks/ipi_arena
# the 8 browser scenarios also need:
pip install -e 'benchmarks/ipi_arena[browser]' && python -m playwright install chromium

autopi run --benchmark ipiarena --models-config configs/models.local.yaml \
  --target arena-target --refiner generator --judge arena-judge --worldsim arena-judge \
  --seed-source agentdojo \
  --strategy configs/strategies/full.yaml --logdir runs/ipiarena-full

IPI Arena has 41 scenarios: 18 tool use, 15 coding and 8 browser. A scenario counts as broken only when every one of its criteria passes, and --ipi-ignore-concealment selects an action-only metric instead. The target uses native tool calls, and browser scenarios need an image-capable target. The benchmark ships no seed, so use --seed-source agentdojo, a --seed-file, or a strategy without a seed block.

More options
  • --mode 3, the default and the paper's setting, shows the generator the user task and the injection goal. Modes 1 and 2 write goal-blind templates, and mode 2 also sees the user task.
  • --no-judge, --no-context and --no-legitimize switch off one of the three adaptive mechanisms.
  • --seed-file uses another seed, and --no-seed opens with the sweep.
  • --taxonomy-path sweeps another taxonomy with the same file layout.
  • --no-early-stop keeps going after the first success, for coverage runs.
  • --shard and --num-shards split a run across workers, and --resume continues an interrupted one.

autopi run --help lists every option.

Search strategies

A strategy is an ordered list of blocks, an attempt budget per scenario and a stop rule. The three files in configs/strategies/ are the paper's AutoPI-Full, AutoPI-Seed-Exploit and AutoPI-L1. They are examples, not limits:

FileBlocksAttemptsGoal
full.yamlseed, L1 sweep, 3 variants of each failed line529coverage
seed-exploit.yamlseed, 600 variants of the seed601ASR with a strong seed
l1.yamlseed, L1 sweep133low cost
# configs/strategies/full.yaml
version: 1
budget: 529
stop: first_success
blocks:
  - kind: seed
  - kind: sweep
    id: exploration
    level: 1
    order: priority
  - kind: sweep_exploit
    source: exploration
    reps: 3
Download Tool