ActGuard

ActGuard is a pre-execution action-auditing defense against indirect prompt injection in tool-using LLM agents. This repository contains the final ActGuard implementation and the AgentDojo-based runtime needed to evaluate it.

The release contains only the final, full ActGuard configuration. Internal components still use the identifier reflective_audit for compatibility with the AgentDojo benchmark and trace formats.

Requirements

• Python 3.10 or newer
• An OpenRouter API key
• Access to the meta-llama/Meta-Llama-3.1-8B-Instruct checkpoint, or an equivalent local checkpoint
• A CUDA-capable GPU is strongly recommended for local log-probability scoring

Installation

Using uv is recommended:

uv sync --frozen --extra actguard

Alternatively, install into an activated Python environment:

python -m pip install -e '.[actguard]'

Quick start

From the repository root:

export OPENROUTER_API_KEY=...
uv run --frozen --extra actguard ./scripts/run_actguard.sh

If the project was installed with pip, run:

export OPENROUTER_API_KEY=...
./scripts/run_actguard.sh

The script evaluates the workspace suite with the important_instructions attack. It uses OpenRouter for the main agent and verifier, and Hugging Face-compatible local models for embedding retrieval and contrastive log-probability attribution. Hugging Face authentication may be required before the default gated Llama checkpoint can be downloaded.

To use downloaded checkpoints, replace the values of --reflective-audit-embedding-model and --reflective-audit-logprob-model in scripts/run_actguard.sh with their local paths. Use uv run --frozen actguard-benchmark --help to see all benchmark options.

Default configuration

Output

Results are written below:

runs/openai_gpt-4o-mini-2024-07-18-reflective_audit/actguard/

The runs/ directory is ignored by Git.

Tests

Run the ActGuard regression tests:

uv run --frozen pytest -q tests/test_agent_pipeline/test_reflective_audit.py

Repository layout

• src/agentdojo/agent_pipeline/reflective_audit.py: ActGuard implementation
• src/agentdojo/agent_pipeline/agent_pipeline.py: pipeline integration
• src/agentdojo/scripts/benchmark.py: benchmark CLI
• src/agentdojo/default_suites/: evaluation suites and task definitions
• tests/test_agent_pipeline/test_reflective_audit.py: regression tests
• src/agentdojo/reflective_audit_trace.py: trace generation and inspection

Run logs, local environments, credentials, paper drafts, temporary analysis outputs, and comparison defenses are intentionally excluded from this release.

License

This project is distributed under the MIT License. It includes an AgentDojo-derived benchmark runtime; see LICENSE for the retained upstream copyright notice.