AgentTriage

AgentTriage is a cross-agent tracing and security-diagnosis system for tool-using LLM agents. It observes source ingress, planning and tool lifecycles, normalizes them into one canonical event contract, reconstructs a Simplified Typed Provenance Graph (STPG), extracts a source-to-sink chain, attributes the likely defense gap, and renders a Defense-Gap Card.

This repository contains the current AgentTriage runtime implementation, five lightweight target overlays, pinned source snapshots of the corresponding open-source agents, one complete 53-case behavior repetition, and one set of 40 label-free diagnosis evidence pages.

Included components

agenttriage/
  canonicalize/        target-neutral event normalization and parent linking
  stpg/                STPG construction, slicing, labeling, and compaction
  chain/               source-to-terminal chain extraction
  attribution/         outcome-aware defense-gap attribution
  render/              JSON/Markdown Defense-Gap Card rendering
  batch/               target launcher, trace validation, and artifact pipeline
targets/               five target-specific runtime overlays
open-source-agents/     pinned source snapshots governed by upstream licenses
schemas/                trace, canonical-event, and target schemas
contracts/              public acquisition/diagnosis contract
examples/               offline diagnosis examples and safe smoke inputs
data/                   sanitized collected cases and label-free evidence pages

The five targets are TaskWeaver, Agent Zero, LangGraph, Goose, and Microsoft Agent Framework. Target-only information must stay under metadata.<target>; all top-level trace events use agenttriage.trace.v0.2 and all target drivers use the common v0.2 interface.

The five overlay definitions and TaskWeaver runtime wrapper correspond to the latest implementation selected for this public source tree. Timing, annotation, and evaluator-only code is intentionally outside this release. Target-specific upstream revisions and dependency locks remain available in each target's version-lock.json and container configuration.

Safety boundary

The target agents can execute code and write files. Use only disposable Docker containers, test-only API keys, the provided local fixtures, and loopback endpoints. Do not mount sensitive host directories or point the examples at production systems. See SECURITY.md.

Requirements

• macOS or Linux;
• Python 3.10 or newer;
• Docker Desktop/Engine for target-agent runs;
• Git is optional after cloning;
• a provider key only for real-model targets/cases.

The host package has one runtime dependency, jsonschema. Target dependencies are isolated in their digest-pinned Docker images.

Install the host controller

cd agenttriage-open-source
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .
python -m pip install pytest

Verify the key-free path:

pytest -q
agenttriage \
  --case examples/sample_case.json \
  --output /tmp/agenttriage-offline-demo \
  --print-markdown

The offline command writes:

raw_events.json
canonical_events.json
executed_chain.json
stpg.json
stpg_slice.json
assessment.json
card.json
card.md

Configure model credentials

Never edit a tracked configuration with a real key. Create only the local env files needed by the targets you intend to run:

cp .env.example targets/agentzero/.env.local
cp .env.example targets/langgraph/.env.local
cp .env.example targets/goose/.env.local
cp .env.example targets/microsoft_agent_framework/.env.local
cp .env.example targets/taskweaver/.env.local

Populate DEEPSEEK_API_KEY for Agent Zero, LangGraph, Goose, and Microsoft Agent Framework real-model modes. Populate LLM_API_KEY for TaskWeaver. The sample Goose/LangGraph/MAF smoke fixtures are deterministic and can be run without a model key, but the runner still requires the configured env file to exist; an empty copied template is sufficient for those fixtures.

Build a target image

The batch runner builds an absent image automatically. To build explicitly:

docker build -f targets/taskweaver/Dockerfile -t agenttriage-taskweaver:runtime-overlay-v0.3 .
docker build -f targets/agentzero/Dockerfile -t agenttriage-agentzero:v2.4-overlay-v0.2 .
docker build -f targets/langgraph/Dockerfile -t agenttriage-langgraph:1.2.9-overlay-v0.4 .
docker build -f targets/goose/Dockerfile -t agenttriage-goose:1.43.0-overlay-v0.3 .
docker build -f targets/microsoft_agent_framework/Dockerfile -t agenttriage-microsoft-agent-framework:1.11.0-overlay-v0.3 .

Image bases and upstream revisions are fixed in each targets/<target>/version-lock.json. If dockerproxy.net is unavailable in your region, replace only the registry host while retaining the pinned digest.

Run one safe target smoke case

Each target has a separate prompt/policy directory under examples/smoke/. The command shape is identical across all five targets:

agenttriage-batch \
  --target langgraph \
  --prompts examples/smoke/langgraph/prompts.jsonl \
  --policy-oracles examples/smoke/langgraph/policy_oracles.jsonl \
  --output runs/langgraph-smoke-01 \
  --runtime docker \
  --instrumentation-mode semantic \
  --build \
  --timeout-seconds 300

Replace langgraph with taskweaver, agentzero, goose, or microsoft_agent_framework and use the matching smoke directory. Always use a new output directory. A case directory contains the driver output and semantic trace plus runner-owned normalized artifacts:

driver_result.json
semantic_trace.jsonl
raw_trace.json
trace_validation.json
case.json
driver.stdout.log
driver.stderr.log
artifacts/canonical_events.json
artifacts/executed_chain.json
artifacts/stpg.json
artifacts/stpg_slice.json
artifacts/card.json
artifacts/card.md

Use --instrumentation-mode off with the same inputs for an untraced behavior check. That mode intentionally does not produce semantic_trace.jsonl.

Target overlay contract

Every driver receives:

• prompt.json;
• policy_oracle.json;
• a semantic-trace output path;
• instrumentation mode (semantic or off);
• safe_test_assets.json and an isolated materialized asset root;
• an optional high-priority system-policy file.

Every driver owns only driver_result.json and semantic_trace.jsonl. The common runner validates the trace, creates raw_trace.json and case.json, runs AgentTriage, and writes STPG/Card artifacts. Invocation and result events share an explicit call_id; chronological adjacency alone is not treated as causal evidence.

TaskWeaver uses the same lightweight runtime-overlay model as the other four targets: the pinned upstream source is not patched. Runtime wrappers are installed at process startup and restored on exit. This release deliberately does not contain AgentTriage's former generic hook-plan/Instrumentation module or a patched TaskWeaver source tree; semantic events are emitted only by the five target-local runtime overlays.

Add another agent

Create targets/<id>/target.yaml, driver.py, Dockerfile, overlay.json, and version-lock.json. Reuse the public schemas; place target-specific fields only in metadata.<id>. Capture source ingress, tool invocation/result with a shared call_id, terminal state, argument fields, and explicit field lineage. Do not add target-specific top-level event fields.

Public case data

data/behavior-collection contains 53 safe cases from one complete repetition across the five targets. The sample includes raw/semantic traces and AgentTriage STPG/Card artifacts, but excludes internal run-control and efficiency summaries.