ukanwat/guardrebindResearch code, benchmark and defense for target rebinding attacks that trick LLM safety guards into grading forged decoy content, plus segment-max distillation hardening.
Code, data and every raw score for the paper (paper/main.pdf, source in paper/).
LLM safety guards receive the content they review by plain string interpolation into their grading prompt, so content that imitates the guard's own prompt format can open a second, forged grading task with a harmless decoy document, which the guard then grades instead of the real content (target rebinding). Appending such a forged task clears Shieldstral-1.0-3B on all 75 pre-registered held-out AdvBench requests it flags, transfers to gpt-oss-safeguard-20B and Llama Guard 4, and in our benchmark, GuardRebind (52 attack families, 5 controls), the strongest family clears 93-100% of flagged items on each of five guards. Segment-max self-distillation trains the guard to score content at least as high as its most harmful part and cuts bypass on held-out attacks from 75% to about 1%, but a score-guided adaptive attacker still clears the trained guard on 94% of items, more than half of them with no forged structure at all.
The attack is described in prose here on purpose. The payloads are generated by
src/attacks.py (confirmatory study) and src/bench/attacks_grammar.py (benchmark), and
are listed in the paper's appendix.
| path | what |
|---|---|
PROTOCOL.md | pre-registration for the confirmatory Shieldstral study (paper Section 3; predictions P1-P9) |
PROTOCOL_v2.md | pre-registration for the paper's benchmark and defense (M1-M5, F1-F6, A1-A3), with logged deviations D1-D4 |
src/*.py | confirmatory study on Shieldstral and the cross-model runs (scorer, payloads, ablation, decoy ladder, OpenRouter clients) |
src/bench/ | GuardRebind: dataset loaders (data.py), guard adapters (guards.py), attack grammar, local and API runners, analysis |
src/fix/ | the defense: segment-max distillation (distill.py), evaluation (evaluate.py), adaptive attacks (adaptive.py), training-free baselines (baselines.py), bounded input format (inputs.py) |
src/mech/ | activation patching, linear probes, representation similarity, decoy dose-response |
scripts/queue_*.sh | resumable job queues that produced the results (one GPU job at a time) |
scripts/pod/ | run the defense queues on a rented CUDA GPU (Runpod) and copy results back |
scripts/analysis_r2/ | CPU statistics behind the defense numbers (Wilson, Newcombe, Hanley-McNeil, paired bootstrap) |
paper/data/ | CPU scripts and their outputs for numbers computed directly for the paper |
paper/figures/make_figures.py | draws the paper's figures from results/ |
results/ | every raw score and summary the paper cites |
Commands are relative to the repo root. "GPU" steps load a guard model; "CPU" steps only
read files in results/.
Section 3, target rebinding (Shieldstral, confirmatory study, PROTOCOL.md)
| paper | result files | produced by |
|---|---|---|
| Figure 1 (anatomy; scores) | results/confirm_shieldstral.csv (first document, none and query_doc) | python -m src.confirm (GPU) |
| Table 1 (ablation) | results/confirm_log.txt [ablation], results/tables.md | python -m src.confirm (stdout is confirm_log.txt), then python -m src.figures (CPU, writes tables.md) |
| Figure 2 (scores; three decoys) | results/confirm_shieldstral.csv, results/confirm_groundtruth.csv | python -m src.confirm, python -m src.groundtruth (GPU, stdout results/groundtruth_log.txt); drawn by paper/figures/make_figures.py (CPU) |
| 100-request run (text) | results/confirm_hundred.csv, results/hundred_log.txt | python -m src.hundred (GPU) |
| softened-query control (text) | results/confirm_soften.csv, results/soften_log.txt | python -m src.soften (GPU) |
| forged verdict vs forged task (text) | results/fix/forgery_controls_base.json | python -m src.fix.forgery_controls --cond base --split test --allow-test --out results/fix/forgery_controls_base.json (GPU; in scripts/queue_fix_full.sh) |
| robustness paragraph (Section 3 text) | results/confirm_log.txt [recency], [markers], [policy]; results/supp_log.txt [A] | python -m src.confirm, python -m src.supp (GPU) |
| Section 3 dose-response paragraph; activation patching (Appendix C) | results/mech/patching_test_n100.json, dose_response_test_n150_d40.json, dose_response_test_n150_d40_rho_ci.json (dev-split runs *_dev_* for comparison) | src.mech.{patching,dose_response} (GPU; step 0 of scripts/queue_fix_full.sh, PART=main); rho intervals: python -m src.mech.dose_response --recompute results/mech/dose_response_test_n150_d40.json (CPU) |
| threshold sensitivity (Section 3 text, tau = 0.1) | results/sensitivity.md | python -m src.sensitivity (CPU, reads confirm_shieldstral.csv) |
Section 4, GuardRebind (PROTOCOL_v2.md)
| paper | result files | produced by |
|---|---|---|
| three guard designs (Section 4 text) | results/confirm_crossmodel.csv, results/confirm_gpt_oss_mechanism.csv, results/gpt_oss_mechanism_log.txt | python -m src.crossmodel 30, python -m src.gpt_oss_mechanism 30 (OpenRouter API, no GPU) |
| Table 2 (benchmark, local guards) | results/bench/v2/<guard>__default.jsonl (raw scores, item ids only), results/bench/v2/bypass.csv, run_config.json | scripts/queue_bench.sh (GPU) -> python -m src.bench.run_bench_v2 ..., then python -m src.bench.analyze --inp results/bench/v2 (CPU) |
| Table 2 (gpt-oss-safeguard row), gpt-oss paragraph (Section 4.2), multi-verdict reading (Appendix C) | results/bench/v2/gpt_oss_safeguard_20b.jsonl, results/bench/v2/gpt_oss_summary.md, results/bench/v2/gpt_oss/*, paper/data/gpt_oss_strict.md | python -m src.bench.api_bench (OpenRouter), python -m src.bench.api_summary (CPU), python paper/data/gpt_oss_strict.py > paper/data/gpt_oss_strict.md (CPU) |
| per-family bypass (Section 4.2 text; full table released, not in the paper) | results/bench/v2/bypass.csv (default policy, bypass column); gpt-oss from paper/data/gpt_oss_strict.md | python -m src.bench.analyze (CPU) |
| benchmark threshold sensitivity (released, not in the paper) | results/bench/v2/bypass_tau.csv | python -m src.bench.analyze (CPU) |
| policy paraphrases and control-token comparison (Appendix C) | results/bench/v2/<guard>__para{1,2}.jsonl, results/bench/v2/policy_range.csv, results/bench/v2/bypass.csv | scripts/queue_bench.sh, src.bench.analyze |
| Qwen3Guard "what counts as a task" probe | results/bench/explore/qwen3guard.md | python -m src.bench.explore.qwen3guard --n 40 (GPU, train split) |
Sections 5-6, the defense (PROTOCOL_v2.md, round 2 = deviation D4)
