Skip to content
KitploitKITPLOIT
ToolsExploitsBlog
Log in
Submit
ToolsExploitsBlog
Submit

Hacking, PenTest, and Cybersecurity Tools for Your Security Arsenal!

Kitploit is a directory of hacking, cybersecurity, and pentesting tools. Discover the latest project updates to find vulnerabilities, analyze systems, automate testing, and strengthen your security.

FeedsContactPrivacy© 2026 Kitploit

Tool Directory

Categories

View all categories
Loading categories
guardrebind — Research code, benchmark and defense for target rebinding attacks that trick LLM safety guards into grading forged decoy content, plus segment-max distillation hardening. | Kitploit
Tools/GitHubGitHub/ukanwat/guardrebind
Defensive ToolsMachine LearningPapers & ResearchLearning & EducationAI SecurityAdversarial Attack
GitHubukanwat/guardrebind

guardrebind

Research code, benchmark and defense for target rebinding attacks that trick LLM safety guards into grading forged decoy content, plus segment-max distillation hardening.

View Repository
1491 month agoNot yet reviewed
Share
Website

Fooling LLM Safety Guards into Judging the Wrong Text

Code, data and every raw score for the paper (paper/main.pdf, source in paper/).

LLM safety guards receive the content they review by plain string interpolation into their grading prompt, so content that imitates the guard's own prompt format can open a second, forged grading task with a harmless decoy document, which the guard then grades instead of the real content (target rebinding). Appending such a forged task clears Shieldstral-1.0-3B on all 75 pre-registered held-out AdvBench requests it flags, transfers to gpt-oss-safeguard-20B and Llama Guard 4, and in our benchmark, GuardRebind (52 attack families, 5 controls), the strongest family clears 93-100% of flagged items on each of five guards. Segment-max self-distillation trains the guard to score content at least as high as its most harmful part and cuts bypass on held-out attacks from 75% to about 1%, but a score-guided adaptive attacker still clears the trained guard on 94% of items, more than half of them with no forged structure at all.

The attack is described in prose here on purpose. The payloads are generated by src/attacks.py (confirmatory study) and src/bench/attacks_grammar.py (benchmark), and are listed in the paper's appendix.

Layout

pathwhat
PROTOCOL.mdpre-registration for the confirmatory Shieldstral study (paper Section 3; predictions P1-P9)
PROTOCOL_v2.mdpre-registration for the paper's benchmark and defense (M1-M5, F1-F6, A1-A3), with logged deviations D1-D4
src/*.pyconfirmatory study on Shieldstral and the cross-model runs (scorer, payloads, ablation, decoy ladder, OpenRouter clients)
src/bench/GuardRebind: dataset loaders (data.py), guard adapters (guards.py), attack grammar, local and API runners, analysis
src/fix/the defense: segment-max distillation (distill.py), evaluation (evaluate.py), adaptive attacks (adaptive.py), training-free baselines (baselines.py), bounded input format (inputs.py)
src/mech/activation patching, linear probes, representation similarity, decoy dose-response
scripts/queue_*.shresumable job queues that produced the results (one GPU job at a time)
scripts/pod/run the defense queues on a rented CUDA GPU (Runpod) and copy results back
scripts/analysis_r2/CPU statistics behind the defense numbers (Wilson, Newcombe, Hanley-McNeil, paired bootstrap)
paper/data/CPU scripts and their outputs for numbers computed directly for the paper
paper/figures/make_figures.pydraws the paper's figures from results/
results/every raw score and summary the paper cites

Where each table and figure comes from

Commands are relative to the repo root. "GPU" steps load a guard model; "CPU" steps only read files in results/.

Section 3, target rebinding (Shieldstral, confirmatory study, PROTOCOL.md)

paperresult filesproduced by
Figure 1 (anatomy; scores)results/confirm_shieldstral.csv (first document, none and query_doc)python -m src.confirm (GPU)
Table 1 (ablation)results/confirm_log.txt [ablation], results/tables.mdpython -m src.confirm (stdout is confirm_log.txt), then python -m src.figures (CPU, writes tables.md)
Figure 2 (scores; three decoys)results/confirm_shieldstral.csv, results/confirm_groundtruth.csvpython -m src.confirm, python -m src.groundtruth (GPU, stdout results/groundtruth_log.txt); drawn by paper/figures/make_figures.py (CPU)
100-request run (text)results/confirm_hundred.csv, results/hundred_log.txtpython -m src.hundred (GPU)
softened-query control (text)results/confirm_soften.csv, results/soften_log.txtpython -m src.soften (GPU)
forged verdict vs forged task (text)results/fix/forgery_controls_base.jsonpython -m src.fix.forgery_controls --cond base --split test --allow-test --out results/fix/forgery_controls_base.json (GPU; in scripts/queue_fix_full.sh)
robustness paragraph (Section 3 text)results/confirm_log.txt [recency], [markers], [policy]; results/supp_log.txt [A]python -m src.confirm, python -m src.supp (GPU)
Section 3 dose-response paragraph; activation patching (Appendix C)results/mech/patching_test_n100.json, dose_response_test_n150_d40.json, dose_response_test_n150_d40_rho_ci.json (dev-split runs *_dev_* for comparison)src.mech.{patching,dose_response} (GPU; step 0 of scripts/queue_fix_full.sh, PART=main); rho intervals: python -m src.mech.dose_response --recompute results/mech/dose_response_test_n150_d40.json (CPU)
threshold sensitivity (Section 3 text, tau = 0.1)results/sensitivity.mdpython -m src.sensitivity (CPU, reads confirm_shieldstral.csv)

Section 4, GuardRebind (PROTOCOL_v2.md)

paperresult filesproduced by
three guard designs (Section 4 text)results/confirm_crossmodel.csv, results/confirm_gpt_oss_mechanism.csv, results/gpt_oss_mechanism_log.txtpython -m src.crossmodel 30, python -m src.gpt_oss_mechanism 30 (OpenRouter API, no GPU)
Table 2 (benchmark, local guards)results/bench/v2/<guard>__default.jsonl (raw scores, item ids only), results/bench/v2/bypass.csv, run_config.jsonscripts/queue_bench.sh (GPU) -> python -m src.bench.run_bench_v2 ..., then python -m src.bench.analyze --inp results/bench/v2 (CPU)
Table 2 (gpt-oss-safeguard row), gpt-oss paragraph (Section 4.2), multi-verdict reading (Appendix C)results/bench/v2/gpt_oss_safeguard_20b.jsonl, results/bench/v2/gpt_oss_summary.md, results/bench/v2/gpt_oss/*, paper/data/gpt_oss_strict.mdpython -m src.bench.api_bench (OpenRouter), python -m src.bench.api_summary (CPU), python paper/data/gpt_oss_strict.py > paper/data/gpt_oss_strict.md (CPU)
per-family bypass (Section 4.2 text; full table released, not in the paper)results/bench/v2/bypass.csv (default policy, bypass column); gpt-oss from paper/data/gpt_oss_strict.mdpython -m src.bench.analyze (CPU)
benchmark threshold sensitivity (released, not in the paper)results/bench/v2/bypass_tau.csvpython -m src.bench.analyze (CPU)
policy paraphrases and control-token comparison (Appendix C)results/bench/v2/<guard>__para{1,2}.jsonl, results/bench/v2/policy_range.csv, results/bench/v2/bypass.csvscripts/queue_bench.sh, src.bench.analyze
Qwen3Guard "what counts as a task" proberesults/bench/explore/qwen3guard.mdpython -m src.bench.explore.qwen3guard --n 40 (GPU, train split)

Sections 5-6, the defense (PROTOCOL_v2.md, round 2 = deviation D4)

Download Tool

Most Popular

View all →

Discover the most used tools by our community.

No popularity data yet.

Explore all tools

Browse our collection of tools

View all tools →