Skip to content
KitploitKITPLOIT
ToolsExploitsBlog
Log in
Submit
ToolsExploitsBlog
Submit

Hacking, PenTest, and Cybersecurity Tools for Your Security Arsenal!

Kitploit is a directory of hacking, cybersecurity, and pentesting tools. Discover the latest project updates to find vulnerabilities, analyze systems, automate testing, and strengthen your security.

FeedsContactPrivacy© 2026 Kitploit

Tool Directory

Categories

View all categories
Loading categories
Tools/GitHubGitHub/egekaya1/qrefusal
Machine LearningPapers & ResearchLearning & EducationAI Security
GitHubegekaya1/qrefusal

qrefusal

Is 4-bit quantization more than noise to a chat model's safety behaviour? Code, pre-registration and labels.

View Repository
15h 18m agoNot yet reviewed
Share

Is 4-bit quantization more than noise to a chat model's safety behaviour?

Attributing behavioural shifts with matched perturbations

Code and analysis for a pre-registered audit of how 4-bit quantization changes the refusals of small chat models, with controls that ask whether a change comes from quantization or would follow from any perturbation of the same size.

Paper: arXiv:2611.22186

What this is about

Chat models that run on phones and laptops are usually 4-bit copies of the model that was evaluated for safety. Audits of such copies disagree: some find that quantization weakens refusal, some find it does not, some find that models refuse more. A 4-bit build is one particular perturbation of every weight, and there are many other perturbations of the same size. So a shift in its refusals could come from the structure of rounding error, or from the model's own lean under any nudge of that size.

We test three 3-4B chat models (Qwen2.5-3B-Instruct, Llama-3.2-3B-Instruct, Qwen3-4B-Instruct-2507) in the round-to-nearest formats that MLX ships for Apple Silicon. Every comparison is paired and read against an 8-bit build, which leaves likelihood unchanged. In brief:

  • Forced start. When the first eight tokens of a compliant reply are prefilled, the default 4-bit build makes Qwen2.5-3B more likely to continue it. Llama-3.2-3B and Qwen3-4B do not move.
  • Free start. On 400 held-out jailbreak prompts with no prefill, 4-bit Llama-3.2-3B becomes less compliant. In an exploratory check it also refuses more benign requests written in jailbreak style, while its handling of plain safe requests is unchanged.
  • Controls. We replace the rounding error with its mirror image, a within-group shuffle and size-matched Gaussian noise, scale it, and compare it with other quantizers in the same format (AWQ, GPTQ) and with a second runtime (llama.cpp). We also measure each build's effect on the model's refusal direction. Together these attribute each shift to the model or to the rounding.

The numbers behind every sentence in the paper are generated by one script (see Reproducing the paper).

Layout

PathContents
PREREG.mdThe pre-registration and its five dated addenda, each fixed before the data it concerns
qrefusal/Shared code: chat formatting with a pinned template date, greedy and sampled decoding, a layer-by-layer forward pass with hooks (MLX has no hook API), refusal directions, paired statistics
runs/One script per experiment; each writes to results/<experiment>/
analysis/make_numbers.pyRegenerates every number in the paper from the raw result files, with integrity checks
analysis/figures.pyDraws the figures from paper/numbers.json
paper/LaTeX source; numbers enter the text only as macros generated by the analysis
scripts/Model conversion, data fetching, and the run queues
tests/Identity tests for the hooked forward pass and the statistics

The experiments, in paper order

PaperScriptWhat it does
§3 Setupscripts/convert.shBuilds bf16, 8-bit, 4-bit (group 64 and 32), mixed 4/6, MXFP4, AWQ and GPTQ copies of a model
§3 Setupruns/r2.py, runs/e7_abliterate.py, runs/e7b_prefixes.pyRefusal directions per layer; an abliterated bf16 copy; the compliant prefixes used for prefill
§3 Setupruns/r1b_judge.py, runs/j2_ensemble.py, runs/l2_pair_sheet.pyContent judge (Qwen2.5-14B), Llama Guard 3, and the blind two-phase human-labelling sheet
§4 Label instability and the 8-bit nullruns/e0_determinism.py, runs/a2_depth.py --tempGreedy rerun identity; the sampling seed floor
§5.1 Forced startruns/a2_depth.py, runs/r1.pyPrefill depths k = 0, 2, 8; generic prefix (--fixed_prefix); length check (--max_tokens 384 --sample 150)
§5.2 Free startruns/x2_sneaky.pyWildJailbreak development and held-out test sets; SORRY-Bench
§6 Two simpler explanationsruns/x1_xstest.py, runs/c1_capability.pyOver-refusal on XSTest and jailbreak-styled benign prompts; likelihood and bits per weight
§7 Matched-perturbation controlsruns/e2_perturb.py, runs/e2_errors.py, runs/g1_gguf.pyMatched-perturbation and scaled builds with a validation gate; per-matrix rounding error; the llama.cpp runtime
§8 Refusal-direction projectionsruns/m1_direction.pyProjections onto the bf16 refusal direction
Appendixruns/a1_prefill.py, runs/r5_runon.pyThe prefill pilot; an end-of-turn failure in MXFP4 Qwen2.5

Reproducing the paper

Hardware: one Apple Silicon Mac with 16 GB of memory (we used an M2 Pro). The full set of runs takes several days of machine time; models are loaded one at a time.

pip install -e ".[test]"                    # mlx 0.32.4, mlx-lm 0.32.2
scripts/fetch_data.sh                        # refusal_direction prompt splits (Apache-2.0)
scripts/convert.sh Qwen/Qwen2.5-3B-Instruct qwen2.5-3b
scripts/convert.sh meta-llama/Llama-3.2-3B-Instruct llama3.2-3b
scripts/convert.sh Qwen/Qwen3-4B-Instruct-2507 qwen3-4b
scripts/convert.sh Qwen/Qwen2.5-0.5B-Instruct qwen2.5-0.5b   # needed by the tests and Appendix G
python -m pytest tests

Three prompt sets are obtained from their authors and placed under data/:

  • XSTest (Röttger et al., 2024): data/xstest/xstest_prompts.csv
  • SORRY-Bench (Xie et al., 2024; gated on Hugging Face): data/sorry/question*.jsonl
  • WildJailbreak evaluation set (Jiang et al., 2024; gated on Hugging Face): data/wildjailbreak/eval/eval.tsv

The run queues in scripts/ give the commands for the later registered analyses in the order we ran them; the table above gives the scripts for the rest. They expect a conda environment named ml, and runs/g1_gguf.py expects llama.cpp (commit 0ec4027) at ../tools/llama.cpp-src and Homebrew llama-quantize. Each experiment skips outputs that already exist, so a queue can be stopped and restarted. Then:

python analysis/make_numbers.py    # paper/numbers.{md,json}, paper/tex/numbers.tex
python analysis/figures.py         # paper/tex/figures/*.pdf
cd paper/tex && tectonic main.tex

Pre-registration

PREREG.md fixes the primary test, the secondary families, the human-label design, the held-out free-start test and the controls' decision rule. Each addendum is dated and was fixed before the data it concerns existed. The paper's appendix lists what was registered when and the changes to the plan. The dated commits are in a private development repository (hashes 50d0225, 2c53665, 86c93e0, 3e2ea52, 02e72c5, 09e49aa), available on request; this repository's history starts at release. Three sentences that pointed to private planning notes were removed from PREREG.md; nothing else was changed.

What is and is not released

Released: the code, the pre-registration, the aggregate results (paper/numbers.{md,json}) and every judge, Llama Guard and human label, with no prompt or reply text, in labels/ (written by analysis/export_labels.py). Rows are keyed by setting, model, build and prompt index i: the row index of the prompt split that runs/ loads (for x2/*, the seeded samples in runs/x2_sneaky.py). Not released: the model replies, which include harmful content produced under a forced start; make_numbers.py reads them from results/, which is excluded from version control. They are available to researchers on request.

License

Code: MIT (LICENSE). Labels and paper text: CC BY 4.0. paper/tex/acl.sty and acl_natbib.bst keep their own terms.

Citation

@article{kaya2026quantization,
  title={Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations},
  author={Kaya, Ege},
  journal={arXiv preprint arXiv:2611.22186},
  year={2026}
}
Download Tool

Most Popular

View all →

Discover the most used tools by our community.

  • 1

    OsintHunter

    letox4/osinthunter

    🎯 The ultimate OSINT toolkit: investigation graph, confidence scoring, alias families, map, timeline and reports. 49 modules, 400+ sites, no API keys required.

    696
  • 2

    ghostlock-app

    yukonga/ghostlock-app

    GhostLock One-Tap Execution App (CVE-2026-43499)

    499
  • 3

    ARTEX

    baarai/artex

    LLM multi-agent autonomous penetration testing platform with planner/worker agents, asset and exploration graphs, MITM traffic recording, and vulnerability retesting.

    396
  • 4

    redforge

    cybersecurity91/redforge

    Browser-side red team toolkit with 102 offline tools for recon, payload generation, credential attacks, crypto, encoding and engagement reporting.

    340
  • 5

    CyberStrikeAI

    ed1s0nz/cyberstrikeai

    The system of action for AI-native cybersecurity—where intent becomes governed execution, evidence becomes operational memory, and every operation improves the next.

    258
  • 6

    antifox

    antifox-ai/antifox

    AI multi-agent autonomous penetration testing platform that runs recon, attack-surface discovery, exploitation validation and evidence-backed report delivery.

    257

Explore all tools

Browse our collection of tools

View all tools →