Skip to content
KitploitKITPLOIT
ToolsExploitsBlog
Log in
Submit
ToolsExploitsBlog
Submit

Hacking, PenTest, and Cybersecurity Tools for Your Security Arsenal!

Kitploit is a directory of hacking, cybersecurity, and pentesting tools. Discover the latest project updates to find vulnerabilities, analyze systems, automate testing, and strengthen your security.

FeedsContactPrivacy© 2026 Kitploit

Tool Directory

Categories

View all categories
Loading categories
signed-but-steerable — Measuring and closing the text–vector integrity gap in offline RAG: margin law, vector-database audits and code-level verification. | Kitploit
Tools/GitHubGitHub/securagenai/signed-but-steerable
Static AnalysisVulnerability AnalysisCode AnalysisPapers & ResearchAI Security
GitHubsecuragenai/signed-but-steerable

signed-but-steerable

Measuring and closing the text–vector integrity gap in offline RAG: margin law, vector-database audits and code-level verification.

View Repository

Most Popular

View all →

Discover the most used tools by our community.

Explore all tools

Browse our collection of tools

View all tools →
281 day agoNot yet reviewed
Share

Signed but Steerable: the text-vector integrity gap in offline RAG

Code, data and results for the paper Signed but Steerable: Measuring and Closing the Text-Vector Integrity Gap in Offline Retrieval-Augmented Generation (G. Feretzakis, A. Papaspyridis, securagen.ai).

DOI

Archived on Zenodo: 10.5281/zenodo.23945569 (all versions; release v1.0.2, cited in the paper: 10.5281/zenodo.23948839). The arXiv identifier of the paper will be added when it is available.

What is here

  • ragtrust/ shared library, including codeverify.py, the reference code-level verifier used for every code-verification result (ambiguity decided only from the verifier's recomputation; neighbour-only acceptance; canonicalisation).
  • desktop/ experiments run on the desktop: abstention scoring (E1) and its MS MARCO check, the vector desynchronisation attack and the margin law (E7), BEIR margins, vector-database exposure, the FAISS demonstration, code-level verification and the determinism of float pipelines.
  • device/ experiments run on the Raspberry Pi 5 CPU: context pruning (E2), the cost of verification next to the language model (E4), device-side embedding and INT8 fidelity (E5), the end-to-end attack (E7) and pack updates (E8).
  • analysis/ analyses, the precision and unit checks, and make_figures.py, which regenerates the figures.
  • data/ the SQuAD-derived corpus, windows, questions and vectors; results/ per-experiment outputs; tests/ unit tests; remote/ helpers to drive the Pi from the desktop (adapt host names).
  • PROTOCOL.md hypotheses fixed before data collection and the log of deviations and exploratory analyses. Experiment numbers follow the protocol; E3 and E6 are not part of this release.
  • pins.json pinned revisions of models and tools.

Quick start

  • Tests: python tests/test_codeverify.py.
  • Desktop environment: bash desktop/setup_desktop.sh; figures: python analysis/make_figures.py.
  • Device experiments: a Raspberry Pi 5, Python with device/requirements_device.txt, and llama.cpp at the commit in pins.json.

Running the device experiments

On the Raspberry Pi, after cp device/config.env.example device/config.env (adapt the paths) and installing device/requirements_device.txt into venv_device:

  • E2 and E7 generation: bash device/inner_generate.sh starts llama-server and runs k3_full (first 60 questions, reproducibility check), k5_ce3, k10_ce3, k10_ce5, k5_bi3, k5_ce3 with the online INT8 selector, and the E7 configurations; the CPU-INT8 end-to-end configuration alone: bash device/run_e7_cpu_int8.sh. The variants k5_ce3w, k3_ce3 and k3_ce3w run with python device/e2_generate.py --config <name> while llama-server is running (started as in device/inner_generate.sh). A configuration is skipped when results/e2/status_<name>.json exists; move those files aside to regenerate the released results.
  • E4: python device/e4_contention.py; E5: python device/e5_reindex.py --backends ort-fp32,ort-int8 --batch 32 (the batch size of the released results: dynamic INT8 depends on the batch, and batches of 64 give different vectors); E8: python device/e8_device_verify.py --vector-backend ort-fp32.
  • From the desktop, remote/pi_sync.sh copies the repository to the Pi and remote/pi_fetch.sh fetches results.

Where the paper's numbers are

  • Margin law on SQuAD (Table 2): results/e7/margin_law_squad.json; other corpora and Fig. 1: results/e7/beir_margins.json and results/e7/margins/; exposure at the CPU-INT8 reference threshold: results/e7/cpu_int8_exposure.json; numerical precision and unit checks: results/e7/margin_precision_check.json; calibration of the INT8 reference threshold (exact quantile, honest windows below it, bit-for-bit re-encoding at batch 32): results/e5/int8_calibration_check.json (analysis/e5_int8_calibration_check.py).
  • Compressed stores: results/e7/vecdb_quant.json; FAISS audit and executed attack: results/e7/faiss_integrity.json; residual budget: results/e7/guard_budget.json.
  • End to end on the device: results/e7/analysis.json (correct answers and refusals) and results/e7/e2e_checks.json (same questions, angular budget, payload in the top-3 context).
  • Context pruning (E2): results/e2/decomposition.json; cost next to the language model (E4): results/e4/.
  • Figures: results/figures/fig_margin_law.pdf (Fig. 1) and results/figures/fig_vdb.pdf (Fig. 2), regenerated by analysis/make_figures.py.

Not included

  • Model weights and exported ONNX graphs (licences): downloaded and rebuilt by the scripts at the pinned revisions.
  • BEIR and MS MARCO data and derived vectors (licences): downloaded by the scripts.
  • Large vector files and caches, regenerated by the scripts: results/e7/squad_question_vectors.npz (analysis/margin_law_squad.py) and results/e7/proxy_vectors.npz (analysis/margin_law.py).
  • Intermediate inputs regenerated or downloaded by the scripts: results/e7/vecs/ (written by desktop/beir_margins.py), results/e7/scale_*.npy (written by the code-verification scripts), data/beir/ and artifacts/models/ (written by desktop/export_onnx.py).
  • Run logs.

Some analysis scripts load caches that they created themselves with numpy.load(..., allow_pickle=True); load such files only from your own runs, never from untrusted sources.

Third-party resources

Models, datasets and their licences are listed in DATA_LICENSES.md. Please also cite SQuAD 2.0 (Rajpurkar et al., 2018), BEIR (Thakur et al., 2021), MS MARCO (Bajaj et al., 2016), BGE-M3 (Chen et al., 2024) and Sentence-BERT (Reimers and Gurevych, 2019) when you use the corresponding parts.

Notes

  • FAISS behaves as designed throughout: it provides storage formats and search, not integrity audits. The threat model is a compromised publisher or writer of the stored vectors, not a defect of any library or product.
  • This is research code, provided as is and without warranty (see LICENSE).
  • Raspberry Pi is a trademark of Raspberry Pi Ltd. Other product names are trademarks of their respective owners.

Licence

Copyright 2026 securagen.ai P.C. MIT for code and our results; SQuAD-derived data, results files that contain SQuAD text and the generated answers keep CC BY-SA 4.0; see LICENSE and DATA_LICENSES.md for the details and for the terms of the other datasets and models.

Download Tool