The five laws

Canonical in chapter 00, copied here. Each is design intent, and the chapter named at the end of a law is where this system is held against it: which parts hold by construction, which hold on only one of the two orchestration paths, which hold on the orchestrator's good behaviour, and which do not hold yet.

1. The model proposes; deterministic code disposes. Give the model a proposal interface, not direct access to the target, raw storage or the last word on severity. Deterministic code validates, executes and records admitted work. The historical system does not enforce that boundary everywhere: both orchestrators can reach a shell, and its write path carries a subcommand that stores a finding without an execution. An invented endpoint might return 404, a login page or an application shell; record the response and judge the claim separately. A shared writer is not a sandbox. Chapters 01 and 02.

2. Claims about the past must quote. Proposals about the future must execute. These are different kinds of statement and they need different gates. A claim about something already observed must cite its own capture; a matching quote establishes citation integrity, not that the conclusion is true. The historical gate leaks: it keeps an item per batch even if none passes, and on one orchestration path a caller-supplied confidence can stand in for the check. A proposed test cannot be validated by quoting an observation it has not made. It may run only after authorization, scope, gate and budget checks permit it, and its outcome still needs interpretation. The law is not permission to execute every proposal. Chapter 02.

3. Severity falls by default and rises only against proof. The deterministic governor can lower a severity or mark a finding false-positive, and cannot raise one. That limits its authority; it does not make its conclusions correct. Under-reporting can hide a real vulnerability, so every lowering rule needs matching and counterexample tests and a reviewable reason. The historical raise endpoint checks a verbatim quote but does not enforce the authored score its contract requests. A quote alone is not exploitability proof. Bind the capture to the finding, apply a reviewed domain proof policy, and retain a separate human review and sign-off process. Chapter 03.

4. Scope is a function, not a sentence. Authorization written into a prompt competes with every other instruction in the context window. Encode the operator's permission as a reviewable policy and enforce it before every outgoing action, with a record of refusals. The historical guard falls short: it is asked at the tool boundary rather than at every request, broadens some host boundaries, and fails open under a kill switch or when constructed without a target. The lab rejects unlisted origins before its trusted callback, but transport containment still belongs in the adapter. A policy decision is only as correct as the authorization and destination it evaluates. Chapter 04.

5. Report what you didn't do. A scan that found nothing and a scan that could not reach anything are different scans, and a report that renders them identically is lying by omission. Coverage, gate status, and a ledger of every host skipped with its reason belong in the deliverable, next to the findings. That is a requirement the historical report did not meet: only coverage arrived, computed against its weakest denominator and under a label naming a different one. The lab accounts for planned tool-and-URL actions, executed work, errors and skips; that denominator does not measure vulnerability coverage. Chapter 05.

Historical case study

The chapters below describe the earlier core/ and walkthrough/ implementation and its published defects. Their absent model-calling verifier remains absent from that historical path. The new harness/ package is a separate offline control reference; it does not retroactively repair the corpus or the historical modules. Read chapter 07's boundary and corrections before copying a historical component.

Point a capable model at a host, hand it a toolbox and tell it to run a penetration test, and it will do something sensible. Run it again tomorrow and it will do something else sensible, and neither run can tell you why it skipped what the other one caught. This handbook argues for a smaller job for the model: give every decision to the cheapest layer that can make it correctly, and spend model capacity only where the answer genuinely is not derivable from what you already have. That ordering, from the decisions a table can make to the few that need a model, is the gradient in chapter 00's title. The chapters follow a working offensive-security agent and the controls its failures demanded.

The historical system has two orchestration paths. On the server-driven path an orchestrator keeps its own phase list and offers the model only the tools that phase allows. On the agent-driven path an orchestrating model plans the run and calls the tools itself, with the layers beneath it constructed but not always consulted. A shared write path makes recorded actions and findings reviewable, but an available shell can bypass it. An invented endpoint earns a captured response, not an automatic vulnerability verdict. The severity governor cannot raise; the separate verifier's historical raise gate checks a quote but does not establish exploitability. Authorization is asked at the tool boundary rather than at every outgoing request. The reporting chapters distinguish a scan that found nothing from a scan refused at the door. Those differences between intent and enforcement are part of the case study, not properties to copy into a new harness.

Every chapter after the first ends with a section on the control's costs and remaining gaps. The case study documents one system and its known control gaps. Appendices D and E report completed verification studies and attribute the measured effect to the model verifier. They do not test scheduler or ranking performance. The selected public-target run is an author-recorded aggregate with two exclusions documented beside it. Its raw findings, ground truth and matcher are not distributed, so its precision cannot be reproduced from this repository. Chapter 05 separates those observations from the untested design argument.

The reference implementation

The historical code under core/ is here to be read, run and disagreed with. It is an educational reference, deliberately non-functional as a live tester: the profiling, the relevance scoring, the scheduling and the tool-call validation are real and runnable, and everything that would put a packet on the wire is withheld. Run ls core/*.py to see what ships rather than trusting a figure written here, which is the kind of claim that goes stale the moment a module is added. The controls the later chapters lean on are among them: the write path is core/store_protocol.py, the severity governor core/severity_governor.py, the scope guard core/scope_guard.py, the gate check core/gate_check.py, and the stage machine walkthrough/run.py. The historical model-calling verifier is withheld. Chapter 06 specifies it; chapter 07 ships a separate deterministic evidence guard, not that verifier and not a human acceptance workflow. Keep their jobs apart: the grounding critic checks citation containment, the governor limits severity, and a raising path must satisfy an independently reviewed proof policy. None of them replaces human review and sign-off.

walkthrough/ drives that stage machine over committed fixtures and writes the artifacts the chapters cite into walkthrough/artifacts/. Regenerate them with python3 -m walkthrough.run, which takes an --out directory if you would rather not touch the committed copies, and tests/test_walkthrough_is_in_sync.py compares a fresh in-memory run against those copies byte for byte, so a fixture edited without a re-run reddens instead of shipping. What that gate does not catch is an artifact that is wrong in both places, and its own docstring says so.

The numbers

Every figure in every chapter resolves to a key in data/stats.json, or carries an annotation naming what the figure is and why it is not a measurement taken from a target. Across chapters 00 through 05 and this README, fourteen annotations name a constant or a property of the code, thirty-five cover a spelled quantity the digit check cannot read, five name an HTTP status code, and one names a comparison between two of this repository's own published snapshots. The statistics file is a frozen snapshot with a published window, not a live query, and chapter 05 explains why re-running the pipeline would not reproduce it.

Chapter 05 reports the system's F1 against a public deliberately-vulnerable application and places it beside an OWASP ZAP passive-scan score. This repository proves the arithmetic and keeps the four aggregate score files synchronized with data/stats.json; it does not contain the raw findings, ground-truth entries, matcher, target identifiers or run identifiers needed to prove that the two tools were evaluated in a controlled head-to-head. Treat the pair as historical, author-recorded data points, not a fair benchmark. The sample size, the spread it therefore does not report, and the runs excluded from it with the reason each was excluded are all in that chapter.

A second measurement exists beside that pair, and it is a different kind of claim: a pre-registered two-arm study of the verifier (a model verifier plus the deterministic handling of its verdicts, ablated as a pair) against the same public lab target. Exact statistics throughout rather than a headline percentage, and a blinded-supported result on shipped precision whose human blind pass is still pending. It says nothing about money or time, and it is not the ranking-layer study chapter 05 still calls unrun. Chapter 05 and appendix D carry the design, the numbers, including corrected recall and severity-integrity accounting, and the limits of each result.

A generated copy of the chapters, with every figure resolved to its value in place and every code citation turned into a link into core/, lives in the rendered tree for reading on GitHub; it is produced by scripts/render.py and kept in step with the source by tests/test_rendered_is_in_sync.py.

If you are publishing the repository, follow PUBLICATION.md. Publish a history-free snapshot into a new public repository; do not change the visibility of the development repository and assume a clean working tree has cleared its reachable Git history. The mandatory pre-commit scripts/publication_gate.sh refuses publication unless the private denylist was actually merged, the public Git author identity matches the approved value, and the staging repository has no prior refs, objects or reflogs.

scripts/audit.sh sweeps the repository for identifiers, scripts/prose_check.sh and scripts/verify_claims.sh sweep the prose, tests/test_gates.sh plants violations against them to prove they still fire, and the test suite holds the reference implementation against what the chapters say about it. A chapter is not finished until each of these passes:

python3 -m pip install -r requirements.txt
                                      # pytest, and nothing else: every module under
                                      #   core/ is standard library only
export HANDBOOK_ROOT=.
bash scripts/audit.sh .               # always the published patterns; the employer, client
                                      #   and host denylist only where it exists, and that
                                      #   file is private, so no clone carries it. Which
                                      #   half ran is in the "sanitization scope:" line
                                      #   this prints and not in the exit status, so read
                                      #   the line
./scripts/prose_check.sh handbook     # common prose patterns
./scripts/verify_claims.sh handbook   # citations, uncited numbers, cross-references,
                                      #   claim anchors, source attribution
./scripts/prose_check.sh README.md    # both gates take a target, and default to handbook/,
./scripts/verify_claims.sh README.md  #   so this file has to be named to be checked
./tests/test_gates.sh                 # the gates against planted violations, the
                                      #   tree, the README, and the chapter claims;
                                      #   the chapters' own prose is covered by the
                                      #   prose and claim gates aimed at handbook
                                      #   above, and not by this sweep
python3 -m pytest tests/              # the whole suite, and it prints its own count
                                      #   rather than having one written here. Every line
                                      #   above runs the gate scripts and the pytest files
                                      #   those wire in, which are the ones about this
                                      #   document; the tests of the controls the five laws
                                      #   are about: the severity governor, the scope
                                      #   guard, the gate check, the shared write path, the
                                      #   grounding critic, are reached by this line and
                                      #   by nothing above it. A break in one of them
                                      #   reddens the lines above only where it also moves
                                      #   the committed walkthrough artifacts

tests/test_chapter_claims.py, inside that suite, is the one worth stealing. It holds assertions against the reference implementation and the published statistics, and each is anchored to the verbatim sentence it backs, so an edit that changes a fact fails a test instead of quietly shipping.