ryehr/rrc_steganographyProvably secure linguistic steganography embedding secret messages into LLM-generated text via rotation range-coding. Includes embed, extract, and round-trip verification scripts.
Rotation Range-Coding (RRC) Steganography — an efficient and provably secure linguistic steganographic method that embeds secret messages into natural-language text generated by large language models.
Paper: Efficient Provably Secure Linguistic Steganography via Range Coding (ACL 2026)
| Step | Description |
|---|---|
| Embed (Algorithm 3) | Convert the secret message to a decimal value and iteratively rotate it within a shrinking interval guided by the LM's probability distribution and a PRNG-generated offset. Each rotation step produces one token. |
| Extract (Algorithm 4) | Re-run the LM on the stegotext to recover interval bounds, then reverse-rotate back to the original decimal value and binarize it. |
The rotation mechanism solves two issues of vanilla range-coding steganography:
.
├── rrc_core.py # Algorithms 3 and 4, shared by every entry point
├── RRC_embed.py # Embedding CLI
├── RRC_extract.py # Extraction CLI (reports recovery accuracy)
├── test_roundtrip.py # Round-trip test across several message lengths
├── fastest_embed.py # Embedding-speed benchmark
├── 0.Prompts.tsv # 1000 prompts (first ~10 words of C4 sequences)
└── requirements.txt
The encoder and the decoder share rrc_core.py rather than holding separate
copies of the loop. The two sides must agree bit for bit, and a divergence
between two independently maintained copies surfaces only as a failed round
trip, never on inspection.
pip install -r requirements.txt
python test_roundtrip.py # GPT-2, 32/128/512 bits
python test_roundtrip.py --model meta-llama/Llama-2-7b-hf
python RRC_embed.py \
--language_model meta-llama/Llama-2-7b-hf \
--bit_length 128 --key 42 --top_k -1
python RRC_extract.py \
--language_model meta-llama/Llama-2-7b-hf \
--bit_length 128 --key 42 \
--input_file 1.RC_decimal_Llama-2-7b-hf_bit128.tsv
The encoder records the secret it embedded and the token ids that carried it, so extraction reports a recovery rate. Two receiver models are scored:
ids — the receiver is given the token sequence. This isolates the coding
scheme; anything below 100% is a numerical or logical fault.text — the receiver is given the detokenised string and tokenises it again,
as a text channel requires. Re-tokenisation does not always reproduce the
sender's segmentation, so this is a lower bound; the gap between the two
measures tokenisation ambiguity rather than the coding scheme. On Llama-2-7b
at 128 bits: 100% for ids, ~91% for text.Important: sender and receiver must share the language model, key, bit length and top-k setting, and must also run the same hardware and library versions. See Numerical requirements.
Shared by RRC_embed.py and RRC_extract.py:
| Argument | Default | Description |
|---|---|---|
--language_model | meta-llama/Llama-3.1-8B | HuggingFace model id |
--bit_length | 128 | Secret message length in bits |
--top_k | -1 | Top-k truncation; -1 = full vocabulary |
--key | 42 | Symmetric key K (PRNG seed) |
--precision_margin | 40 | Decimal digits beyond those 2**bit_length needs |
--max_ctx_len | 3900 | KV-cache window |
--device | auto | cuda or cpu |
RRC_embed.py also takes --prompts, --output, --limit, --seed,
--max_tokens, --resume, --no_verify, and --part / --part_max for
sharding a prompt set across processes (--part_max 3 with --part 0,1,2, one
per GPU).
RRC_extract.py also takes --input_file, --mode {ids,text,both},
--limit and --verbose.
RRC narrows an interval of width 2**l down to width ~1, so the recovered
value is sensitive to relative error by a factor of 2**l: at 128 bits a
relative perturbation of 1e-16 anywhere in the chain displaces the result by
~1e22 against a tolerance of 0.5. Extraction does not degrade gracefully — it
either returns the message or returns unrelated bits. Measured on Llama-2-7b:
| Setting | Relative perturbation of the probabilities | Extraction |
|---|---|---|
| Repeating the same batch-1 decode | 0 (bitwise identical) | works |
| The same row at batch 2 / 8 / 16 | ~1e-2 | fails |
| Left-padding the prompt | ~1e-2 | fails |
| One teacher-forced pass instead of incremental decoding | ~4e-3 | fails |
Two consequences shape the implementation. Batching is unavailable, so sequences are processed one at a time. And the receiver replays the sender's incremental decoding step for step rather than scoring the stegotext in a single forward pass, even though it holds every token in advance. Both shortcuts measure ~9x faster and neither preserves the probabilities.
Everything after the logits is float64 and produces bitwise identical boundaries on CPU and GPU, so the forward pass is the only step that ties a stegotext to its hardware.
Precision is derived from the message length as
ceil(bit_length * log10(2)) + margin and set before anything else runs. It
scales with the message: 2**256 needs 78 digits and 2**8192 needs 2466.
Precision below that threshold does not fail cleanly. The low-order digits of the secret value are rounded away, the interval collapses within a few tokens and the termination test fires early, so a run can complete and report figures that are arithmetically unreachable. Holding the precision at 50 digits regardless of message length gives:
| bits | utilisation at 50 digits | samples completing | with derived precision |
|---|---|---|---|
| 256 | 157.05% | 8/20 | 100.17% |
| 512 | 299.67% | 9/20 | 102.48% |
| 1024 | 660.91% | 7/10 | 102.58% |
| 2048 | 1046.40% | 1/5 | 98.18% |
| 4096 | 2483.92% | 3/3, no error raised | 100.87% |
| 8192 | 4766.49% | 1/2 | 99.07% |
--precision_margin adjusts the headroom above the derived minimum; the
minimum itself is enforced.
Algorithm 3 stops when (L + R) / 2 - d_s falls in (-0.5, 0.5]. That test
can fire while the interval is narrower than 1, and the window
[mid - 0.5, mid + 0.5) is then wider than the interval carrying it. Reverse
rotation is an isometry modulo the interval width, so a window straddling the
cut leaves the receiver a full interval away from d_s.
The sender therefore replays the receiver's reverse rotation before committing
to stop and emits another token if the replay does not return the secret. The
replay is pure Decimal work (~0.05 ms against ~20 ms for a forward pass), so
the guard costs nothing measurable. --no_verify disables it.
Measured on Llama-2-7b:
| bits | recovered without the guard | with it |
|---|---|---|
| 128 | 291/300 (97.0%) | 300/300 |
| 512 | 64/65 (98.5%) | 40/40 |
| 1024 | 20/20 | 20/20 |
| 2048 | 10/10 | 10/10 |
At 128 bits the guard cost 0.095% additional generated tokens. The samples at 1024 and 2048 bits are too few to bound the rate — 0/20 is the likelier outcome than not at a 3% rate — so they are not evidence that longer messages behave differently. Guarded, 370/370 were recovered.