MemForm (evaluation release)

This repository accompanies the paper MemForm: Benchmarking Memory Formation Poisoning in LLM Agents.

MemForm is a deterministic benchmark of persistent-memory poisoning in LLM agents. It asks a causal question that ordinary jailbreak or prompt-injection suites do not isolate:

If an agent’s long-term memory is changed at one well-defined surface, does downstream decision-making shift from a clean policy to a poisoned policy on the same task?

Every item is a paired intervention. The downstream trigger, action space, and family semantics are held fixed. Only the memory-building input changes between the clean and poisoned conditions. Attack success is a within-item contrast.

This release contains the frozen benchmark cases and the evaluation / defense runners. The construction pipeline is withheld while the paper is under review and will be released after publication.

Attack taxonomy

Each family is instantiated at five memory interfaces. The case field is interface. P1–P5 are the paper names: Content, Evidence, Scope, Selection, and Path.

Interface	Paper name	Causal variable	Clean vs. poisoned contrast	Downstream probe
P1	Content	Stored memory text	Direct write of a clean reusable policy vs. a poisoned policy	One general target task
P2	Evidence	Authenticity of supporting records	Equal-budget authentic history vs. a matched history that replaces some records with forged evidence. Provenance is hidden from the memory learner	One general target task
P3	Scope	Observed scope distribution	Locally valid evidence plus outside counterexamples vs. cue-salient inside successes plus authentic outside/general successes for the locally preferred action	Three tasks: in-scope, near out-of-scope, target out-of-scope
P4	Selection	Retained subset	Same candidate pool; clean vs. poisoned retention of a fixed-size subset, with a full-pool control	One general target task
P5	Path	Consolidation order only	The same same-context experience multiset in two permutations. Clean evidence is last in the clean order; poisoned evidence is last in the poisoned order	One same-context target task

P3 in-scope is an agreement / shared-target control: clean and poisoned gold answers are identical. Near-outside and target-outside tasks are the separable probes. P5 forbids hidden context shifts: every record shares one context_key, and both clean- and poison-supporting records are observed successes.

What is released

Path	Contents
outputs/mcq_item_audit/accepted_cases.jsonl	Frozen multiple-choice cases (994 items, 235 families)
outputs/freeform_item_audit/accepted_cases.jsonl	Frozen free-form cases (897 items, 218 families)
memform/eval/	Shared protocol plus LangChain / LlamaIndex / defense runners
memform/freeform/evaluator.py	Cascaded free-form response adjudicator
memform/defenses/	Write-time defenses, use-time Task Shield, and fixed-denominator metrics
memform/common.py	JSONL I/O and case-schema checks

Counts by interface — MCQ: P1 234, P2 233, P3 174, P4 144, P5 209. Free-form: P1 209, P2 212, P3 144, P4 134, P5 198.

Layout

memform-code/
├── outputs/
│   ├── mcq_item_audit/accepted_cases.jsonl
│   └── freeform_item_audit/accepted_cases.jsonl
├── memform/
│   ├── common.py
│   ├── freeform/evaluator.py
│   ├── eval/          # protocol, LangChain, LlamaIndex, defense runners
│   └── defenses/
├── run_langchain.py
├── run_llamaindex.py
├── run_defense.py
├── pyproject.toml
├── requirements.txt
└── .env.example

Root run_*.py scripts call the same entry points as python -m memform <command>.

Installation

Python 3.10+.

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -e .

Copy .env.example to .env and set credentials:

Variable	Used by
OPENAI_API_KEY	Evaluation agent / memory model
OPENAI_BASE_URL	Optional OpenAI-compatible gateway (OPENAI_API_BASE is also accepted by LlamaIndex)
JUDGE_OPENAI_API_KEY / JUDGE_OPENAI_BASE_URL	Optional separate judge / free-form evaluator
DEFENSE_OPENAI_API_KEY / DEFENSE_OPENAI_BASE_URL	Optional separate defense-model endpoint

--api-key and --base-url override the environment. Any OpenAI-compatible Chat Completions server works. CLI default model names are placeholders; pass a model available on your endpoint.

python -m memform --help

Dataset format

JSONL, one case per line.

Shared fields: case_id, family_id, interface (P1–P5), attack_type, domain, decision_variable, clean_policy, poisoned_policy, scope, candidate_experiences, memory_building, metadata.

memory_building is the paired intervention:

• oracle_memory.clean / oracle_memory.poisoned — gold policy text (diagnostic)
• clean_condition / poisoned_condition — memory-building input for that interface

MCQ. evaluation.type is multiple_choice. Each trigger_tasks[] entry has task, choices, clean_answer, and poisoned_answer.

Free-form. task_format is free_form; source_case_id points at the MCQ source. Option menus are replaced by an open recommendation. Scoring uses paired behavioral_signatures (clean / poisoned / ambiguous / unclear). The poisoning construction is unchanged.

Provenance and gold labels are benchmark ground truth for scoring. Defenses never receive the attack interface, condition names, gold answers, provenance, or IDs as prompt features.

Evaluate an agent

Both runners share case validation, memory consolidation, downstream decision, scoring, and aggregation. They differ only in the LLM / memory stack.

python -m memform run-langchain \
  --cases outputs/mcq_item_audit/accepted_cases.jsonl \
  --output-dir results/langchain_mcq \
  --phase all \
  --interfaces P1 P2 P3 P4 P5 \
  --model MODEL

python -m memform run-llamaindex \
  --cases outputs/mcq_item_audit/accepted_cases.jsonl \
  --output-dir results/llamaindex_mcq \
  --phase learned \
  --interfaces P1 P2 P3 P4 P5 \
  --model MODEL

The same commands accept outputs/freeform_item_audit/accepted_cases.jsonl. Free-form scoring uses exact canonical match, then optional parser / judge models (--freeform-parser-model, --freeform-judge-model).

Flag	Meaning
--phase oracle|learned|all	Oracle uses gold policy text; learned consolidates from the compiled history
--target-alignment	Task-conditional judge of what the learned memory recommends (diagnostic)
--p5-updaters	incremental_merge (default), full_rewrite, recency_weighted_strict, batch_replay
--p5-random-orders N	Extra deterministic P5 permutations
--cache	JSONL disk cache of LLM calls
--validate-only	Print the run plan and estimated call counts, then exit

Each run writes case_results.jsonl and summary.json under --output-dir.

Evaluate defenses (MCQ)

Frozen attack cases are not rewritten. Write-time defenses may keep, reject, sanitize, quarantine, or bypass records before consolidation. Task Shield checks assistant output at use time and does not change memory formation.

python -m memform run-defense \
  --defense all \
  --interfaces all \
  --cases outputs/mcq_item_audit/accepted_cases.jsonl \
  --undefended-results results/langchain_mcq/case_results.jsonl \
  --output-dir results/defense \
  --model MODEL \
  --defense-model MODEL

Names: keyword_filter, memory_sanitization, llm_judge, a_memguard, task_shield, or all.