owasp/the-agents-epistemic-baggageAn audit-oriented risk model for how agent-generated inferences can change epistemic status, gain operational authority, and influence later decisions through persistent memory.
An audit-oriented risk model for how agent-generated inferences can change epistemic status, gain operational authority, and influence later decisions through persistent memory.
An agentic AI system with writable persistent memory does not only retrieve previous information. It may also store its own summaries, conclusions, plans, and execution histories.
This creates a governance question:
How does the system represent and preserve the distinction between received information and agent-generated inference?
If that distinction is lost, an inference may acquire or retain a status that its evidence does not justify. Such a record may then influence decisions or actions, even through policy-permitted use. Unauthorized authority escalation or use beyond permitted authority is a separate control failure that the project also examines.
This repository develops a risk model for this drift process, called self-reinforcing epistemic drift, and explores how to make it auditable. The full argument, including the memory schema, drift mechanisms, and lifecycle test protocol, is developed in The Agent's Epistemic Baggage. A first implementable specification, typed audit objects, transition rules, controls, metrics, and a reproducible test, is developed in Agent Memory Drift: Audit Operationalization v0.1. If you are about to run an audit rather than read the specification cover to cover, start with the User Guide instead.
The model applies to agentic AI systems with writable persistent memory, where agent-generated representations can be stored, retrieved, consolidated, and used to influence later inferences, tool use, decisions, or actions.
It focuses on the memory pathway of self-evolving agents. Changes to model weights, tools, and workflows have distinct risk profiles and are outside the current scope.
This is currently a conceptual risk model with an audit orientation. The drift mechanisms, audit schema, and lifecycle tests form the proposed core. An initial audit specification defines proposed control requirements, provisional metrics, and acceptance criteria. Implementation and empirical validation remain open work.
A write and retrieval loop can propagate agent-generated material recursively when four structural links hold. They are necessary but not sufficient for drift: the loop must also carry inaccurate, unsupported, misclassified, or epistemically degraded material.
Each link is also a potential control point.
Similarity-based retrieval is not required, but it can amplify the loop. Empirical work documents two components of this amplification: experience-following, where agents tend to produce similar outputs when a current task resembles a retrieved experience, and error propagation, where erroneous stored experiences are retrieved, imitated, and stored again.
Drift can also arise without a write and retrieval loop:
The corruption can occur during consolidation before the altered memory is retrieved. It acquires operational consequences when that memory is later used.
A single status label is insufficient. An auditable memory record should keep distinct dimensions separate:
| Field | Examples |
|---|---|
| Representation type | received input, sensor event, tool output, source claim, agent inference |
| Provenance | source, timestamp, tool, agent, link to underlying evidence |
| Validation state | untested, supported, contradicted, validated, rejected |
| Temporal validity | current, stale, expired |
| Evidence relation | records or sources that support or contradict the record, including the strength, independence, and quality of that support |
| Change history | created, summarized, reclassified, superseded |
| Epistemic transition basis | new evidence, corroboration, contradiction resolution, validated outcome |
| Authority basis | policy rule, risk classification, human approval, delegated permission |
| Usage and decision lineage | retrieval event, rank, retrieval score, prompt position, explicit reference in reasoning or output, consuming agent, affected decision or action |
Provenance establishes origin, not truth. A trusted source can be wrong, and an authenticated agent inference remains an inference. Representation type covers epistemic records only; a decision or an action does not carry a meaningful truth-based validation state, an action is authorized, executed, successful, or failed rather than supported or contradicted, so it is represented as a linked operational event rather than as a further representation type. It still carries its own provenance and event history, and its weight is assessed through its underlying evidence together with its usage and decision lineage.
Uncertainty should be grounded in evidence and validation state rather than relying solely on a model's self-reported confidence, which is why evidence relation also carries the strength, independence, and quality of the supporting material.
Validation state and temporal validity are kept separate because a record can remain validated while becoming stale. Temporal currency and superseded status are also independent: a record can remain temporally current while having been replaced by a newer version. Change history records that replacement.
Epistemic transition basis and authority basis are also kept separate: the first is the evidential or review basis for a change in validation state, the second is why a record was allowed to influence a decision. A policy approval does not make a claim truer, and new evidence does not by itself grant permission to act.
This is a logical audit schema, not a requirement that every field applies to every record type or that all records coexist in a single physical table: field applicability is conditional on representation type, and operational events can be linked to the epistemic records they consume.
Operational authority is not a single degree but a profile of four related, distinct variables: permitted authority, observed exposure, recorded use, and causal influence. It is a separate audit object from epistemic status: a well-supported record can still be granted more authority than policy allows, and a drifted record can still reach a decision through entirely policy-compliant authority, for example if policy grants influence to any record labeled validated. The risk compounds whenever a record with unearned epistemic status reaches a decision or action, whether that reach was policy-permitted or unauthorized.
Permitted authority may be created through instruction priority, tool permissions, mandatory-use rules, and downstream decision policies. Observed exposure is shaped by separate mechanisms such as retrieval ranking, retrieval frequency, and position in context; neither, by itself, establishes recorded use or causal influence.
