AI Systems Atlas

S-07Memory Poisoning

/security/memory-poisoning

An attacker plants content that gets written to long-term memory — and every future session inherits it as fact.

Step-by-step walkthrough2 variants · 7 steps

Attack

  1. Attacker-controlled content flows through an ordinary session…
  2. …and gets written to long-term memory like any other “fact”.
  3. Weeks later a fresh session recalls it — stripped of where it came from.
  4. One injection, every future session. Memory outlives the attack.

Defended

  1. Writes are gated: unverified claims from untrusted sources rarely qualify.
  2. What does get stored carries its provenance forever.
  3. A later session sees “source: untrusted web” and treats it as a claim, not a fact.
Key insight

Memory is an injection that persists. Provenance is the antidote: a “fact” that remembers it came from an untrusted webpage can be treated accordingly, forever.

Failure mode

Provenance-free stores: once written, attacker text and user truth are indistinguishable.

Further reading
RelatedWrite policies & decayIndirect prompt injection

Elsewhere in the atlas

Harnesses →Evals →Context →Coding Agents →Tools →Ops →