AI Systems Atlas

Context Engineering & Memory

The model sees exactly one thing: the context window. Context engineering is deciding what earns a place in that finite budget, in what order — and memory is everything you keep outside it, plus the machinery for bringing the right pieces back.

Index X-01 Anatomy of a Context WindowX-02 Context CompactionX-03 Retrieval & RAGX-04 What Should Enter Context?X-05 Context OverflowX-06 Context RotX-07 Sliding WindowX-08 The Four MemoriesX-09 Write Policies & DecayX-10 State RepresentationX-11 Cache-aware Context LayoutX-12 Observation CompressionX-13 Files as External ContextX-14 Subagent Context Isolation

X-01Anatomy of a Context Window

/context/context-window-anatomy

Everything the model will ever know about this moment shares one finite window.

Key insight

Instructions, tools, memory, retrievals and history all compete for the same budget. Most of a mature agent’s window is spent on its own past actions and observations.

Failure mode

Treating the window as infinite until the day it isn’t: overflow behavior chosen by accident (truncation) instead of by design (compaction).

Further reading
RelatedContext compactionContext budget

X-02Context Compaction

/context/context-compaction

When the window fills, compress the trajectory into a summary plus explicitly preserved state.

Step-by-step walkthrough2 variants · 10 steps

Careful

  1. A long trajectory: reads, edits, failures, and two easily-missed critical facts.
  2. The window is nearly full. Something has to go.
  3. The compactor reads everything…
  4. …emits a short summary plus explicitly preserved state.
  5. 8.4k → 1.9k tokens. The constraint and the key path survived by design, not luck.

Naive

  1. Same trajectory, same two critical facts buried in it.
  2. A generic “summarize this” pass compresses by salience — and constraints aren’t salient.
  3. The summary preserves the gist.
  4. The constraint and the key path silently vanish.
  5. Twenty steps later the agent violates a rule it no longer knows exists.
Key insight

Compaction is a write policy, not a summary: decide what must survive (constraints, decisions, open questions, current diffs) and preserve it verbatim — summarize only the rest.

Failure mode

Salience-based summarization drops exactly the things that matter most: user constraints and one-line facts are rarely “salient”, until they’re violated.

Further reading
RelatedLong-running agentContext budget

X-03Retrieval & RAG

/context/retrieval

Cast a wide net with similarity search, then rerank hard, so only genuinely relevant content spends context budget.

Step-by-step walkthrough4 steps
  1. The query becomes a vector; similarity search casts a wide net.
  2. Candidates are recalled — similar is not the same as relevant.
  3. A reranker reads each candidate against the actual question.
  4. Only the survivors spend context budget.
Key insight

Recall is cheap; precision is what protects the window. The reranker — which actually reads the candidates — is where quality comes from.

Failure mode

Stuffing top-20 raw chunks into context: similar-but-wrong documents don’t just waste tokens, they actively mislead the model.

Further reading
RelatedContext budgetMalicious retrieved content

X-04What Should Enter Context?

/context/context-budget

Every candidate item has a cost; toggle what goes in and watch the budget respond.

Key insight

Almost every “too big for the window” problem is really a selection problem: the compact alternative usually carries 90% of the signal at 15% of the cost.

Failure mode

Additive thinking — everything helpful goes in. Helpful-but-diluting content crowds out the few tokens that were essential.

Further reading
RelatedAnatomyRetrievalCompaction

X-05Context Overflow

/context/context-overflow

The window fills mid-task; without a plan, the harness truncates — usually the oldest, often the most important.

Step-by-step walkthrough4 steps
  1. The window is nearly full and the task isn’t done.
  2. A large observation arrives. Something must go.
  3. Default truncation evicts the oldest — the system prompt and the user’s constraint.
  4. “Staging only” is gone. The agent no longer knows it ever existed.
Key insight

Overflow behavior is an architectural decision that most systems make by accident. Decide it before you need it: what gets evicted, what is untouchable.

Failure mode

Oldest-first truncation: the system prompt and the user’s constraints are, by definition, the oldest things in the window.

Further reading
RelatedContext compactionContext budget

X-06Context Rot

/context/context-rot

Stale errors, dead ends and duplicates dilute attention long before the window is full.

Step-by-step walkthrough4 steps
  1. Early on, the key fact dominates a small context.
  2. Failed attempts and duplicates accumulate — none of them deleted.
  3. The key fact is still there — and the model increasingly acts as if it isn’t.
  4. Quality degrades well before the limit. Prune actively, don’t just wait for overflow.
Key insight

Context quality is not the same as context quantity. The key fact can still be in the window and functionally invisible.

Failure mode

Treating the token limit as the only budget: quality degrades continuously, not at a threshold.

Further reading
RelatedContext overflowSliding window

X-07Sliding Window

/context/sliding-window

Keep the last N turns verbatim; summarize everything older into a rolling digest.

Step-by-step walkthrough3 steps
  1. Turns age out of the verbatim zone into a rolling summary.
  2. The model always sees recent turns exactly, older ones compressed.
  3. Simple and predictable — but importance-blind: a critical fact ages out like any other turn. Pair it with pinned facts.
Key insight

The cheapest compaction strategy that works — predictable cost, predictable behavior, one prompt to maintain.

Failure mode

Importance-blindness: a critical fact ages out exactly like small talk. Pair it with pinned, never-evicted facts.

Further reading
RelatedContext compactionContext rot

X-08The Four Memories

/context/memory-types

Working, episodic, semantic, procedural — four stores with different lifetimes, indexes, and write rules.

Step-by-step walkthrough6 steps
  1. Four kinds of memory, four different lifetimes. Click any store to inspect it.
  2. Working — in the window, gone when the task ends.
  3. Episodic — “what happened”, indexed by time and situation.
  4. Semantic — “what is true”, updated as evidence changes.
  5. Procedural — “how to do it”, loaded when the task matches.
  6. Confusing them is the classic bug: storing episodes as facts, or facts as one-off events.
Key insight

Each type earns its own machinery: working memory lives in the window; the rest live outside it and come back through retrieval, each indexed differently.

Failure mode

One undifferentiated “memory” blob: episodes stored as facts, preferences stored as events, and retrieval that can’t tell them apart.

Further reading
RelatedWrite policies & decayRetrieval

X-09Write Policies & Decay

/context/memory-write-policies

What earns a memory write — and what is allowed to fade.

Step-by-step walkthrough4 steps
  1. Not everything observed deserves a write.
  2. The gate asks: durable? reusable? confirmed? — write too much and retrieval drowns.
  3. Stored memories that go unused fade on a schedule.
  4. Forgetting is a feature: never-forgotten errors become permanent errors.
Key insight

The write gate is the quality control for every future retrieval. Write too much and search drowns; decay is how wrong memories stop being permanent.

Failure mode

Write-everything, forget-nothing: a year later, retrieval surfaces confident notes from a codebase that no longer exists.

Further reading
RelatedThe four memoriesMemory poisoning

X-10State Representation

/context/state-representation

How task state is written down decides whether it survives compaction, restarts, and model swaps.

Step-by-step walkthrough2 variants · 6 steps

Prose

  1. State recorded as loose prose — already hedged, already vague.
  2. Every compaction paraphrases the paraphrase.
  3. Three summaries later, the file name is gone and the step count never existed.

Structured

  1. State recorded as an explicit object with a schema.
  2. Compaction has a rule: structured state is preserved, not summarized.
  3. Byte-identical on the other side. Schema beats vibes.
Key insight

Structured state (JSON, checklists, diffs) can be copied verbatim through every compression; prose has to be re-understood every time, and drifts.

Failure mode

Narrative state: each summarization paraphrases the last paraphrase until file names and step counts dissolve.

Further reading
RelatedContext compactionCheckpointing

X-11Cache-aware Context Layout

/context/cache-aware-layout

Order the window by how often each part changes, because a prompt cache is a prefix match and the first mutation invalidates everything after it.

Step-by-step walkthrough7 steps
  1. The window is not a bag of content. It is an ordered sequence, and order is the whole lever.
  2. Sort it by how often each part changes: never, rarely, every turn.
  3. Only the tail moves between turns.
  4. So the long, expensive prefix is served from cache — a large fraction of the tokens, at a fraction of the price and the latency.
  5. Now put a timestamp in the system prompt, or let the tool list come back in a different order.
  6. Everything after that byte is re-read. The cache is a prefix match, not a similarity match.
  7. This is why compaction is expensive twice: it rewrites the middle, so the whole tail after it pays again.
Key insight

This reframes half of this collection: compaction, sliding windows and memory loading are not only token decisions, they are cache decisions. A rewrite in the middle makes the whole tail after it expensive again.

Failure mode

A timestamp, a session id or an unsorted tool list in the system prompt. One volatile byte at the front and the cache never hits — for every request, forever.

Further reading
RelatedAnatomy of a context windowContext compactionSliding windowCost monitoring

X-12Observation Compression

/context/observation-compression

Shape tool output before it enters the window: the errors, the tail, the counts, and a pointer to the rest.

Step-by-step walkthrough6 steps
  1. A single test run returns twelve thousand tokens. Most of it is “ok, ok, ok”.
  2. It does not go straight into the window. Something between the tool and the context decides what matters.
  3. Keep the failures — the whole reason the agent ran this.
  4. Keep the tail, where stack traces and summaries live, and the counts that say how bad it is.
  5. Keep a pointer to the full log. Elision the model can see through beats elision it cannot.
  6. Six hundred tokens carry the signal of twelve thousand — and the loop can afford another twenty steps.
Key insight

Previous actions and observations are the largest slice of a mature agent’s window — bigger than the system prompt, the tools and the retrievals combined. Shaping them is the highest-leverage context work available, and it is harness code, not prompting.

Failure mode

Blind truncation to a character limit. It cuts mid-traceback and keeps the passing tests, which is the exact inverse of what the agent needed.

Further reading
RelatedAnatomy of a context windowContext budgetTest loopDesigning tools

X-13Files as External Context

/context/files-as-context

The agent writes findings to disk and reads back the section it needs, so the window holds pointers instead of everything learned.

Step-by-step walkthrough6 steps
  1. A long task learns far more than the window can hold.
  2. So the agent writes findings out as it goes, instead of carrying them.
  3. The plan lives on disk too — a checklist it can tick, not a paragraph it has to remember.
  4. When a step needs a fact, it reads that section back. Search the file; don’t reload it.
  5. The window ends up holding pointers and the file currently in hand.
  6. And none of it is lost to compaction — the summary can drop a detail the file still has, verbatim.
Key insight

The filesystem is memory the model can address, and unlike the window it is not lossy. A note written in step 3 is still exact in step 40, after every compaction in between.

Failure mode

Writing notes and never reading them. Without a habit of searching its own files first, the agent re-derives what it already wrote down and pays twice.

Further reading
RelatedState representationContext compactionCheckpointing

X-14Subagent Context Isolation

/context/subagent-isolation

Delegation as a context move: the subagent spends a fresh window on the search and returns only the answer.

Step-by-step walkthrough7 steps
  1. The parent has a plan and a nearly empty window. It intends to keep it that way.
  2. It needs an answer that costs a lot of reading to find. It delegates the question, not the work.
  3. The subagent opens forty files and burns ninety thousand tokens finding out.
  4. It returns eight hundred tokens: the answer, the files that matter, the traps.
  5. Only the digest crosses back. The parent’s window grew by a paragraph, not by a codebase.
  6. Everything else is discarded with the subagent’s window — which is exactly the point, and also the risk.
  7. What the subagent didn’t write down, the parent can never ask about. Delegation is lossy by construction.
Key insight

Read the org chart as a context diagram. The parent pays for the digest, not the exploration — which is why a subagent is worth it even when a single agent could have done the work.

Failure mode

A digest that omits what the parent didn’t know to ask for. Everything not written down dies with the subagent’s window, and the parent cannot tell the difference between “no traps” and “traps not mentioned”.

Further reading
RelatedHierarchicalResearcher + ImplementerContext budgetAgent handoffs

Elsewhere in the atlas

Harnesses →Security →Evals →Coding Agents →Tools →Ops →