AI Systems Atlas

Context Engineering & Memory

The model sees exactly one thing: the context window. Context engineering is deciding what earns a place in that finite budget — and memory is everything you keep outside it, plus the machinery for bringing the right pieces back.

Index X-01 Anatomy of a Context WindowX-02 Context CompactionX-03 Retrieval & RAGX-04 What Should Enter Context?X-05 Context OverflowX-06 Context RotX-07 Sliding WindowX-08 The Four MemoriesX-09 Write Policies & DecayX-10 State Representation

X-01Anatomy of a Context Window

/context/context-window-anatomy

Everything the model will ever know about this moment shares one finite window.

Key insight

Instructions, tools, memory, retrievals and history all compete for the same budget. Most of a mature agent’s window is spent on its own past actions and observations.

Failure mode

Treating the window as infinite until the day it isn’t: overflow behavior chosen by accident (truncation) instead of by design (compaction).

RelatedContext compactionContext budget

X-02Context Compaction

/context/context-compaction

When the window fills, compress the trajectory into a summary plus explicitly preserved state.

Key insight

Compaction is a write policy, not a summary: decide what must survive (constraints, decisions, open questions, current diffs) and preserve it verbatim — summarize only the rest.

Failure mode

Salience-based summarization drops exactly the things that matter most: user constraints and one-line facts are rarely “salient”, until they’re violated.

RelatedLong-running agentContext budget

X-03Retrieval & RAG

/context/retrieval

Cast a wide net with similarity search, then rerank hard, so only genuinely relevant content spends context budget.

Key insight

Recall is cheap; precision is what protects the window. The reranker — which actually reads the candidates — is where quality comes from.

Failure mode

Stuffing top-20 raw chunks into context: similar-but-wrong documents don’t just waste tokens, they actively mislead the model.

RelatedContext budgetMalicious retrieved content

X-04What Should Enter Context?

/context/context-budget

Every candidate item has a cost; toggle what goes in and watch the budget respond.

Key insight

Almost every “too big for the window” problem is really a selection problem: the compact alternative usually carries 90% of the signal at 15% of the cost.

Failure mode

Additive thinking — everything helpful goes in. Helpful-but-diluting content crowds out the few tokens that were essential.

RelatedAnatomyRetrievalCompaction

X-05Context Overflow

/context/context-overflow

The window fills mid-task; without a plan, the harness truncates — usually the oldest, often the most important.

Key insight

Overflow behavior is an architectural decision that most systems make by accident. Decide it before you need it: what gets evicted, what is untouchable.

Failure mode

Oldest-first truncation: the system prompt and the user’s constraints are, by definition, the oldest things in the window.

RelatedContext compactionContext budget

X-06Context Rot

/context/context-rot

Stale errors, dead ends and duplicates dilute attention long before the window is full.

Key insight

Context quality is not the same as context quantity. The key fact can still be in the window and functionally invisible.

Failure mode

Treating the token limit as the only budget: quality degrades continuously, not at a threshold.

RelatedContext overflowSliding window

X-07Sliding Window

/context/sliding-window

Keep the last N turns verbatim; summarize everything older into a rolling digest.

Key insight

The cheapest compaction strategy that works — predictable cost, predictable behavior, one prompt to maintain.

Failure mode

Importance-blindness: a critical fact ages out exactly like small talk. Pair it with pinned, never-evicted facts.

RelatedContext compactionContext rot

X-08The Four Memories

/context/memory-types

Working, episodic, semantic, procedural — four stores with different lifetimes, indexes, and write rules.

Key insight

Each type earns its own machinery: working memory lives in the window; the rest live outside it and come back through retrieval, each indexed differently.

Failure mode

One undifferentiated “memory” blob: episodes stored as facts, preferences stored as events, and retrieval that can’t tell them apart.

RelatedWrite policies & decayRetrieval

X-09Write Policies & Decay

/context/memory-write-policies

What earns a memory write — and what is allowed to fade.

Key insight

The write gate is the quality control for every future retrieval. Write too much and search drowns; decay is how wrong memories stop being permanent.

Failure mode

Write-everything, forget-nothing: a year later, retrieval surfaces confident notes from a codebase that no longer exists.

RelatedThe four memoriesMemory poisoning

X-10State Representation

/context/state-representation

How task state is written down decides whether it survives compaction, restarts, and model swaps.

Key insight

Structured state (JSON, checklists, diffs) can be copied verbatim through every compression; prose has to be re-understood every time, and drifts.

Failure mode

Narrative state: each summarization paraphrases the last paraphrase until file names and step counts dissolve.

RelatedContext compactionCheckpointing

Elsewhere in the atlas

Harnesses →Security →Evals →Coding Agents →