X-11Cache-aware Context Layout

/context/cache-aware-layout

Order the window by how often each part changes, because a prompt cache is a prefix match and the first mutation invalidates everything after it.

Key insight

This reframes half of this collection: compaction, sliding windows and memory loading are not only token decisions, they are cache decisions. A rewrite in the middle makes the whole tail after it expensive again.

Failure mode

A timestamp, a session id or an unsorted tool list in the system prompt. One volatile byte at the front and the cache never hits — for every request, forever.

X-11Cache-aware Context Layout

Elsewhere in the atlas