AI Systems Atlas

S-20Denial of Wallet

/security/denial-of-wallet

An attacker — or a bug — drives an agent into endless loops, giant contexts or wide fan-out until the money or the shared quota runs out.

Step-by-step walkthrough2 variants · 12 steps

Attack

  1. One request arrives — crafted to be endless, or simply a task the agent cannot finish.
  2. The loop never converges. Nothing in the harness counts the retries.
  3. Every turn re-sends a context that only grows, so each iteration costs more than the last.
  4. It fans out: two hundred subagents, each running the same unbounded loop.
  5. Cost is tokens × turns × agents, and all three are growing at once.
  6. By morning the budget is spent and the shared quota is gone — an outage bought one token at a time.

Defended

  1. The same crafted task arrives. You cannot stop requests from being expensive to answer.
  2. Every step passes a governor before tokens are spent — the harness meters, not the model.
  3. It enforces ceilings per run and per user, so one caller cannot spend everyone’s quota.
  4. Fan-out is capped: the agent asked for two hundred subagents and got four.
  5. The same call with the same arguments comes round a third time. Loop detection trips.
  6. The run stops and says why — worst case is the run budget, not the month’s.
Key insight

An agent’s cost is tokens × turns × agents, and an adversary can push all three from one request. Enforce ceilings per run and per user in the harness, because the model has no idea what anything costs.

Failure mode

An organization-wide spend cap as the only limit. It fires once the month’s budget is gone, and then it takes every other user down with the one runaway run.

Further reading
RelatedBudgets & stopping rulesCost & latency as scoresAudit logs

Elsewhere in the atlas

Harnesses →Evals →Context →Coding Agents →Tools →Ops →