AI Systems Atlas

S-11Unsafe Side Effects

/security/unsafe-side-effects

No attacker required: write access plus a misunderstanding equals irreversible damage.

Key insight

Design for the honest mistake, not just the adversary — dry-runs, approval on irreversibility, and soft-deletes turn catastrophes into undo operations.

Failure mode

Hard deletes as the default: the cheapest possible implementation of the most expensive possible failure.

Further reading
RelatedHuman-in-the-loopRetry loop

Elsewhere in the atlas

Harnesses →Evals →Context →Coding Agents →