AI Systems Atlas

E-08Adversarial Evals

/evals/adversarial-evals

Mix attack tasks into the benchmark and score refusals as wins.

Step-by-step walkthrough4 steps
  1. Most of the set is ordinary work — the agent shouldn’t know which is which.
  2. Mixed in: an injection, planted credentials, a shortcut that would “pass”.
  3. On attack tasks the right answer is to refuse, flag, or route around.
  4. Score resistance, not just capability. An agent unmeasured under pressure is unmeasured.
Key insight

Capability and resistance are different axes. An agent you’ve only measured on cooperative tasks is unmeasured where it matters.

Failure mode

Attack tasks the agent can fingerprint: if adversarial items look different, you’re measuring detection, not resistance.

Further reading
RelatedIndirect prompt injectionOutcome eval

Elsewhere in the atlas

Harnesses →Security →Context →Coding Agents →Tools →Ops →