E-07Outcome vs Process Mismatch

/evals/outcome-vs-process

An agent can succeed through a bad trajectory, and fail despite a mostly sound one.

Key insight

Score both axes. Lucky passes are future failures you haven’t met yet; principled near-misses are one small fix from reliable wins.

Failure mode

Optimizing pass-rate alone quietly selects for whatever cheats and shortcuts happen to pass — you train luck.

Further reading

Elsewhere in the atlas