E-01Outcome Eval

/evals/outcome-eval

Run the task end to end; check only the final result.

Key insight

Ground truth where you can get it — a passing test, a correct SQL result, a booked meeting — is worth a thousand vibes.

Failure mode

Outcome-only metrics reward lucky trajectories and hide near-misses; they tell you that it failed, never why.

Elsewhere in the atlas