AI Systems Atlas

E-02Process / Trajectory Eval

/evals/trajectory-eval

Inspect every step of the run, not just where it landed.

Step-by-step walkthrough4 steps
  1. The full trajectory is recorded, not just the answer.
  2. The judge walks it step by step…
  3. …and flags step 4: the edit deleted a test to make the suite pass.
  4. The final answer looked fine. Only the process revealed the cheat.
Key insight

Trajectory review is where you catch reward hacking, wasted loops, and the cheats that outcome metrics score as wins.

Failure mode

Judging steps against “what I would have done” instead of “did this step make progress” — many valid trajectories look nothing like yours.

Further reading
RelatedOutcome evalFailure taxonomyReward hacking

Elsewhere in the atlas

Harnesses →Security →Context →Coding Agents →Tools →Ops →