AI Systems Atlas

H-07Best-of-N / Search

/harnesses/best-of-n

Sample several independent attempts in parallel and keep the one a judge scores highest.

Step-by-step walkthrough4 steps

One prompt, N independent samples — in parallel.
Attempts vary: temperature, seeds, even different models.
The judge compares all candidates side by side.
B wins. You paid 4× inference for one better answer.

Key insight

When attempts are cheap and judging is easier than generating, N tries plus a selector beats one careful try.

Failure mode

The judge is the ceiling: if it can’t tell good from bad, you pay N× inference to select noise.

Further reading

Self-Consistency Improves Chain of Thought Reasoning in Language Models Wang et al., ICLR 2023
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Snell et al., 2024

Elsewhere in the atlas