Benchmark definition

A standardized test used to compare models on a defined task.

What it actually is under the hood

A benchmark fixes a dataset, scoring rule, and often a prompt format. It measures performance on that setup, not every real-world use case.