Competition is the best quality control
Human experts and AI models compete to most accurately label the same data. Only the best rise to the top, and the responses of both are combined using weighted aggregation to produce the highest quality datasets.
What's a Centaur Aggregation score? It's the accuracy-weighted consensus of every expert and model on a case, calibrated against ground truths and scored across every case in the contest. See it live →
01
A dataset opens as a contest
Some contests are open to all. The complex, regulated ones are gated behind verified credentials and background checks.
02
Experts and AI models compete
Everyone labels the identical cases and is graded on ground-truth answers. Where models enter, the leaderboard doubles as a benchmark.
03
Consensus becomes the label
Proven-accurate voices count more. The weighted consensus outperforms any single expert or model, and ships with the dataset.
Featured contests
Calorie Estimation
Meal photographs estimated for total calories by nearly a thousand competing labelers, with per-case estimates aggregated into a calibrated consensus value.
Sample records available under NDA
Leaderboard
GPT-5.2 trails the Centaur Aggregation by 11.0 points on this task.
Task prompt
How many calories do you think are in this meal?
Who was allowed in
The best data comes from where AI still loses
Every competition between frontier models and experts creates a real-time benchmark. The gap between the leading model and Centaur Aggregation pinpoints the hardest problems — generating the training signal needed to build better models.
Four ways into the Arena
Pick the one closest to what you need. We'll put you on the calendar with the right person on our team.
Buy a dataset
Any completed contest is a licensed dataset: records, accuracy-weighted aggregation labels, per-labeler provenance, and the model benchmark.
Benchmark your model
Your model labels the same cases as the expert field and lands on the leaderboard. See exactly where it trails the Centaur Aggregation.
Run a private arena
Your cases become a private contest. We recruit and gate the right specialists, enter the models you care about, and hand back aggregation labels.
Source data you don't have
Tell us the capability you're building toward. We source the raw data through our network of partners, recruit the expert field, and run the contest end to end.
A good starting point: data modality (image, text, audio, video), domain or specialty, estimated volume, and timeline.