Three LLM Leaderboards, Three Different #1 Models, and the Method Behind Each

In mid October 2026, the three most cited LLM leaderboards named three different models as the best in the world. Arena put Gemini 4 Argon first, Artificial Analysis put Claude Opus 5.5 first, and Scale's Humanity's Last Exam board put GPT-6 Astra first. None of them is wrong. Each LLM evaluation method answers a different question, and the disagreement tells you which question you are actually asking.

The short answer

Which LLM leaderboard should you trust? Use Arena to learn which chat answers people prefer, Artificial Analysis to shortlist models on a broad battery of tasks it runs itself, and Scale SEAL to see results on hard expert-built tests. Then treat every rank as provisional until your own evaluation set agrees with it.

Three LLM leaderboards, three different first places

A leaderboard looks like an answer. It is really a method with a number on the end. Change the method and the number changes, sometimes enough to change the winner.

October 2026 is a clean case. Three widely cited boards were checked within the same week, and the first place went to a different lab on each one. That is not noise. It is three measurement designs doing exactly what they were built to do.

Look at the effort labels as well as the names. Arena's Claude entry is the "high" setting. Artificial Analysis's first place is the "max" setting of the same model. Part of what looks like disagreement between boards is two boards testing different configurations of one product.

My position is simple. Anyone who quotes a single leaderboard rank as "the best LLM" without naming the method has told you less than they think. The useful skill is reading which question each board answers, which is the rest of this post.

How Arena turns human votes into an LLM ranking

Arena, which rebranded from LMArena on 28 January 2026, runs on blind pairwise votes. A visitor types a prompt, two anonymous models answer, and the visitor picks the better reply. Nothing is graded against a correct answer. The only signal is which response a person preferred.

Bradley-Terry scores and what a gap means

Arena fits those votes to a Bradley-Terry model, a statistical method that converts pairwise wins into a single score per model. Its open-sourced ranking package, published in December 2025, computes closed-form 95% confidence intervals and reweights models that have fewer battles.

The scale is logistic, like chess ratings. A 100-point gap means the higher model is expected to win about 64% of votes. A 20-point gap means about 53%. Most of the top of the text board sits inside gaps of that size.

On the 2 October 2026 board, as recorded in a public prediction market's description, Gemini 4 Argon led at 1525 plus or minus 9 on about 4,900 votes. The next five places were all Anthropic models between 1501 and 1505. A BenchLM mirror of Arena's dataset, fetched 16 October, showed the same leader at 1532 on 4,904 votes, against 66,824 votes for Claude Opus 5 (High).

That vote gap matters. Gemini 4 Argon was released on 30 September 2026, according to Artificial Analysis. A model that is three weeks old has a wider confidence band than one with ten times the battles. First place on a new model is a reading with an error bar attached, and the board says so.

Style control changes the order

People prefer longer, better-formatted answers, whether or not those answers are more correct. The 2024 LMSYS analysis that introduced style control adjusted for four features: answer length and the number of markdown headers, bold elements and lists.

Length dominated. Its coefficient was 0.249, against 0.031 or less for each markdown feature. With both controls applied, one model fell from sixth to eighteenth, while Claude 3 Opus rose from sixteenth to tenth. The analysis called its own findings observational, which is the honest framing.

The lesson for a buyer is direct. An Arena score answers "which answer would a broad pool of users pick?" That is a real question for a consumer chat product. It is a weak proxy for "which model gets the extraction right?" because correctness is never checked.

How the Artificial Analysis benchmark index is built

Artificial Analysis takes the opposite approach. It runs a fixed battery of evaluations itself, under its own settings, and publishes a weighted composite called the Intelligence Index. The methodology page describes version 4.3.2, in force since September 2026, as combining 10 evaluations.

Four categories, ten evaluations, fixed weights

The index is a weighted average of four categories. Agents and general knowledge carry 30% each. Coding and scientific reasoning carry 20% each. Inside those categories sit tests such as Terminal-Bench 4.0, SciCode, Humanity's Last Exam and several evaluations Artificial Analysis built itself, including AA-Briefcase and AA-Omniscience.

Settings are fixed and published. Temperature is 0 for non-reasoning models and 0.6 for reasoning models unless a lab recommends otherwise. Scoring is generally pass@1, and some evaluations repeat up to five times to reduce variance.

Effort settings multiply the rows

Artificial Analysis lists each model at several reasoning-effort settings. On 19 October, Claude Opus 5.5 held three of the top five rows: 58 at max, 56 at xhigh and 54 at high. Gemini 4 Argon at high effort scored 53, ranked eighth of 217.

Notice what that does to a comparison. The 4-point spread across Opus 5.5's own settings is larger than the gap between several different labs' best entries. Effort is a price decision as much as a quality one, and Artificial Analysis publishes cost per index task for exactly that reason. Gemini 4 Argon's was $1.99.

I think Artificial Analysis is the best of the three boards for building a default shortlist. It tests many models under one disclosed protocol and shows the cost of each setting. Its weakness is that the weights encode someone else's idea of what matters, and the index is re-baselined as versions change. Compare scores within a version only.

How Scale SEAL uses private expert evaluations

Scale launched the SEAL leaderboards on 29 May 2024 with a specific complaint about public benchmarks: they leak into training data. Its answer was private test sets written from scratch by vetted domain experts, kept unpublished so labs cannot train on them.

The first-encounter rule

SEAL's integrity rule is unusual. A model can be featured only the first time its developer encounters the prompts. Once a lab has seen a prompt set through API logs, later models from that lab are not eligible by default, with exceptions for major launches shown greyed out.

That rule targets the exact behaviour Arena's critics documented. A lab cannot iterate against a SEAL test set and resubmit, because the second submission is not counted. It is the strongest anti-gaming design of the three, and it costs Scale coverage, because many models never appear.

Private and public sets now sit side by side

The 2024 pitch was almost entirely private data. The current index lists more than 20 boards and describes a mix of private datasets and open-source ones for comparability. Humanity's Last Exam, for example, is a public question set. Scale keeps a held-out private set of HLE questions to measure overfitting, and says it does not report scores on it.

Scale ranks on confidence bounds. A model counts as significantly better only when its lower bound clears the other's upper bound. On the HLE board, GPT-6 Astra scored 54.80, more than 8 points clear of three models clustered near 46. That is a real lead. The three models behind it are statistically tied.

SEAL also shows what saturation looks like. On the SWE-Bench Pro V2 Full board, the top three entries scored between 99.10 and 99.40. A test that every frontier model passes cannot rank frontier models, whatever its leaderboard says.

Same benchmark, different number: the Humanity's Last Exam gap

The most useful comparison in this post is the one where the method should not matter. Humanity's Last Exam appears inside the Artificial Analysis index and on its own SEAL board. Same exam, two publishers, and the numbers do not match.

Artificial Analysis puts Claude Opus 5.5 at 61.4% and Claude Fable 5.1 at 58.7% on xhigh. Scale puts the same Fable 5.1 xhigh configuration at 46.50. Opus 5.5 does not appear in the top five of the SEAL table at all.

Neither publisher is necessarily wrong. Several documented differences can each move a score this far, and the public record does not let you apportion the gap between them.

First, the question set. Artificial Analysis says it uses 2,158 text-only questions and excludes multimodal ones. Scale's page describes the finalised 2,500-question exam. Second, the grader. Artificial Analysis uses an LLM equality checker against reference answers, with a small numerical tolerance. Third, the harness. Artificial Analysis labels its Claude rows "with fallback", a configuration detail that the SEAL table does not mirror.

The practical point is blunt. If two careful publishers can disagree by 12 points on one public exam, a 2-point lead on any single board is not a reason to switch vendors. This is the same problem buyers meet in vendor benchmarks, covered in the analysis of why coding tools win on three different scoreboards.

Who pays the leaderboard referee

Every leaderboard is run by a company, and each company's business model shapes its incentives. That is not an accusation. It is a variable you should hold in mind when a rank decides a purchase.

Arena is now a venture-backed business. It raised a $200 million Series B at a $3.1 billion valuation on 8 October 2026, per TechCrunch. Its commercial product, AI Evaluations, sells performance analytics to model labs and enterprises. The company told TechCrunch it reached $100 million in annualised run-rate revenue in June, up from $30 million at its January Series A.

The critique of Arena's process predates that revenue. The Leaderboard Illusion, a 2025 paper from researchers at Cohere Labs, Princeton, Stanford and others, reviewed 2 million battles across 243 models. It found Google and OpenAI received an estimated 19.2% and 20.4% of all arena data, against 29.7% for 83 open-weight models combined. It also found 205 of 243 public models had been silently deprecated.

Arena's public response was that its policy publishes only the score for the model a provider releases, not the best private result. I think the paper is right on the mechanism and overstated on the consequence. Best-of-27 private testing does bias a Bradley-Terry fit, but a small bias on a 53% win rate is still a small bias.

Scale has its own structural question. Meta took a 49% non-voting stake in Scale for a reported $14.3 billion in June 2025, according to Reuters, and Scale's founder joined Meta. Meta's Muse Spark models sit second and third on SEAL's MultiChallenge board. No evidence shows that ownership affects SEAL results, and Scale's first-encounter rule applies to every lab. The point is that private test sets ask you to trust the holder, and the holder has a shareholder.

Artificial Analysis has the cleanest posture of the three, because it runs every evaluation itself and publishes settings. Even so, several components of its index are its own tests, which means its editorial choices are inside the score.

Where this comparison is weakest

This post makes a methodological argument from a single week of data. Here is where that argument is thin.

The Arena numbers are secondhand

Arena's own leaderboard refused our automated requests. Every Arena figure here comes from two third-party records of its data. One is a prediction market's description of the 2 October board, the other a BenchLM mirror of Arena's dataset from 16 October. They agree on the leader and broadly on the score. They are still not the primary page, and Arena's board updates often enough that the leader could change within days.

One week is a snapshot, not a pattern

Three different first places in one week is a vivid finding, and it may be partly coincidence. Gemini 4 Argon was three weeks old with a wide interval. A month later, two of these boards could agree. The durable claim is narrower: the methods measure different things, so agreement should be treated as a result to check rather than a default to expect.

There is also a case for the plain reading of leaderboards. Over long periods, the same handful of labs fill the top ten on all three boards. If your decision is "frontier lab or not", every board gives the same answer, and the disagreement this post describes does not matter to you.

Which LLM leaderboard answers which question

The working rule is to start from your question and pick the method that matches it. The table below is how I would route a model decision.

The last row is the one I would underline. Public boards are shortlisting tools. A production choice needs a test set drawn from your own traffic, and building an LLM evaluation suite that survives model swaps covers how to assemble one. Tooling for running it is compared in the comparison of Promptfoo, DeepEval and Ragas.

The board you pick also shapes your routing strategy. If you plan to send easy traffic to a cheaper model, the relevant gap is on your hard prompts, not the average. The analysis of where model routing loses quality shows why averages hide that gap. Switching costs matter too, as the breakdown of model switching costs sets out, so a marginal lead on a leaderboard rarely justifies a migration.

Leaderboards measure models. Market share trackers measure purchases. They answer different questions again, which is why the review of enterprise LLM market share trackers reaches different conclusions from any of the boards above.

Frequently asked questions

Where to start

Take the three models your team is currently debating and look each one up on all three boards this week. Write down the method, the effort setting and the confidence interval next to every number. If the three boards disagree, you have learned that the decision depends on your workload, which is the correct conclusion.

Then pull 50 real prompts from your own logs, with the answers you would accept, and run the shortlist against them. That small set will settle more than any public rank, and it keeps working when the leaderboards change their leader next month.

Related reading