LLM as a Judge: What 9 Studies Say About GPT, Claude and Gemini as Graders

GPT-4 agreed with expert human graders 85% of the time in the 2023 paper that made LLM as a judge the default method for model evaluation. On a 2024 benchmark built from hard, objectively checkable questions, GPT-4o picked the correct answer 50.86% of the time, which is a coin flip. Both numbers are real. The gap between them is the whole case for knowing which tasks your judge can grade.

The short answer. LLM judges are reliable enough to rank chat responses on style, helpfulness and instruction-following, where the best models match human-human agreement. They are not reliable for grading factual or reasoning correctness without a reference answer. Always swap answer order, report kappa rather than raw agreement, and keep human review for hard and high-stakes items.

What an LLM as a judge score actually measures

LLM as a judge means using one language model to grade the output of another. It replaced human raters in most evaluation pipelines because it is fast, cheap and repeatable. The method only works if the judge's verdicts track what a careful human would say.

Every study in this post asks that one question in a different setting. The answer changes sharply with the setting, which is why a single "accuracy" number for any judge model is close to meaningless.

Pairwise grading and single-answer grading are different tests

In pairwise grading the judge sees two answers and picks the better one. In single-answer grading it assigns a score, often 1 to 10, to one answer with or without a reference. Pairwise grading exposes position bias. Single-answer grading exposes scale drift, where two judges agree on the ranking but disagree on the scores.

Most published agreement figures come from pairwise setups. If your pipeline scores single answers on a rubric, the headline numbers below are an upper bound on what you should expect, not a measurement of your system.

Agreement with humans is not the same as being right

Agreement measures whether the judge matches human raters. On open-ended chat prompts, humans themselves disagree often, so matching them is the best available standard. On questions with a checkable answer, agreement with humans can hide the fact that both the humans and the judge got it wrong.

JudgeBench was built around this distinction. Its labels reflect objective correctness rather than crowd preference, and that one design choice is what drops judge scores to near chance.

LLM judge agreement rates, study by study

The table below compiles every study we opened for this post, with what it tested and what it found. Each row links to the paper so you can check the method yourself. The judges tested are mostly 2023 to 2025 models, which is the first limitation to hold in mind.

Kappa values from Thakur et al. are reported in the paper on a 0 to 100 scale and are converted here. Rows are not directly comparable: each study uses its own tasks, prompts and agreement metric, which is the point the rest of this post makes.

MT-Bench: 85% without ties, 66% with them

The founding result is from Zheng et al. at NeurIPS 2023. Expert raters, mostly graduate students, compared answers on 80 multi-turn questions. GPT-4's agreement with them reached 85% once tie votes were removed, higher than the 81% agreement among the humans themselves.

That 85% is the number quoted in vendor documentation. The same table also reports the setup that keeps ties and inconsistent votes, where GPT-4 agreement falls to 66% against 63% between humans. Both comparisons say the same thing: GPT-4 was about as consistent with experts as experts were with each other. Neither says the judge was right 85% of the time.

Kappa tells a different story from percent agreement

Thakur and colleagues graded answers to 400 TriviaQA questions with judges ranging from 7B models to GPT-4 Turbo. Only the largest judges aligned well with humans: GPT-4 Turbo reached a Cohen's kappa of 0.84 and Llama-3 70B 0.79, against 0.96 between humans.

Cohen's kappa is agreement corrected for what two raters would hit by chance. The paper's sharper finding is that judges with similar percent agreement assigned scores up to 20 points apart, while kappa kept that spread to about 5 points. A simple lexical check, whether the answer string contains the reference, scored a higher kappa than half the LLM judges. I would treat that last result as the most useful sentence in the literature for anyone grading short factual answers.

Position bias: the order of answers changes the verdict

Position bias means the judge prefers an answer because of where it sits in the prompt, not what it says. It is the oldest documented LLM judge bias and still the cheapest to test for. You run every pair twice with the order swapped and count how often the verdict holds.

In the MT-Bench study, the answers were identical across both runs and only their order changed. GPT-4 held its verdict 65% of the time, GPT-3.5 46.2% and Claude-v1 23.8%. Claude-v1 favoured whichever answer came first in 75% of cases.

Wang et al. showed the commercial consequence the same year. With ChatGPT as the judge, simply reordering answers let Vicuna-13B beat ChatGPT on 66 of 80 queries. GPT-4 as judge still changed its verdict on 46.3% of that same comparison.

Newer models are better but not cured. Shi et al. ran 15 judges across 22 tasks and more than 150,000 evaluations. Claude 3.5 Sonnet was nearly position-fair on chat questions and showed a strong lean toward the later answer on software development tasks. The paper's central finding is that position bias grows when the two answers are close in quality. That is exactly where you need the judge most.

LLM judge bias toward its own answers and longer answers

Two further biases matter for anyone comparing their own model against a competitor's. Both push the verdict away from quality and toward features the judge happens to like.

Models recognise their own writing and score it higher

The MT-Bench authors reported that GPT-4 gave itself a 10% higher win rate and Claude-v1 a 25% higher one, then said plainly that their data could not confirm the effect. That hedge is worth repeating every time the figures are quoted.

The stronger evidence came in 2024. Panickssery, Bowman and Feng showed at NeurIPS that GPT-4 and Llama 2 can tell their own text from other models' text with non-trivial accuracy. When they fine-tuned models to recognise themselves better, self-preference rose in a linear relationship. The practical rule is simple. Never let a model family grade a contest it is competing in.

Longer answers win unless you control for length

Verbosity bias is the preference for longer answers regardless of content. In the MT-Bench "repetitive list" test, where answers were padded with restated items, Claude-v1 and GPT-3.5 were fooled 91.3% of the time and GPT-4 8.7%.

AlpacaEval's team measured the fix rather than the problem. Dubois et al. regressed the judge's preference on answer length and removed that component. The length-controlled score's Spearman correlation with Chatbot Arena rose from 0.94 to 0.98. Small, but it is a free correction, and I would apply it by default.

Hard questions break LLM judges, including the expensive ones

Every number so far comes from prompts where a human can judge quality by reading. JudgeBench removed that comfort. Its 350 response pairs cover knowledge, reasoning, maths and coding, and each pair has one answer that is verifiably correct.

GPT-4o grading hard pairs is close to a coin flip

With a plain judging prompt, GPT-4o scored 50.86% overall and 44.16% on knowledge questions. The better Arena-Hard prompt lifted it to 56.57%. Google's own Vertex AI evaluation service, running Gemini 1.5 Pro, scored 44.57%.

The authors explain why the scoring is harsh. Each pair is judged twice with the order swapped, and a verdict that flips counts as wrong. That is the right rule. A judge that changes its mind when you reorder the inputs has not judged anything.

The pattern in the figure is the most useful single finding in this post. The judges that did well were the ones that reason before answering: o3-mini at high effort, o1-preview and DeepSeek-R1. A judge has to be able to solve the problem before it can tell which answer solved it. Fine-tuned reward models such as Skywork-Reward-Gemma-2-27B, at 64.29%, matched Claude 3.5 Sonnet at a fraction of the cost.

The same pattern shows up in broader task sets. Bavaresco et al. tested 11 models across 20 human-annotated datasets for ACL 2025. GPT-4o was the strongest judge overall, and its average Cohen's kappa on categorical tasks was still 0.28, with Spearman correlation of 0.50 on graded tasks. On one dataset it reached 0.84 on natural examples and fell to 0.58 on adversarial ones. The authors' conclusion was substantial variance across models and datasets, which matches every other study here.

Which LLM judge to trust for which evaluation task

Pulling the studies together gives a usable map. It is a map of task types, not of vendors, because the same model swings from human-level to below chance depending on what it is asked to grade.

The 5% human sample is our working rule, not a published figure. The last row concedes the strongest point in the judges' favour: a top model gives the same verdict on the same input far more reliably than a pool of crowd raters.

The current-generation picture is thinner but consistent. The Judge's Verdict study found GPT-4.1 at a kappa of 0.792, Claude Sonnet 4 at 0.768 and Gemini 2.0 Flash at 0.769, all inside the human-like band. GPT-4o and GPT-5-chat did not make its top tier. Newer is not automatically a better judge, and that finding alone justifies testing before you swap.

If you route traffic across several models, judge quality becomes one more axis in the trade-off examined in the analysis of quality gaps in model routing. A cheap judge that agrees with an expensive one on easy items, and disagrees on hard ones, is a routing problem with a hidden failure rate.

Where this evidence is weakest

The studies above are good work. They also share limits that a careful reader should weigh before acting on any row of the tables.

Almost every study tests a model you no longer run

MT-Bench graded with GPT-4 and Claude-v1. JudgeBench tested GPT-4o, Claude 3.5 Sonnet and Gemini 1.5 Pro. None of the peer-reviewed work here covers the frontier models most teams use in late 2026. The only study with Claude Sonnet 4 and GPT-4.1 is a 2025 preprint, and it does not test the reasoning-heavy grading tasks where JudgeBench found the largest gaps.

The direction of travel favours the judges. Each generation has scored better on consistency and bias. A team that assumes today's models still flip 35% of verdicts on order is probably too pessimistic. A team that assumes the problem is solved has no published evidence for that either.

Benchmarks are not your data

Bavaresco's 20 datasets produced kappas from 0.34 to 0.84 for the same judge. Your support tickets, legal clauses or code reviews will land somewhere on that range, and no paper can tell you where. The competing view, that a strong judge generalises well enough to skip calibration, wins on chat-style tasks. It loses on anything with a domain-specific notion of correct.

There is also a measurement problem underneath the human baseline. Human-human agreement was 81% on MT-Bench and 0.96 kappa on TriviaQA. Calling a judge "human-level" means very different things in those two settings, and most vendor claims do not say which one they mean.

How to calibrate an LLM judge on your own evaluation data

The studies converge on a short procedure. None of it needs a new tool, and the tooling options are compared in the breakdown of Promptfoo, DeepEval and Ragas.

First, label a gold set by hand. A few hundred items is enough to see the problems the papers describe, provided it includes the hard cases rather than a random sample of easy ones. Thakur's study used 400 questions, MT-Bench used 80 questions with multiple votes. How to structure that set so it survives a model change is covered in the guide to building an LLM evaluation suite.

Second, run every pairwise comparison in both orders and count a flip as a failure, exactly as JudgeBench does. Report the flip rate alongside the score. If it exceeds what you would accept from a human rater, the judge is not ready for that task.

Third, report Cohen's kappa against your gold labels, not percent agreement. On imbalanced labels, percent agreement rewards a judge that always says "pass". Kappa does not.

Fourth, control for length and model family. Regress out answer length where you can, and never let a model family judge its own outputs against a rival. Then keep a standing human review queue for disputed and high-stakes items, designed along the lines of the human-in-the-loop architecture patterns.

Finally, keep checking after launch. A judge prompt that agreed with humans in March can drift as your traffic changes, which is the same problem described in the piece on agent drift in production. Re-score the gold set whenever you change the judge model, the judge prompt or the product.

Frequently asked questions

Where to start

Take the judge you already run and re-score last week's pairwise comparisons with the answer order reversed. Count the flips. That one number, which takes an afternoon to produce, tells you more about your evaluation pipeline than any published agreement rate in this post.

If the flip rate surprises you, pull the 50 items where the judge changed its mind and have someone on the team grade them by hand. Those 50 items are the start of your gold set, and they are drawn from exactly the cases where your judge is weakest.

Related reading