TL;DR
LLM-as-a-judge means using one language model to score another’s output against a written rubric. It works well for questions about meaning, such as relevance, groundedness or policy compliance, and it scales to thousands of outputs a day. In the original study, strong judges agreed with human preferences more than 80% of the time, about as often as humans agree with each other. But judges have known biases: they favor answers in certain positions, longer answers, and sometimes their own model’s answers. Use code checks for anything a rule can decide, calibrate every judge against human labels before trusting it, and re-check it whenever the model, rubric or data changes.
What is LLM-as-a-judge?
An LLM judge reads an output, plus the question and any context or reference answer, and decides whether it meets a standard you’ve written down. Most open-ended AI outputs have no single correct answer, which makes rule-based scoring very hard; a judge model can assess meaning instead (Zheng et al.).

how an LLM judge works
There are two common setups. Single-answer grading scores one output against a rubric, usually pass or fail with a reason. Pairwise comparison shows the judge two outputs and asks which is better, which is useful for comparing prompts or model versions. Both are widely used inside evaluation suites; see our guide to AI agent evaluation for where judges fit alongside code checks and human review.
When it works
The case for judges rests on solid evidence. Researchers at UC Berkeley and other universities compared judge verdicts with thousands of expert and crowdsourced human votes, and found that a strong judge matched human preferences at an agreement rate above 80%, the same level at which humans agree with one another (Zheng et al., arXiv). They concluded that LLM-as-a-judge is a scalable and explainable way to approximate human preferences that would otherwise be very expensive to collect (NeurIPS).
Judges are a good fit when:
- The question is about meaning. Does the reply answer the question? Is every claim supported by the source? Does it follow the policy?
- There’s no single right answer. Open-ended replies, summaries and conversations can’t be matched against one reference.
- You need scale. Scoring a steady sample of production traffic, or thousands of test cases on every release, is impractical for people alone.
- The rubric is clear. Specific, binary criteria give consistent verdicts; vague ones don’t.
- You have human labels to check against. A judge you’ve measured is a tool. A judge you haven’t is a guess.
When it misleads
The same research documented several ways judges go wrong (Zheng et al., arXiv):
| Bias or limit | What happens | How to reduce it |
| Position bias | In pairwise comparisons, the judge favors the answer in a particular position, often the first (Emergent Mind) | Run each comparison twice with the order swapped; count only consistent verdicts |
| Verbosity bias | The judge favors longer answers, even when they’re no clearer or more accurate (Emergent Mind) | Tell the judge length is not a quality; include short correct answers in your calibration set |
| Self-enhancement bias | The judge favors answers written by its own model | Use a judge from a different model family than the one being judged |
| Limited reasoning | The judge misgrades math, logic or facts it can’t check itself | Give it a reference answer, ask it to reason before deciding, or use code checks instead |
Later research adds a further warning: judge preferences can favor style over factuality and safety, rewarding answers that read well over answers that are correct or safe (as summarized alongside the original paper by FuguMT). That matters most for exactly the checks teams care about most.
Judges also mislead when:
- A rule would do. Asking a judge whether a reply is under 100 words, or whether a refund tool was called, adds cost and error to something code can check exactly.
- The rubric is vague. “Is this a good answer?” gives verdicts that drift from run to run.
- Nobody checks them. A judge model update, or a rubric edit, can quietly change scores with no change in the system being judged.
- They’re judging something adversarial. A judge asked whether a reply is safe can itself be influenced by the manipulative content it’s reading. Judges are not a substitute for adversarial testing; automated red teaming for AI agents explains why.
How to calibrate a judge
Calibration means measuring how often your judge agrees with people on your own data, then fixing it until the agreement is good enough to rely on.

calibrating an LLM judge
- Label a sample by hand. Pick 50 to 100 real outputs, including clear passes, clear failures and borderline cases. Have someone who knows the domain mark each one against the rubric. For important checks, use two reviewers and settle disagreements.
- Run the judge on the same sample. Same rubric, same inputs, no hints about the human labels.
- Measure agreement. Count how often the judge and the people agree, and look closely at where they don’t. Separate the two kinds of error: a judge that passes bad outputs is more dangerous than one that fails good ones.
- Fix the rubric, not the numbers. Most disagreements trace back to vague wording. Rewrite the pass rule, add an example, or split one check into two, then score the sample again.
- Deploy, then keep checking. Once agreement is good enough for the stakes involved, use the judge at scale, and re-check it on a fresh labeled sample regularly, and always after changing the judge model or the rubric.
What counts as good enough depends on the check. For a tone check, occasional disagreement is tolerable. For a policy check on refunds, it isn’t, and a code check on the action is usually better than any judge.
Writing a rubric a judge can follow
| Weak rubric | Better rubric | Why |
| “Is the answer good?” | “Does the answer resolve the customer’s stated problem, or clearly explain why it can’t?” | Specific and checkable |
| “Rate helpfulness from 1 to 10” | “Pass if every step the customer needs is included; fail if any is missing” | Binary verdicts are more consistent than fine scales |
| “Is it accurate and polite and concise?” | Three separate checks, one per quality | One check per judge call keeps verdicts clean |
| “Is it grounded?” | “Pass only if every factual claim appears in the provided documents” | Defines the standard and the evidence |
Three more habits help: ask the judge to explain its reasoning before giving a verdict, give it the evidence it needs (the source documents or a reference answer), and include short example verdicts for borderline cases.
Judging agents
Agents add a complication: the thing being judged is a sequence of actions, not just a reply. A polite final message can follow a wrong refund. Three rules make judges useful for agents:
- Give the judge the trace, not just the reply. It needs to see the tool calls, arguments and results to judge whether the agent did the right thing.
- Use code for actions, judges for meaning. Whether a refund was issued outside policy is a rule; check it with code on the trace. Whether the agent’s plan made sense for the request is a judgment; use a judge.
- Judge the whole conversation for multi-turn tasks. Many agent failures, including slow manipulation over several turns, only show up across the full conversation. Multi-turn attacks on AI agents shows what that looks like.
AI agent evaluation metrics that matter covers which agent metrics suit code checks and which suit judges.
Using judges in production
- Score a steady sample, plus every high-risk action, rather than everything.
- Pin the judge model version. If the judge changes, scores change, even when the system being judged doesn’t.
- Watch the judge, not just the scores. A sudden shift in pass rates with no release on your side may be the judge drifting.
- Spot-check with people every month, weighted toward borderline and high-stakes cases.
- Keep adversarial testing separate. Judges score the traffic you have; red teaming finds the attacks you haven’t seen yet. How to build an AI red teaming dataset for agents covers building those cases.
For where judges fit among other scoring methods, see the LLM evaluation metrics cheat sheet and how to build an LLM evaluation framework.
The bottom line
LLM judges are one of the most useful tools in AI evaluation, and one of the easiest to misuse. Use them for meaning, not for anything a rule can decide. Write specific, binary rubrics. Calibrate against people before you trust the scores, watch for position, length and self-preference bias, and re-check whenever anything changes. And never let a judge stand in for adversarial testing.
BotGauge LLM evaluations combines LLM evaluators, code-based checks and custom criteria, and BotGauge LLM red teaming finds the failures judges can’t. For other tools, see the 12 best AI observability and evaluation tools for agents.
Think your agent is safe? Prove it. BotGauge checks what your agent does, not just how its replies read. Try Botgauge for Free or book a 30-minute demo.
