On This Page
A reply about a returned order looks ready to send: grammatical, friendly and short. In this illustrative case, an automated score gives it full marks. The assistant also tells the customer a refund has been issued, although nobody checked the payment system.
The error is not a weak sentence. It is a weak definition of good.
In an ingested evaluation demonstration, a team builds scores for an agent that writes software documentation. The interesting moment is when the person who knows the work describes what quality means, and a tool turns that description into criteria. Later, a low score provokes the right doubt: perhaps the agent needs a fix, or perhaps the evaluation does.
The rubric is a theory about the task. Test the theory.
Start with the disqualifying mistake
Teams often begin with dimensions that are easy to score: clarity, tone, structure, completeness. They are useful. They are rarely sufficient.
For the illustrative refund reply, “never claim a completed refund without a payment receipt” belongs above prose quality. For a data sync, “do not map a billing email to the account owner’s email” matters more than whether the field-map explanation reads well.
Ask the domain expert for three kinds of cases: one obviously good, one obviously wrong and one case where informed people might disagree. Preserve the source records and the reason for each judgment. If a case depends on a rule that may change, version the rule with the label.
The first pass can be small. It cannot be vague.
- Task owner
good, bad and disputed cases
- Exact checks identity · permission · destination
code enforces facts
- Semantic rubric
judge only what needs interpretation
- Disagreement review
correct labels and criteria
| Check | Better owner | Why |
|---|---|---|
| Required permission exists | Code and authoritative records | The fact is exact and consequential |
| Destination ID matches source identity | Code and a test record | A model should not guess an identifier |
| Reply answers the customer’s question | Human-reviewed rubric, possibly a model judge | Meaning needs interpretation |
| Tone suits the situation | Human-reviewed rubric, possibly a model judge | More than one acceptable form exists |
Our Jev use cases article makes the same boundary for typed decisions: code copies original values and checks constraints after a model selects among candidates. The model’s confidence does not become permission.
A score inherits the rubric’s blind spots
The objection is fair: “A human cannot review every agent run. We need a scalable judge.” Yes. The question is what the judge is scaling.
If the rubric rewards a polished answer and never checks whether the order was actually refunded, automation will scale the omission. A number with two decimal places looks precise. It can still be measuring the wrong thing.
Google Cloud’s judge-evaluation documentation describes a practical check: prepare cases with human ratings, run the model-based metric, and compare its scores with those ratings. The point is not that one person’s view is infallible. It is that the model needs an external standard before its score can carry weight.
Inspect disagreements, not only average agreement. If the judge passes a reply that a support lead rejected, ask which missing criterion explains it. If two experts disagree, keep both labels and find the rule they interpret differently. Do not silently average away the dispute.
Sampling matters as much as the wording of a criterion. A collection of routine successes can make a judge look reliable while leaving the hard cases untouched. Include incomplete records, conflicting instructions and cases where the correct answer is to stop and ask. Keep a separate set of cases the rubric authors did not tune on, so a change that improves familiar examples has to prove itself elsewhere too. When the business rule changes, refresh both labels and expected outcomes; a frozen test set can reward yesterday’s policy.
Keep the outcome attached to the evaluation
A rubric for a sync agent should follow the customer’s result. Did the correct contact change? Was consent respected? Can a failed record be replayed without repeating a consequential action?
The Brevo connector guide explains why a successful API call can coexist with a missing attribute. An evaluation that checks only whether the agent called the endpoint will miss the outcome. One that checks only the destination may miss a prohibited path that happened to end with the right value. Both matter when the path carries risk.
Start with one narrow task and a few failure modes. Label the cases with the person who owns the task. Put exact checks in code, and use a model judge only where language or judgment truly requires it. Run the same cases after a prompt, model or tool change. Record what later failures reveal about the rubric.
The cost is expert time up front. The win is that every later score has a meaning someone can defend.
A dashboard can show a score. Only the rubric tells you what you counted.
Frequently Asked Questions
Who should define an AI agent's evaluation criteria?
The people who know the task and its consequences should define acceptable results and disqualifying errors. Engineers can turn those judgments into repeatable checks and evaluate whether a model judge agrees with human ratings.
Is a high judge score proof that an agent works?
No. A score measures agreement with a rubric on the cases tested. A weak rubric, unrepresentative cases or a poorly calibrated judge can give a high score to a harmful result.
What should be checked by code instead of a model judge?
Exact permissions, record identifiers, required tool calls and destination state should be checked with code whenever possible. A model judge is useful for semantic qualities that exact rules cannot settle.