On This Page
The number 420 can be correct for two incompatible reasons. In this illustrative example, two agents return the same contact segment and both list 420 customers. One used the approved store audience and checked consent. The other copied a stale export from a different workspace and happened to get the same count.
A result-only score gives both agents full marks. That is the wrong grade.
A discussion of verification in agent environments separates final output from the route used to produce it. In its spreadsheet example, the right number is only part of the task. Formulas, source references and required formatting also matter. For integrations, the same distinction is sharper: where a result came from can decide whether it was permitted at all.
A final value cannot prove its lineage
Suppose an agent proposes a field map from a store customer object into a marketing system. It maps email to email, firstName to first name and a consent field to the destination’s marketing status. The preview looks plausible.
What did it read to make those choices? The current schema, an old sample, or a document from another account? Did it inspect the consent field’s meaning, or infer it from its label? Did it treat a null as “not subscribed” or as “unknown”?
Each answer changes whether the same final mapping is trustworthy. The map is an output. The evidence trail is the argument for using it.
That trail must identify which version of each source was available at decision time. A later screenshot of a corrected schema does not explain a mapping proposed yesterday. Capture the relevant schema or document revision with the decision, then keep the link between that revision and the rule that was approved. The aim is to reconstruct the decision, not to save every page the agent saw.
This is why a tool trace can matter more than a confident explanation. The trace records the concrete calls and identifiers the agent used. A written summary can be useful, but it cannot prove that a source was consulted. In an integration, evidence should point to the source object, the destination contract and the specific rule that will move the data.
The guide to integrating business tools begins with field ownership because the names on two sides rarely settle who has authority. A process check asks whether the agent actually consulted that ownership decision before it drafted a write.
Write the allowed route into the task
A good test case has two parts. First, state what the destination should contain. Second, state which evidence and actions are allowed on the way there.
For the illustrative segment, the expected output might be 420 contacts, but the approved route could also require a store-scoped query, the current consent mapping and a preview that names excluded records. It could forbid cross-workspace reads and direct campaign sends.
This makes the verifier specific. It can reject a cross-workspace identifier mechanically. It can check that a preview call occurred before activation. It can confirm that the data source named in the output matches the source in the trace. These are rules, not model opinions.
A remaining question may need judgment: did the selected consent field actually mean permission for this campaign? That is where a person or a narrow decision model should examine the field definition and the intended use. The rule and the judgment do different jobs.
This gives a test a useful failure message. “Wrong answer” leaves the team guessing. “Correct count, but a cross-workspace read supplied the audience” identifies a data-boundary failure. “Correct map, but no consent definition was available” identifies missing evidence. A verifier should report the failed condition and the trace entry that caused it, so a repair can change the route rather than polish the final wording.
What does the trace establish?
-
Approved source and required checks
Accept the route
The result has the evidence the task required.
-
Cross-workspace source
Reject the route
A matching count cannot authorize that read.
-
Required evidence missing
Hold for review
The output alone cannot establish how it was derived.
Do not make the trace a ritual
A fair objection is: “An agent can perform the required calls and still ignore them.” Correct. A call list proves a call happened. It does not prove that the returned evidence informed the decision.
One way to test this is to vary the evidence. Present two otherwise similar records with different consent states. If the agent gives the same mapping and explanation in both cases, its apparent source consultation may be theater. Another is to remove a required source and require the agent to stop. An agent that fills the gap from memory has not followed the route.
The cost is more work in the test fixture. Someone must define the authorized sources, expected distinctions and forbidden shortcuts. A generic “did the answer look good?” score is cheaper to build. It teaches less.
Reverse ETL turns a mapping into repeated movement. Once the mapping is approved, ordinary code can apply it without an agent revisiting the evidence on every row. The route audit belongs at the decisions that create or alter that rule, then again when a changed contract calls it into question.
A right answer reached through the wrong account is not a near miss. It is a different task.
Frequently Asked Questions
Why inspect the path if the final answer is correct?
The agent may have used unauthorized data, skipped a required check or reached the right value by chance. The output alone cannot establish whether the route was safe or repeatable.
Does every intermediate action need human review?
No. Record tool calls and enforce deterministic rules for prohibited actions. Review the consequential or ambiguous steps and test representative traces.
How can a team test this?
Pair expected final states with allowed evidence sources and required checks. Replay cases where a shortcut could produce the same answer and see whether the verifier rejects it.