Measure cost per accepted result

A team celebrates a low cost per AI run. Some runs, however, need a retry. Some outputs require a person to correct a field. A few trigger an external lookup. The cost of one run stays low while the cost of one accepted work item grows. This is an illustrative accounting example, not a measured result.

An agent operations demonstration shows token and work-cost attribution across sessions and teams. That visibility is useful, but spend dashboards alone do not prove value. To assess an automation, the denominator matters as much as the numerator.

Define the result before adding the costs

For a data sync, a model might classify a free-text field before a record moves. A “successful run” could mean the model returned a label. A useful result is stricter: the destination accepted the right record, the permission rule held, and no correction was required during a defined review window. Those are different units.

Choose the unit that the workflow owes the user. It might be a resolved support case, an approved invoice or an accurately updated customer profile. Define what acceptance means and who can reject a result. Without that definition, a team can make a cost chart improve by lowering the bar.

Then track the path to each accepted result.

1. Attempt

   model call
   may require
2. Tools and retries

   extra work
   then
3. Review

   correction time
   passes checks
4. Accepted result

   defined outcome

Count model usage, tool calls, retries, human review and rework. Keep these components visible rather than blending them into one unexplained currency figure. Tool costs can include paid API calls; human time can be reported in minutes and converted to money only with an explicit local assumption.

Quantity	Example definition	Why it matters
Attempt	One initiated run	Shows load, not delivered value
Accepted result	Work item passing stated checks	Denominator for useful output
Retry	Another attempt for the same item	Reveals hidden compute and delay
Correction	Change after initial acceptance	Reveals quality debt
Review time	Human minutes spent checking	Reveals shifted work

This table proposes bookkeeping categories; it supplies no benchmark for agents or syncs.

Follow the item, not just the model

A model can be inexpensive inside an expensive workflow. Suppose a classification takes one call but the system fetches three records, waits on an external API and asks an operator to reconcile a mismatch. Reducing tokens by half may barely affect total cost. Conversely, a slightly more expensive classification could save review time if it gives better evidence and fewer mistakes. Neither result follows automatically; measure it.

Use an item identifier to join attempts, tool calls and outcomes. Keep privacy constraints on the trace: accounting needs counts and references, not an unrestricted copy of customer data. Record model or rule version because a later change may alter both cost and error patterns.

Averages can hide the expensive tail. Report a distribution or at least a high percentile alongside the median. One rare path that repeatedly retries can consume a large budget or delay a time-sensitive send. Distinguish failed work that was safely stopped from bad work that looked accepted and needed correction later.

Define quality before comparing costs

Some results do require judgment. That is a reason to specify a review rubric, not a reason to use cost per run. For a support reply, acceptance might require factual correctness, authorized action and a tone standard. Different reviewers may disagree; track that disagreement and avoid a false precision claim.

For exact sync steps, acceptance can be more deterministic. The destination has the expected ID and fields, the source event is acknowledged once, and a duplicate does not create another customer. A later correction should be attributed to the same work item. This prevents the dashboard from counting a fast incorrect sync as completed value.

Cost is not the only objective. Latency, reliability, privacy and customer harm can dominate. A cheaper workflow that makes rare but severe mistakes may be a worse choice. Compare cost only among approaches that meet the necessary quality and permission bar.

A comparison also needs a baseline. Measure the existing manual path or the previous automated path with the same acceptance definition. Otherwise a new agent can look efficient merely because its failed work is counted differently. Set a fixed observation window for late corrections and report cases that have not matured yet rather than silently marking them successful. If an item is reviewed by two people, count the actual review effort and note disagreement. The AI workflow implementation guide covers rollout choices; this ledger tests whether one choice delivers accepted work at a sustainable cost.

The most useful accounting question is not “How cheaply did the model speak?” It is “What did it cost to deliver one result we were willing to keep?”