Call checks

Call checks decide what counts as a good call. The same checks that grade an evaluation also grade real customer calls, so a failing check on Monday's calls is directly comparable to a failing check in a test.

Every check is on unless you switch it off. A new agent is fully scored from its first call, with no setup.

What runs after a call

When a call ends, three passes run.

Deterministic checks. Measured from the transcript, the timings and the recording. No language model is involved, so they cost nothing and always run.
Judge metrics. A language model reads the transcript and answers one question per metric. Only the metrics active on this agent are judged, so an agent with no active judge metrics makes no model calls and is charged nothing for this pass.
Post-call analysis. One model pass that writes the summary, sentiment, outcome and fields, and rolls the call into caller memory when caller memory is on. Post-call functions, follow-ups, alerts and the call_log.completed webhook run after it.

Post-call analysis has its own switch, Post-call analysis, in the Post-call analysis panel on the agent's Workflow page. It covers every version. Turning it off stops the summary, outcome, fields, caller memory updates, post-call functions, follow-ups, alerts and the call_log.completed webhook. Checks still run and are still charged.

A text or bridge evaluation scores the conversation but fires nothing downstream. A phone evaluation is an ordinary inbound call, so it runs the whole chain.

The call checks page

The page has three parts:

Grading model. The model that runs the judge metrics. It opens the Grading model page under Tests.
Checks. The checks you can switch off, each with its price per call and a note explaining when switching it off makes sense.
Always on. Paid checks with no switch, because a failure means something went wrong rather than that this agent works differently. The card heading shows their combined price per graded call.

Free checks are not listed, which is why switching everything visible off does not switch scoring off. A check that does not apply to a call sits it out rather than failing it.

How a call is judged successful

A call is successful when its outcome is Completed, or Transferred and the transfer connected. Post-call analysis sets the outcome. Checks are shown for review and do not change a call's outcome.

In an evaluation, a failed critical check still fails the run. A check is critical if its definition says it affects call success. Four do today: task success, hallucination, tool call success and disclosure spoken.

What scoring costs

Call checks are billed per active judge metric on the call, never as a flat rate.

Deterministic checks are free.
Each judge metric has its own price, shown on the Call checks page next to its switch, and priced at cost.
Only a check that actually produced a result is billed. Errored, untriggered or skipped checks are not.

See Call check catalogue for the full catalogue and which single deterministic check is not free.

A scenario scoring three judge metrics costs materially less than one scoring twenty. The run estimate before launching an evaluation adds up the metrics active on that scenario rather than applying a flat price. Treat any estimate as an estimate: billing follows the metered minutes and the metrics that actually ran.

Switching off a judge metric that does not suit the line removes its price from every call. That is what the Checks card is for.

Where scores appear

Scores land on the call, not in a separate report.

The call page opens with a summary card, a result card and an analysis card. The analysis card shows what post-call analysis recorded: call successful, urgency, caller type and your own fields. The result card carries a headline strip (task success, hallucination, CSAT and latency), a count of checks passed, failed and measured, and the failing checks, each linking to its row. Metrics that errored or were skipped are left out of the strip.

The Checks tab lists every check as a row with its verdict or the value it measured. Opening a row shows the judge's reasoning, the judge prompt and a thumbs-down to flag a verdict you disagree with. Run checks scores a call that has none, and Run again scores it again.

Rated checks

A rated check has no pass or fail. It is a score out of 100, shown as a percent.

CSAT is built from three ratings the judge gives the call, each from 0 to 4:

Resolution, 45% of the score: whether the caller got what they called for.
Ease, 30%: how little the caller had to repeat or correct.
Mood, 25%: how the caller sounded at the end of the call.

Opening the CSAT row shows all three.

On the calls list, each row lifts the task check, hallucination and latency, with a metrics pill whose tooltip names up to five failing checks.