Docs · Quickstart

Test your voice agent on a real call

callerlab dials your agent through your own Twilio account, plays a realistic caller, has a judge model give a verdict and can time the call from an optional recording. Everything starts as a dry run that dials nothing.

After install · $0Install and open the demoAfter install, npm run dash serves a local demo dashboard with sample data and prints its link.

No calls · $0Dry run a packPlan, price and blockers. Dials nothing.

Your Twilio · paidPlace a real callTwilio, a tunnel and an allowlisted number. Then --yes.

Install

Run callerlab from a clone; it isn’t on npm yet. Needs Node.js 22.6 or newer.

Terminal

git clone https://github.com/RobStrayer/callerlab.git
cd callerlab
npm ci
npm link        # puts `callerlab` on your PATH
pipx install edge-tts   # free default voice, online (Microsoft Edge); needs ffmpeg
npm run dash           # local demo dashboard with sample data
# what works now, what a real call is missing
callerlab doctor

Step 1: Dry run

First set CALLERLAB_DIAL_TO to a number you own or may test (E.164). The dry run plans the calls to it and prints the projected spend and its bound. Free, and dials nothing.

Terminal

# first: export CALLERLAB_DIAL_TO=<a number you own, E.164>
# prints the plan, the projected spend and its bound;
# dials nothing
callerlab explore --pack examples/packs/home-services.json \
  --to "$CALLERLAB_DIAL_TO"

explore plans and, with --yes, places calls; a model holds the conversation, then the judge scores the transcript. Options:

--temperamenteasygoing · impatient · pushy · argumentative · relentless · heated
--bednoise bed, e.g. car-cabin
--snrsignal-to-noise in dB, e.g. 0
--repeatN
--max-usdmodeled run-cost cap in US dollars; no default
--recordsave a local two-channel WAV on the stream transport

Every flag and value: docs/CLI.md.

Step 2: First real call

What it needs, then the same command with --yes.

Need	Set with
Twilio account and credentials	`TWILIO_ACCOUNT_SID` + `TWILIO_AUTH_TOKEN` (validates callback signatures)
Public HTTPS tunnel to 127.0.0.1:8787	`cloudflared tunnel --url http://127.0.0.1:8787` → `CALLERLAB_PUBLIC_URL`
A destination you may call	`CALLERLAB_DIAL_TO` or `CALLERLAB_ALLOWED_DESTINATIONS`
Caller ID and harness secret	`CALLERLAB_DIAL_FROM`, `CALLERLAB_HARNESS_SECRET`
Speech-to-text endpoint (for the default streaming transport)	`CALLERLAB_STT_BASE_URL`, `CALLERLAB_STT_MODEL`, `CALLERLAB_STT_API_KEY`. Without them a run falls back to deprecated Gather
Caller model (for explore)	`CALLERLAB_MODEL_BASE_URL`, `CALLERLAB_MODEL_API_KEY`

Terminal

# places real, paid calls through your Twilio account
# no default spend cap: add --max-usd <dollars>
callerlab explore --pack examples/packs/home-services.json \
  --to "$CALLERLAB_DIAL_TO" --yes

--yes enables paid calls. Confirm permission and consent first.

To measure reply gaps and conversation timing, configure a speech-to-text endpoint and add --transport stream --record. Recording is off by default; without it those timing measures are not available.

There is no default spend cap. --max-usd caps modeled run cost, not every provider invoice.

Step 3: Score it

Both run offline on the report a real call wrote, so they cost nothing.

Terminal

callerlab scorecard --report .cache/reports/<stamp>
callerlab ci --report .cache/reports/<stamp> --fail-on fail=1

`callerlab scorecard`

Five lanes, each measured, thin or not measured. Same bytes every run.

`callerlab ci`

The same report as a build gate, for CI (Gate a build, Exit codes).

Agent quickstart

Coding agents can install callerlab, dry-run a pack and read results. The skills carry callerlab’s calling rules.

Agents should dry-run the exact plan first and wait for your go-ahead before placing paid calls.

Claude Code

/plugin marketplace add RobStrayer/callerlab
/plugin install callerlab@callerlab

Includes the call-guard hook: a second lock that asks before commands it recognises as dialling. It reads command text, so some commands get past it.

Terminal

npx skills add RobStrayer/callerlab

Carries the rules but not the call-guard hook.

Terminal

# read-only stdio server: list/read runs, compare,
# test a --fail-on gate, price a call
callerlab mcp

It cannot place a call or spend money. Configs for Claude Code, Codex, Cursor and Hermes are in docs/MCP.md.

AGENTS.md / CLAUDE.md (excerpt)

Voice-agent testing uses callerlab (run `callerlab help`).
- Run this without --yes first, and show the dry run to the human:
    callerlab explore --pack examples/packs/home-services.json \
      --to "$CALLERLAB_DIAL_TO"
- Add --yes only after the human says go, for this exact plan.
- Call only numbers the human owns or has written permission to test.
- Never read or print .env or any key.
- Report calls, conditions and judge model. One call is not a rate.

Prompts that work

“Set up callerlab and dry-run the home-services pack against my test number in $CALLERLAB_DIAL_TO. Don’t dial.”

“Score the last run and tell me which lane is thin and why.”

Gate a build

Terminal

callerlab scorecard --report .cache/reports/<stamp> \
  --require "task.pass.low>=0.7,speed.p95<=3500"
callerlab ci --report .cache/reports/<stamp> \
  --fail-on fail=1,flaky>=1

Both write summary.json and junit.xml beside the report. A judged fail does not set the exit code on its own; --fail-on decides that. A call that fails as a call, or is unjudged in a gated run, can still exit 1 or 2. For a workflow to copy, see the GitHub Actions example in docs/ci.md.

Exit codes

Exit	Meaning	Agent should
`0`	No gate tripped; may be ungated or not evaluated	Read the verdict
`1`	A gate or call failed, or the run ended early	Read the output
`2`	No verdict, including configuration or evaluation problems	Read the cause and fix it
`3`	explore watchdog ended a stalled step	Read which step
`4`	Inconclusive: a call hit its time limit without a defect	Consider a longer call
`75`	live or explore: only the phone line failed, or line outcomes stopped the run	Try later, not immediately

Environment

Every flag and default is in docs/CLI.md. CALLERLAB_EDGE_PYTHON can only come from your shell environment, not a project’s .env.

Variable	What it sets	Needed for
`TWILIO_ACCOUNT_SID`	Twilio account	Live dials
`TWILIO_AUTH_TOKEN`	Validates X-Twilio-Signature; enables caller keypad	Live dials
`CALLERLAB_PUBLIC_URL`	Your HTTPS tunnel URL	Live dials
`CALLERLAB_DIAL_FROM`	A Twilio number you own	Live dials
`CALLERLAB_DIAL_TO` / `CALLERLAB_ALLOWED_DESTINATIONS`	Numbers you may call (E.164, comma-separated)	Live dials
`CALLERLAB_HARNESS_SECRET`	Signs step tokens	Live dials
`CALLERLAB_MODEL_BASE_URL` / `_API_KEY`	Caller model (and judge by default)	explore, judge
`CALLERLAB_JUDGE_BASE_URL` / `_API_KEY`	Separate judge endpoint	Optional
`CALLERLAB_TTS_PROVIDER`	edge (the free voice, used when no other voice key is set); see `docs/CLI.md` for other providers	Optional
`CALLERLAB_STT_BASE_URL` / `_MODEL` / `_API_KEY`	Speech to text; all three or none	stream, transcribe
`CALLERLAB_FAIL_ON`	Default --fail-on gate	CI
`CALLERLAB_ALERT_WEBHOOK`	Canary state changes (https only)	Canaries

How it works

Architecture

How a test call is put together, and where each decision in it is made.

What you install today: the self-install CLI defaults to Media Streams. If no transport was specified, it falls back to deprecated Gather when the stream cannot carry the run, including when no speech-to-text endpoint is configured. Explicit stream requests are refused rather than falling back. Recording is off by default; explore --record saves a streamed call locally, and the provider records nothing. See docs/CLI.md and Environment.
Diagrams 1 to 4 show the streaming caller: callerlab’s own speech-to-text, caller model and text-to-speech over one live audio stream, drawn on Twilio Media Streams. The diagrams show one configured Media Streams setup; their “defaults” refer to that illustration. See the CLI documentation for current self-install defaults.
On the streaming path, Twilio is the phone line only: it dials, carries audio and keypad tones, and hangs up.
Planned means not built yet; a figure marked [E] is an estimate, not a measurement.

Timing figures publish after we re-run the timing measurements. Until then the diagrams below leave timings out, and the two drawn to a time scale (the latency waterfall and the worked call) are not shown. Measured LLM, TTS and STT results are on the Benchmark page; real-call reply gaps follow the re-run.

1 · System overview

Where each part runs on the streaming path, drawn on Twilio Media Streams. STT, the caller model, TTS, the monitor and the judges run inside callerlab; Twilio dials, carries audio and DTMF, hangs up and reports line quality. The decider is a decision model: it is opt-in and picks the caller’s next move, and a rule decides by default and as fallback (see Decision points). The affect judge is experimental.

Text version

System overview: callerlab does the thinking, Twilio is only the pipe. STT, TTS, the caller model, decider, monitor and judges run inside callerlab. Twilio dials, carries audio, sends DTMF, hangs up and reports line quality.

callerlab (all intelligence lives here)

CallControllerLifecycle, user-set spend cap, if any (before dial, every 30 s),keypad, latency metrics, hangup, ledger lines
tier: standard
Our STT (listen)Streaming STT live (opt-in)mu-law 8 kHz, interim resultsfinal/endpointing
TurnManagerEndpointing, barge-in, floor stateemotion tag: check vs persona arc,strip, map to TTS control, log
Decider (opt-in) (decide)Small model via a gatewayopt-in: rule by defaultpicks the caller’s next move
Caller LLM (think)Hosted open-weights modelalternate host→ {say, emotion, intensity 0–3}
Our TTS (speak)Default streaming voiceOptional alternate voice (laughter and joking)
Monitor (every turn)Runs on every turn, off theaudio hot path; watches the callagainst the scenario
Post-call judgesScore transcript, audio latencymetrics, emotion/prosody vsexpected outcomesAffect judge: EXPERIMENTAL,not yet reliable
Line Check (Voice Insights)MOS, jitter, loss, latency,who hung up, per legCarrier-explainable FAIL →“inconclusive: carrier”
Results + cost ledgerreport.json latency, lineCheck,turns[].callerAffect; ledger
Keypad safety (unchanged)sendDigitsProblem checks;911, short codes, N11 blocked
Twiliotelecom pipe onlyDialREST Calls createCarry audio<Connect><Stream> (duplex)Send DTMFCalls update: <Play digits>Hang upStatus=completedLine qualityVoice Insights → Line CheckNot used when streaming× ASR / <Gather input="speech">× <Say> or any provider TTS× transcription× Voice Intelligence× AMD / MachineDetection× provider recordingVoice Insights is network telemetry,not AI, and stays on.
PSTNpublic phone networkcarries the call
Target voice agentthe system under teste.g. a hosted voice agent+1 (•••) •••-••••number masked
Audio on the wireInbound: mu-law 8 kHz, 20 ms frames → callerlab. Outbound: our TTS → 8 kHz mu-law, 20 ms frames, paced at real time with ≤ 200 ms lead, a mark per chunk.During a turn there are no webhooks: the only HTTP is the initial TwiML and status callbacks. Line Check data arrives 30 s–10 min after hang-up.

Flows (arrows):

Our STT → TurnManager
TurnManager → Decider (opt-in)
Decider (opt-in) → Caller LLM
Caller LLM → Our TTS
Our TTS → Post-call judges
Post-call judges → Results + cost ledger
Line Check (Voice Insights) → Results + cost ledger
callerlab ↔ Twilio (WS)
Twilio ↔ PSTN
PSTN ↔ Target voice agent
Twilio → Line Check (Voice Insights)

Affect judge is experimental. Phone numbers masked.

callerlab design and bench data, Oct 13, 2026

2 · One turn

One caller turn, from the target’s speech to callerlab’s streamed reply, with the emotion-tag, barge-in and DTMF paths in the side panels. The decision model is opt-in, with a rule as the default and fallback: when it is on, the next-move decision starts with the caller line rather than before it, an as-planned move keeps that line, and any other move asks for it again, once. Timings are left out until the re-run; the values shown are spec parameters.

Text version

One turn, step by step. The target speaks; callerlab listens, works out the caller’s line and the opt-in next move together, and answers with an emotion-tagged line over the same stream.

Lanes, left to right: Target agent, Twilio + PSTN, WS + AudioPipeline, STT, TurnManager + decider, Caller LLM, TTS

1 agent speech: inbound media, mu-law 8 kHz, 20 ms
2 decode, ring buffer → push(frame)
3 finals, endpointing, utterance_end
4 decider (opt-in): next move, as 5 starts
5 {say, emotion, intensity 0–3}
6 tag checked vs arc, stripped, mapped → TTS control
7 streamed audio, TTFA → 8 kHz mu-law, ≤ 200 ms lead
8 media + mark say-<turn>-<chunk> → played to the agent
9 mark echo = true play time; transcript keeps played words

Steps 4 and 5 start together; another move asks for the caller line again, once. Cut-ins come from the stream settings, not the decider. Values shown are spec parameters.

Emotion tag on every caller turn (affect)Caller LLM outputs {say, emotion, intensity 0–3}1 TurnManager checks the tag against persona.arc (invalid → clamped, logged)2 strips the tag before TTS3 maps it to the provider: steering tags or audio tags (a voice without emotion steering → wording/speed fallback)4 logs turns[].callerAffect {emotion, intensity, providerControl, clamped}
Every turn and after the call (judge)Every turn: decider (small model opt-in; rule by default) picks the caller’s next moveEvery turn: monitor runs alongside, off the audio hot pathAfter the call: judges score transcript, audio latency metrics and emotion/prosody vs expected outcomesAffect judge: EXPERIMENTAL, not yet reliable (chance-level today)Line Check: carrier-explainable FAIL → “inconclusive: carrier”
Barge-in paths (clear)Agent interrupts caller (default onTalkedOver: yield) if agent onset ≥ 250 ms (debounced) while our audio plays: 1 send clear 2 stop TTS, cancel LLM 3 record yielded, sentFraction, cutAtMs from returned marks ignore agent onsets within 150 ms of a clearCaller interrupts agent: stream settings decide; BargeInSettings .afterMs starts our line (test: 1200 ± 40 ms)
DTMF path (keys)Validate first: sendDigitsProblem / sendDigitsCodeProblemDefault: POST /Calls/{Sid} with Twiml <Play digits="4w0w8"/> + <Connect><Stream> resume token; new stream within 10 s; each w = 0.5 s; reconnectMs recordedFallback, in-band tones as media: 70–100 ms, −6 dBFS, mu-law; used for keypad.mode: inband, or update fails / takes > 2 sServer may send only media, mark, clear (else error 31951)

Twilio carries audio and DTMF only. Affect judge is experimental.

callerlab design and bench data, Oct 13, 2026

3 · Latency waterfall

This diagram draws each stage’s timing to scale, so it returns after the re-run.

4 · Cost per call: historical Gather vs streaming

Not like-for-like. The Gather column runs the advanced judge and the streaming columns assume a cheaper one that you set; a run with no judge set uses the preset’s default, the advanced judge. The streaming columns also use the STT provider’s promo rate, so only part of the gap comes from the path.

One answered call on the Gather path beside two streaming options on Media Streams. Per answered call: 2.7 min, ~10 caller turns, ~900 caller characters. Each line carries its basis: actual bill, published price or estimate. The totals include the target agent’s own platform cost ($0.126 [A]); the pricing FAQ covers callerlab’s side only: its “about $0.10/min” is the Gather column without the target agent’s line, with a basic or standard judge in place of the advanced one. Both streaming totals use the STT provider’s promo rate; at list price the STT line, and so the total, is higher. [E] estimated[P] published price[A] actual bill

Text version

Before vs after: Gather webhooks → Media Streams. Per answered call: 2.7 min, ~10 caller turns, ~900 caller characters.

Before: Gather + webhook turnsTwilio does speech recognition (<Gather input="speech">)One webhook per turn → callerlab → TwiML reply15 s webhook timeout → application errorsAdvanced judge on every call
After: bidirectional Media Streamcallerlab runs its own STT, LLM and TTSAudio both ways over one WebSocket; no webhooks per turnDTMF via Calls update or in-band tonesVoice Insights kept for Line Check; own recording

Line item	Gather today	(a) Default streaming voice	(b) Optional alternate voice
Twilio minutes, both legs	$0.071 [A]	$0.071 [A]	$0.071 [A]
Twilio media streams	$0.013 [A]	$0.011 [A rate]	$0.011 [A rate]
Twilio Voice Insights (Line Check)	$0.016 [A]	$0.016 [A]	$0.016 [A]
Twilio recording	$0.008 [A]	$0 own	$0 own
STT: Gather → streaming STT	$0.138 [A]	$0.013 [P]	$0.013 [P]
Caller LLM	$0.002 [E]	$0.001 [E]	$0.001 [E]
Caller TTS (today → streaming)	$0.015 [E]	$0.0225 [P]	$0.036 [P]
Target agent’s platform	$0.126 [A]	$0.126 [A]	$0.126 [A]
Judges / monitor / decider	$0.090 [E]	$0.003 [E]	$0.003 [E]
Total per call	~$0.48	~$0.26	~$0.28
Per 10-scenario pack	~$4.8	~$2.6	~$2.8

Both streaming totals assume the basic/standard judge, which you set. With none set, the preset’s default judge is the advanced one, which adds ~$0.09 [E].Target agent’s platform $0.126 [A] is half or more of each streaming total and belongs to the agent under test. Twilio minutes $0.071 [A], Voice Insights $0.016 [A].The alternate voice is optional (laughter and joking); its higher-tier model would add ~$0.036. The streaming STT line is at a $0.0048/min promo; $0.021 at list price.

[A] actual Twilio/campaign bill · [P] published price (Oct 13, 2026) · [E] estimate.

callerlab design and bench data, Oct 13, 2026

Decision points

These maps are snapshots from Oct 13, 2026, showing decisions by rule, language model, decision model or classifier. The then-current map has 22 points: 12 rules, 4 LLM-driven and 6 decision-model points; the proposed map has 37. Decision models are now available as an opt-in; rules are the default and fallback. See docs/DECISION-MODELS.md for current behavior.

Decisions on Oct 13, 2026

The Oct 13 snapshot, by stage. Timeouts and thresholds were set in code, not measured. [S] set in code or spec, not a price

Text version

Oct 13, 2026 map: 22 decision points, and all 6 decision-model calls run their rule. Six points can ask a small decision model (DM). Each one has a rule fallback, and today the rule runs.

Legend: Rules, LLM, DM: runs its rule today

22 = 6 DM + 4 LLM-driven + 12 rules

Decision-model points run their rule todaySix points (C1–C6) can ask a small decision model for a choice.Each DM call has a deadline and a rule fallback.The decision-model path is not switched on yet,so every DM point runs its rule.
decideChoice (src/decide/decide.ts)All 6 DM points go through it. It falls back to the rule on:no-decider · timeout · error · low-confidence · no-allowed-optionTimeout 600 ms in call [S]; 15,000 ms after call [S]; confidence < 0.3 [S]Decision model, opt-in

Before / at pickup

Pre-dial (3 pts)

C7 (Rules)Real-run blockers: STOP file, user-set spendcap (if any), scope, allowlist, voice
C8 (Rules)Static keys allowed? (911 / emergency / N11 /length)
C9 (Rules)Per-call wall budget and watchdog phases

Pickup (1 pt)

C10 (Rules)Who answered: live, IVR, voicemail, unknown,intercept; voicemail → hang up

Turn-taking (partly on the streaming path) (2 pts)

C16 (Rules)Agent talks over caller → yield afteryieldAfterMs (600 ms [S])
C17 (Rules)Caller cuts in on chosen turns(bargeIn.afterMs)

In the call (C1 is the only DM in a live call)

Caller turn (3 pts)

C1 (DM, runs its rule today)caller-move: the caller's tactic this turnIn call, each caller turn, only with--adaptive-callerRule today: MOVE_RULE (the caller's plan)
C11 (LLM)What the caller says (and whether to end)caller model; invalid turn → retry with nudge
C12 (Rules)Live keypad press allowed?

Escalation / persona (1 pt)

C5 (DM, runs its rule today)next-call: queue a follow-up call?Between calls, with follow-ups onRule today: none (no follow-up)

Monitor (1 pt)

C13 (LLM)Monitor notes; whisper a hint to the caller?LLM monitor + rule whisperFor; 20,000 ms deadline[S]

Hang-up / retry (3 pts)

C14 (Rules)Stall guard: hint after 3 turns, end after 5
C15 (LLM)Caller model ended the callLLM output + rule
C6 (DM, runs its rule today)repeat-pause: pause a technique that keepsending the same way?Across a sweep, with --pause-repeats. Asked onlyafter the same measured failure ends 2 calls in arow with more queued; deadline ≥15 sNone / timeout / error → continue; 'pause' skipsthe test's remaining queued callsRule today: continue (don't pause)

After the call / across a run

Judging (5 pts)

C18 (LLM)Verdict per callbasic/standard judge / advanced judge
C19 (Rules)Non-live callee → line / unjudged verdict
C20 (Rules)Borderline → full panel
C2 (DM, runs its rule today)judge-escalation: send a verdict to the fullpanel?After call, only if the borderline rules did notescalateRule today: verdict stands
C22 (Rules)Ending and timing facts given to the judge

Cost / report (3 pts)

C21 (Rules)Rule-based failure causes (ambiguous → C3)
C3 (DM, runs its rule today)failure-cause: root cause of an ambiguousfailureAfter runRule today: unplaced
C4 (DM, runs its rule today)fix-area: which area a fix belongs toAfter runRule today: rule routing
Escalation: the ask-human option lives inside C1. Only C1 runs inside a live call; C2–C6 run after the call, in judging or across a run.Total on that map: 22 points (C1–C22). Line Check has no point yet.

[S] set in code/spec. DM = decision model.

callerlab decision map, Oct 13, 2026

Proposed decisions on Oct 13, 2026

Show the planned map (37 points)

Wide map: scroll sideways to read all of it.

The Oct 13 proposal: every point placed by stage, with its type and status at that date (TODAY, BUILT, PLANNED). See the current documentation for what has since shipped. [M] measured[S] set in code or spec, not a priceP1, P2… point numbers

Text version

Planned: 37 decision points on a fully wired streaming call (Media Streams). Every point placed in the call pipeline with its build status. A point with parts at different stages is filed under its newest part.

Status: TODAY, BUILT, PLANNED

Type: Rules, DM, LLM, Classifier

dotted rose = decision model, runs its rule today

By type: 6 DM invocations (P24 is an option of P15) · 6 LLM-driven · 1 classifier · 24 rules. By status (newest part wins): TODAY 10 · BUILT 13 · PLANNED 14. Two-tone chip = mixed parts.

TODAY runs on calls now. BUILT is in the codebase but not yet wired into every call. PLANNED is designed, not built: e.g. the P31 safety stop, the P37 hand-off rule, P15 parallel firing.

Before the call

Pre-dial (7 pts)

P1 (Rules, TODAY)Real-run blockers: STOP, user-set spendcap (if any), scope, allowlist
P2 (Rules, BUILT)Missing setup for a stream call (STT/TTSkeys, tunnel)
P3 (Rules, PLANNED)Tier resolution → caller / TTS / STT /judge / decider models
P4 (Rules, PLANNED)Projected cost vs user-set spend cap, ifany → refusereservation today, ledger planned
P5 (Rules, BUILT)Voice pick: default streaming voice,alternate voice per scenario
P6 (Rules, BUILT)TwiML / params telecom-only (no <Say>,<Gather>, recording, AMD)
P7 (Rules, PLANNED)Line Check precondition: Voice Insightson (warn only)one-off

Pickup / hand-off (2 pts)

P8 (Rules, TODAY)Who answered → hang up on voicemailafter first agent utterance
P37 (Rules, PLANNED)IVR → live-agent hand-off detection(hold music / "this is Dana" → calleekind ivr→live; judge treats later turnsas the agent's)≤ one endpoint today classifyCallee runs on the first agent line only

In the call: listen & speak

Turn-taking (every agent turn) (6 pts)

P9 (Rules, BUILT)Agent turn over? STT endpoint + "soundsunfinished" + VAD + maxListenMsSTT + patienceMs
P10 (Rules, BUILT)STT dropped → batch fallbackon socket close
P11 (Rules, PLANNED)Speculative LLM start on pause; discardif agent resumessaves a wait
P12 (Rules, PLANNED)Agent talks over caller → yield (clear,fade, cancel TTS + LLM)yield after 600 ms today [S], 250 ms target (planned)yield built · LLM/TTS cancel planned
P13 (Rules, BUILT)Caller cuts in on chosen turns(bargeIn.afterMs)afterMs [S] per technique
P14 (Rules, PLANNED)Ignore agent onsets within 150 ms of aclear (echo)150 ms [S]

Escalation / persona (2 pts)

P24 (DM, TODAY, runs its rule today)Ask for a human / escalate persona withintemperamentbranch/option inside the P15 caller-move DM call, not aseparate invocation (6 DM invocations total) · ruletoday + rules allowedMoves
P25 (DM, TODAY, runs its rule today)Follow-up: repeat / add noise / raiselevelrule today · 15 s [S]

Monitor / safety (1 pt)

P31 (LLM, PLANNED)Monitor reading → whisper; plannedsafety stop (abuse, PII, emergency)basic/standard every 2nd turn / advanced every turn;off hot path; 20 s [S]whisper today, cadence planned

In the call: each caller turn

Caller turn (9 pts)

P15 (DM, TODAY, runs its rule today)caller-move: caller tacticrule today · decision model; 600 ms deadline [S]planned: fire at speech-start, in parallel withendpointing
P16 (LLM, PLANNED)What to say + {emotion, intensity 0-3}hosted modelssay today · emotion parse built · tier planned
P17 (Rules, BUILT)Emotion tag valid for persona arc? Clampemotion_clamped
P18 (Rules, BUILT)Emotion → native TTS controlsteering tag / audio tag / fallback
P19 (Rules, BUILT)Sentence chunk → start TTS before LLMfinishesfirst chunk ≥ 8 chars [S]
P20 (Rules, BUILT)Plan-turn validity → retry with nudge+1 LLM call on retry · base today
P21 (Rules, BUILT)Keypad vs speak: fully named field → senddigits
P22 (Rules, TODAY)Key safety: 911 / emergency / N11 / >6digits
P23 (Rules, BUILT)Keys method: REST <Play digits> +reconnect, or in-band tonesupdate ≤2 s [S]; reconnect ≤10 s [S]not yet called by the engine

End and after the call

Hang-up / retry (5 pts)

P26 (Rules, TODAY)Stall guard: hint at 3, end at 5 turns
P27 (Rules, PLANNED)Wall budget / --max-turns / user-setspend cap, if any → hang-upper-call budget [S] · budget today, mid-call spendcheck planned
P28 (LLM, TODAY)Caller model or branch ends the callLLM + rules
P29 (Rules, BUILT)Retry or redial a lost / never-startedcall
P30 (DM, TODAY, runs its rule today)Pause a technique that keeps ending thesame wayrule today: continue (don't pause) · 15 s [S]

Judging (3 pts)

P32 (LLM, TODAY)Verdict + citationsbasic/standard judge, advanced judge; seconds
P33 (DM, PLANNED, runs its rule today)Escalate to panel / advanced re-judgerules → decision model (rule today) → advanced LLMjudge; 15 s [S] DMrules + DM today; tier triggers planned
P34 (Classifier, PLANNED)Affect judge: did the voice carry theemotion?accuracy 0.20 [M] = chance; needs retraining

Line Check (1 pt)

P35 (Rules, PLANNED)Badge from worst leg; red →audio-explainable FAILs becomeinconclusive: carrierasync, retries ≤15 min [S]; amber → carrier-possible

Cost / report (1 pt)

P36 (DM, PLANNED, runs its rule today)Ledger: actual vs estimate,policy_violation; failure cause + fixareaRules + DM (C3/C4, rule today); 15 s [S] DM

Stage	Points today	Planned points	DM today / planned	Status today / built / planned	Biggest change
Pre-dial	C7, C8, C9 (3)	P1–P7 (7)	0 / 0	1 / 3 / 3	Tier resolution, telecom-only guard, setup check
Pickup / hand-off	C10 (1)	P8, P37 (2)	0 / 0	1 / 0 / 1	IVR→agent hand-off rule (planned)
Turn-taking	C16, C17 (2, partial stream)	P9–P14 (6)	0 / 0	0 / 3 / 3	Own endpointing, STT fallback, yield, echo guard
Caller turn	C1, C11, C12 (3)	P15–P23 (9)	1 / 1	2 / 6 / 1	Emotion tag + arc clamp, chunking, keypad-vs-speak, REST DTMF
Escalation / persona	(in C1), C5 (1)	P24, P25 (2)	1 / 2	2 / 0 / 0	—
Hang-up / retry	C14, C15 (2)	P26–P30 (5)	1 (C6) / 1	3 / 1 / 1	Lost-call handling, mid-call spend check
Monitor / safety	C13 (1)	P31 (1)	0 / 0	0 / 0 / 1	Safety stop (planned)
Judging	C2, C18, C19, C20, C22 (5)	P32–P34 (3, merged)	1 / 1	1 / 0 / 2	Tier re-judge triggers, affect judge
Line Check	— (0)	P35 (1)	0 / 0	0 / 0 / 1	New: carrier verdict adjustment
Cost / report	C3, C4, C21 (3)	P36 (1, merged)	2 / 2	0 / 0 / 1	Ledger, policy_violation
Total	22	37	6 / 6 (rule today)	10 / 13 / 14

[M] measured · [S] set in code/spec.

callerlab decision map, Oct 13, 2026

One whole call

This worked call places every decision at its time, so it returns after the re-run.

How a score is made

The Standard defines every score, its interval and how it is versioned.

Read the Standard See the benchmark