Our voice-agent latency budget, published in the open
We publish our voice-agent latency budget openly, broken down by stage and backed by the full methodology, because customer experience starts here. The figures below are the voice path's per-stage p50/p95 SLO targets — the budget the production wire path is engineered against.
Target p50
1100 ms
Steady-state turn, post-streaming chunker
Target p95
1500 ms
End-to-end turn envelope
Per-stage budget (E1–E5)
Every turn passes through five measurable stages. The total is the sum minus parallel overlap (LLM TTFT and TTS first byte run concurrently once the streaming chunker is engaged).
| Stage | What it measures | Target |
|---|---|---|
| Endpointing (E1) | Time from the user finishing their utterance to Deepgram's `UtteranceEnd` event. Driven primarily by the `DEVOTEL_VOICE_UTTERANCE_END_MS` setting (default 600 ms). | 600 ms |
| STT final (E2) | Time from Deepgram's `UtteranceEnd` to the final `Transcript.is_final = true` event with the full transcript. Includes Deepgram's nova-2 / nova-3 model post-processing. | 50 ms |
| LLM TTFT (E3) | Time-to-first-token from Anthropic Claude Sonnet via the agent-runtime `/chat/stream` endpoint. With prompt-cache hit (warm session) we typically see 300-400 ms; cold turn-1 hits 600-1500 ms but is masked by the W10-G pre-warm IIFE. | 400 ms |
| First chunk → TTS (E4) | Time from receiving the first streamed LLM chunk to handing it to the TTS pipeline. Pure orchestration latency; bounded by the sentence-chunker's boundary detection. | 100 ms |
| TTS first byte (E5) | Time from TTS handoff to the first audio byte arriving from Cartesia / ElevenLabs failover. Sub-150ms with Cartesia Sonic; ElevenLabs Flash v2.5 fallback adds ~50 ms. | 150 ms |
| Live per-stage figures publish here once post-migration telemetry lands. | ||
Sum of targets: 1300 ms (parallelism collapses ~200 ms — net p50 target 1100 ms).
Methodology
Every voice turn emits a Prometheus histogramvoice_agent_turn_stage_duration_mswith labelsstage / tenant_org_id / model. The per-stage budget on this page is the E1–E5 SLO target the production wire path is engineered around — Deepgram nova-3 STT, Anthropic Claude Sonnet 4.6 via the agent-runtime /chat/stream endpoint, Cartesia Sonic TTS with the ElevenLabs Flash v2.5 failover armed, served from Orbit's self-hosted media plane (europe-west1).
The histogram is fail-open: if the metrics module is absent (boot-check harness, local dev) the timing accumulator silently becomes a no-op, so a misconfigured probe never takes the voice path down. Verification path documented atapps/voice-gateway/src/lib/voice-metrics.ts.
Once post-migration production telemetry begins emitting, this page refreshes automatically from a scheduled probe against Google Cloud Monitoring, replacing the targets above with the live measured p50/p95.
How we compare
Public competitor numbers, attributed inline and grouped by how they were obtained — independently measured versus vendor-marketed. Orbit's own row is its internal p50 target, shown apart from the measured competitor bars rather than ranked against them.
Internal SLO target · sum of the E1–E5 per-stage budget, not a measured number — so we don't rank it as a bar against competitors' measured p50s. Live measured percentiles republish here once post-migration telemetry re-baselines on Orbit's self-hosted media plane.
Independently measured
Third-party p50 from a common test harness across vendors (tested.media, March 2026).
Independent · tested.media March 2026 · Marketed ~600 ms; independent p50 680 ms / p95 920 ms. Best in AI-voice-native cohort.
Independent · tested.media March 2026 · Marketed <500 ms; independent p50 720 ms / p95 1050 ms. Flexibility tax shows in tail latency.
Independent · tested.media March 2026 · Marketed 400 ms historically (no longer published); independent p50 850 ms / p95 1180 ms.
Vendor-marketed (unverified)
Best-case figures each vendor publishes. Not independently verified — several measure slower in the cohort above.
Marketed · livekit.io public claim · Claims sub-200 ms with their own global media-server mesh; no independent number published.
Marketed · elevenlabs.io Turbo tier · Turbo v2 ~400 ms marketed (gpt-4o-mini + Flash v2).
Marketed · Developer reports, 2026-Q1 · Native speech-to-speech; <500 ms reported in early-access community. No published p95.
What moves the number
Real-world latency is sensitive to factors we don't control from a single page:
- Region. Today we serve voice from
europe-west1. A US-East caller eats ~100 ms RTT on every hop. Multi-region (us-east-1 + ap-south-1) is on the 90-day roadmap. - Model selection. Claude Sonnet 4.6 is the production default. A faster classifier-tier model (Claude Haiku) shaves 100–200 ms off the LLM TTFT stage for simple intents — tunable per-agent in the dashboard.
- Prompt length. Prompt caching (Anthropic
extended-cache-ttl-2025-04-11beta) keeps the system prompt + tool registry pinned for the session. Optional cold-turn pre-warm heats the cache before the first turn, so the opening reply doesn't pay a cold-start penalty. - Tool calls. Each LLM-initiated tool call adds round-trip latency. Async tool support — on the roadmap — lets long-running tools resolve in the background without blocking the next turn.
- Speculative-on-partial. We pre-fire the LLM on a stable interim transcript so the model is already streaming when STT emits final. It's tunable per agent — turn it off if you'd rather have a deterministic cold-floor.
Test our voice agent yourself
Spin up a sandbox tenant, attach a knowledge base, place a test call — the dashboard renders the per-turn stage histogram inline so you can verify these numbers on your own traffic.