Our voice-agent latency budget, published in the open
We publish our voice-agent latency budget openly, broken down by stage and backed by the full methodology, because customer experience starts here. The figures below are the voice path's per-stage p50/p95 SLO targets — the budget the production wire path is engineered against.
Target p50
1100 ms
Steady-state turn, post-streaming chunker
Target p95
1500 ms
End-to-end turn envelope
Per-stage budget (E1–E5)
Every turn passes through five measurable stages. The total is the sum minus parallel overlap (LLM TTFT and TTS first byte run concurrently once the streaming chunker is engaged).
| Stage | What it measures | Target |
|---|---|---|
| Endpointing (E1) | Time from the user finishing their utterance to Deepgram's `UtteranceEnd` event. Driven primarily by the `DEVOTEL_VOICE_UTTERANCE_END_MS` setting (default 600 ms). | 600 ms |
| STT final (E2) | Time from Deepgram's `UtteranceEnd` to the final `Transcript.is_final = true` event with the full transcript. Includes Deepgram's nova-2 / nova-3 model post-processing. | 50 ms |
| LLM TTFT (E3) | Time-to-first-token from Anthropic Claude Sonnet via the agent-runtime `/chat/stream` endpoint. With prompt-cache hit (warm session) we typically see 300-400 ms; cold turn-1 hits 600-1500 ms but is masked by the W10-G pre-warm IIFE. | 400 ms |
| First chunk → TTS (E4) | Time from receiving the first streamed LLM chunk to handing it to the TTS pipeline. Pure orchestration latency; bounded by the sentence-chunker's boundary detection. | 100 ms |
| TTS first byte (E5) | Time from TTS handoff to the first audio byte arriving from Cartesia / ElevenLabs failover. Sub-150ms with Cartesia Sonic; ElevenLabs Flash v2.5 fallback adds ~50 ms. | 150 ms |
| These are the published per-stage SLO targets the voice path is engineered against. Measured per-stage figures may be added from production telemetry in a future update. | ||
Sum of targets: 1300 ms (parallelism collapses ~200 ms — net p50 target 1100 ms).
Methodology
Every voice turn emits a Prometheus histogramvoice_agent_turn_stage_duration_mswith labelsstage / tenant_org_id / model. The per-stage budget on this page is the E1–E5 SLO target the production wire path is engineered around — Deepgram nova-3 STT, Anthropic Claude Sonnet 4.6 via the agent-runtime /chat/stream endpoint, Cartesia Sonic TTS with the ElevenLabs Flash v2.5 failover armed, measured from Orbit's self-hosted media plane (europe-west1).
The histogram is fail-open: if the metrics module is absent (boot-check harness, local dev) the timing accumulator silently becomes a no-op, so a misconfigured probe never takes the voice path down. Verification path documented atapps/voice-gateway/src/lib/voice-metrics.ts.
The per-stage budget above is the published SLO target the voice path is engineered against. If a scheduled probe against Google Cloud Monitoring later captures post-migration production percentiles, this page can add the measured p50/p95 alongside these targets.
How we compare
Public competitor numbers, attributed inline and grouped by how they were obtained — independently measured versus vendor-marketed. Orbit's own row is its internal p50 target, shown apart from the measured competitor bars rather than ranked against them.
Internal SLO target · sum of the E1–E5 per-stage budget, not a measured number. We publish this openly for procurement transparency so you can see the latency footprint we engineer against. Competitors' p50s below are independently measured or vendor-marketed figures from their own claims or tested.media.
Prior synthetic number · the only measured Orbit latency we have () was captured in a synthetic test environment on LiveKit Cloud media plane (europe-west1), decommissioned 2026-05-19 after the 2026-05-16 migration to Orbit's self-hosted media SFU. That retired wire path no longer exists, so those numbers are superseded and do not appear above. Post-migration measured values are pending production telemetry.
Independently measured
Third-party p50 from a common test harness across vendors (tested.media, March 2026).
Independent · tested.media March 2026 · Marketed ~600 ms; independent p50 680 ms / p95 920 ms. Best in AI-voice-native cohort.
Independent · tested.media March 2026 · Marketed <500 ms; independent p50 720 ms / p95 1050 ms. Flexibility tax shows in tail latency.
Independent · tested.media March 2026 · Marketed 400 ms historically (no longer published); independent p50 850 ms / p95 1180 ms.
Vendor-marketed (unverified)
Best-case figures each vendor publishes. Not independently verified — several measure slower in the cohort above.
Marketed · Open-source LiveKit Agents framework claim · Claims sub-200 ms when self-hosted on your own media infrastructure; no independent number published.
Marketed · elevenlabs.io Turbo tier · Turbo v2 ~400 ms marketed (gpt-4o-mini + Flash v2).
Marketed · Developer reports, 2026-Q1 · Native speech-to-speech; <500 ms reported in early-access community. No published p95.
What moves the number
Real-world latency is sensitive to factors we don't control from a single page:
- Region. Live media can be pinned per-room to EU or US regions via the
regionfield at room-create time for lower latency and data residency. AI agent inference can also run in-region for compliance boundaries. The figures on this page are measured fromeurope-west1. - Model selection. Claude Sonnet 4.6 is the production default. A faster classifier-tier model (Claude Haiku) shaves 100–200 ms off the LLM TTFT stage for simple intents — tunable per-agent in the dashboard.
- Prompt length. Prompt caching (Anthropic
extended-cache-ttl-2025-04-11beta) keeps the system prompt + tool registry pinned for the session. Optional cold-turn pre-warm heats the cache before the first turn, so the opening reply doesn't pay a cold-start penalty. - Tool calls. Each LLM-initiated tool call adds round-trip latency. Async tool support — on the roadmap — lets long-running tools resolve in the background without blocking the next turn.
- Speculative-on-partial. We pre-fire the LLM on a stable interim transcript so the model is already streaming when STT emits final. It's tunable per agent — turn it off if you'd rather have a deterministic cold-floor.
Test our voice agent yourself
Spin up a sandbox tenant, attach a knowledge base, place a test call — the dashboard renders the per-turn stage histogram inline so you can verify these numbers on your own traffic.