Every AI voice platform ships with a speech-to-text engine picked for you. The engine is baked into the bundle, its word-error rate was measured on the vendor's cleanest dialect, and your callers become the test set nobody ran: crosstalk, headset mics, cellular codecs, a second language mid-sentence. Devotel Orbit takes the opposite position. Speech-to-text is a benchmark-switchable commodity, and the STT Playground, under Voice → STT Playground in the dashboard, is where you prove which engine hears your callers before an agent answers one.
Speech-to-text is now a benchmark-switchable commodity
For most of the category's history, the speech-to-text decision was invisible: the platform vendor paired you with one engine, and the pairing was the product. That pairing is an unnecessary bet. The engines themselves have become close enough that switching is a configuration change rather than a migration, and the quality gap between them moves with the audio mix rather than with the brand.
Orbit models speech-to-text as a catalog of engines, not a single bundled vendor. The same catalog that powers the playground drives the per-agent speech-to-text selection and failover chain in production, so what you evaluate is what your agents run. The architecture behind that split, and when to own it versus rent it, is covered in our STT and TTS orchestration deep dive. Platforms that lock the pairing in, the Vapi, Retell, and Synthflow shape of the category, ship you the bundled default and a support ticket when it mishears your market.
The workflow: one clip, every engine, side by side
The playground takes a recording that is real for your business. Record a clip from the browser microphone or upload one, MP3, WAV, OGG, AAC, or M4A up to 25MB, and one comparison run sends the same clip to every speech engine in the catalog. The results land on a single screen: each engine's transcript, its confidence score, and its response time on that clip. The comparison is fair by construction because every engine hears the same bytes.
Two details make it an evaluation instrument rather than a demo. An engine your account has not provisioned is labeled not yet available instead of being dropped silently, so you always know what the comparison covered. And a run resolves your own registered provider key once one is active under Settings → Voice → Speech-to-text, falling back to the platform credential only when you have not brought one, so the transcript and latency you measure are what your production routing would see. Nothing you record or upload is stored; the clip is discarded when you replace or discard it, which is what makes it safe to evaluate on a real customer call excerpt.
Because the clip is yours, the reference is yours too. Write down what was actually said on the call, which takes minutes for a recording you made, and read each engine's transcript against that human reference line by line. The side-by-side layout is exactly that read done in one pass: a WER-style comparison computed on your audio instead of the vendor's. The broader system around the playground, including how transcripts ground an agent through knowledge bases, is covered in the STT Playground and RAG knowledge bases explainer; this post stays focused on the evaluation itself.
Score engines on your dialect and noise mix
A comparison run only answers a question if you know which question your audio asks. Before reading the transcripts, name your dominant mix: the dialects and accents your callers actually have, the noise floor they call from, the codec your telephony applies on the way in. Then read the side-by-side with that mix in mind, because the mix is exactly where engines diverge.
The practical rule is to score each vendor on your mix instead of the vendor's clean test set. An engine that is two points better on a public benchmark and worse on your cellular codec is the wrong engine for your deployment, and the playground is the cheapest place to find that out: one uploaded clip, one run, no pilot project.
From the playground to the preset your agent runs
A comparison that never changes a production setting is a science fair. In Orbit the engine you validate is the engine you configure: the speech-to-text choice lands in the model preset the voice agent actually runs, next to the LLM and TTS choices and the latency and cost tradeoffs between them. How model presets trade latency against cost and quality covers that configuration surface in detail.
Budget the engine inside your end-to-end latency target, not outside it. Speech-to-text is the first slice of every voice agent's latency budget, and the per-engine response time the playground reports is the number to carry into that budget. Reducing latency in AI voice agents walks the full budget from first audio byte to first spoken token.
Do not ship on published WER
One warning deserves its own heading: do not ship based on one engine's published word-error rate. Published WER is always measured against the vendor's best dialect on clean audio, which means it is a score on a test set the vendor chose. On your mix it is a prior at best. Run your audio, keep your reference, and let the side-by-side decide; if a vendor objects that its benchmark should be enough, that objection is itself a result.
Three runs to start with
Three clips cover the failure modes that clean benchmarks hide:
- A sales call with crosstalk. Two speakers talking over each other is where engines diverge most: some merge both voices into one garbled transcript, others keep each speaker legible. The winner is the engine whose transcript still matches your reference at the overlap, the difference between an agent that answers the customer and one that answers the average of the room.
- IVR input on a bad cellular line. Short utterances and spoken digits on codec-degraded audio separate telephony-trained engines from meeting-room engines. Check the digit strings and short confirmations against your reference; confidence that looked fine on the sales call drops exactly here for engines that never met telephony noise in training.
- A multilingual caller, ideally one who switches language mid-call. Language detection and code-switching are where bundled defaults fail silently, transcribing a second language with first-language confidence. The winner detects the switch and keeps both languages readable in the transcript.
In today's catalog, Deepgram is the first engine with a wired comparison client; the remaining catalogued engines, Soniox, Speechmatics, AssemblyAI, and xAI, show as not yet available until provisioned rather than as fabricated results. That honesty is the point of the instrument: the same clip stays one click from a rerun as each engine comes online for your account, and the winner of each run is whichever engine's transcript keeps matching your reference on your mix.
Related reading
- AI voice infrastructure: the STT and TTS orchestration decision, the architecture behind switchable speech engines.
- Reduce latency in AI voice agents, placing the STT choice inside the end-to-end latency budget.
- Model presets for voice agents, the production setting your comparison flows into.