Why voice latency under 300ms changes everything

Human conversation has its own rhythm. Studies on turn-taking in dialogue put the average gap between one person finishing a sentence and the other starting theirs at somewhere between 200 and 300 milliseconds — the length of a comfortable breath, not a pause.

Any system that reads a call in real time is competing with that rhythm. A transcript or an alert that lands a second late is not late by human standards — it is late by conversation standards, which is a much smaller number.

Above roughly half a second of trailing latency, a real-time system can only ever be a summarizer — useful after the call, unhelpful during it. Below 300ms, three things become possible that were not possible before:

Mid-call prompts an agent can actually use

A compliance reminder that appears four seconds after the moment it was needed is a note for the file. One that appears while the sentence is still being spoken is a save.

Escalation before the caller asks for it

Frustration has a shape in speech — shorter sentences, flatter pitch, repeated phrasing. Catching that shape early enough to route the call before the caller says “let me speak to a manager” requires the model to be nearly caught up to the audio, not trailing it.

Live coaching instead of retrospective coaching

A supervisor whispering a suggestion into an agent’s ear only works if the suggestion is timely enough to change what happens next in that same call.

We got under 300ms by refusing to treat transcription as a single monolithic step. Audio is chunked and streamed continuously; a fast, lower-precision pass produces a working transcript within tens of milliseconds, while a slower, more accurate pass corrects it a few hundred milliseconds behind — close enough that the correction usually lands before a human would have noticed the first version was slightly wrong.

The lesson generalizes past voice: for any system meant to act inside a live process, the interesting engineering problem is rarely raw accuracy. It is how much accuracy you can afford to trade for the right amount of speed.