WebSocket vs REST TTS APIs for Voice Agents

Choosing the wrong protocol for your WebSocket TTS integration adds architectural debt that compounds every time you scale. a persistent WebSocket connection can save 50 to 100 milliseconds per request compared to REST in multi-turn conversations. for voice agents running at scale, that latency gap translates directly to interrupted callers and a broken experience.

If your voice agent takes longer than 500 milliseconds to respond, you have already lost the conversation. humans hand off conversational turns in roughly 200 milliseconds. every millisecond spent opening HTTP connections is a millisecond you cannot spend on LLM reasoning.

How REST and WebSocket handle audio delivery

Protocol choice comes down to one question: can playback start before synthesis finishes?

REST text-to-speech follows a straightforward pattern. you send text, the server synthesizes the entire audio file, and you get the complete response back. the trade-off is clear. you avoid connection state, keep-alives, and reconnection logic. but your user waits for synthesis to complete before hearing anything.

The WebSocket protocol establishes a persistent, bidirectional connection. once that connection is open, your TTS provider sends audio chunks as they are generated. you can start playback while synthesis continues.

HTTP chunked transfer encoding splits the difference. you make a standard HTTP request, but the server sends the response in chunks. you get progressive audio delivery without WebSocket’s connection management overhead. Speechify supports this via the /v1/audio/stream endpoint, returning raw PCM chunks. I wrote about this approach in streaming TTS directly to the browser with Web Audio API.

When REST is the right call for TTS

REST is the right choice when the user can wait for a complete audio asset or when you want simple, stateless failure handling.

If you are pre-generating podcast intros, narrating help articles, or producing audiobook chapters, streaming adds little value. you need the complete file. REST’s stateless architecture means simpler retries and easier caching.

For short, infrequent prompts, the overhead of maintaining a persistent WebSocket connection can outweigh the benefit. REST’s per-request overhead is negligible at low volume, and operational complexity stays low.

When WebSocket wins for voice agents

WebSocket TTS earns its complexity when your text source is itself a stream (LLM token output) and when interruption handling changes user experience.

Voice agents need to start speaking before they have finished “thinking.” when your LLM generates tokens incrementally and your TTS needs to synthesize incrementally, WebSocket is the cleanest architecture. the production pattern is usually LLM tokens flowing into a buffer, that buffer feeding a streaming TTS API over WebSocket, and audio chunks streaming to the client for immediate playback.

In a multi-turn conversation with ten or more exchanges, connection reuse adds up quickly. REST pays connection overhead on every request. WebSocket pays it once, then reuses the connection. at scale, multiplexing sessions over a persistent WebSocket connection shaves another 50 milliseconds off time-to-first-audio.

Interruption is also normal. your pipeline should treat TTS output as cancellable work, not a fire-and-forget response. this is where WebSocket control messages and session state pay for themselves. if the user barges in, you can send a flush command over the open socket immediately.

How telephony changes the protocol math

For PSTN voice agents, WebSocket’s transport advantage often gets partially absorbed by the rest of the telephony pipeline.

PSTN’s 8kHz sampling rate creates a 4kHz audio bandwidth ceiling. any wideband TTS audio gets downsampled before reaching the caller. the practical implication is simple. protocol choice will not rescue a poor telephony audio path. if your agent sounds bad, the fix is usually codec selection and transcoding hygiene, not switching REST to WebSocket. use ulaw_8000 for telephony endpoints to avoid transcoding hops.

Your TTS output still has to be packetized into fixed-duration RTP frames and smoothed by jitter buffers. if upstream audio arrives in uneven chunks, your jitter buffer can grow to compensate, erasing any protocol-level win.

Making the decision

You can usually pick the right protocol by checking three things.

First, look at your text source. if it is a stream of LLM tokens, that points to WebSocket. if it is a complete block of CMS content, that points to REST.

Second, look at your playback target. if the user needs audio immediately, interactive playback points to WebSocket or HTTP chunking. async delivery points to REST.

Third, look at your connection state tolerance. if you want to avoid managing reconnect storms during deploys, HTTP chunking gives you progressive playback without the persistent socket overhead.

Frequently asked questions

Which protocol has the lowest TTS latency?

WebSocket typically offers the lowest time-to-first-audio for multi-turn conversations because it eliminates connection setup overhead after the first request. multiplexing sessions over a single WebSocket can save an additional 50 milliseconds per turn compared to opening new connections.

Can I stream TTS audio without WebSockets?

yes. HTTP chunked transfer encoding allows progressive audio delivery over a standard REST-style POST request. the Speechify /v1/audio/stream endpoint uses this to stream raw PCM audio chunks back to the client as they generate.

Why do voice agents need streaming TTS?

voice agents use streaming TTS to start speaking before the LLM has finished generating the complete response. by feeding LLM tokens into a streaming TTS API at sentence boundaries, the agent can begin audio playback while the next sentence is still generating. this keeps latency under the 500-millisecond threshold required for natural conversation.

What is the best TTS model for voice agents?

Simba 3.2 is the recommended model for English voice agents on the Speechify API. it is optimized for low-latency streaming and sits at the top of independent leaderboards like Artificial Analysis for voice quality and naturalness.