How silence, pauses, and speech interruptions are handled in realtime transcription.
VAD and speech segmentation
Server-side VAD automatically detects speech segments. The WebSocket events pushed to your client are:
- Speech detected →
input_audio_buffer.speech_started(abbreviated asspeech_started) - Speech continues → audio accumulated
- Silence detected →
input_audio_buffer.speech_stopped(abbreviated asspeech_stopped) - Transcription complete →
conversation.item.input_audio_transcription.completed(abbreviated ascompleted)
Use the full type string in your event dispatcher. See Server Events for the complete payload schemas.
Clients do not need to implement VAD — just consume events.
Silence behavior
Short pauses
Normal speaking pauses (commas, sentence ends) do not trigger segment splits. The server-side VAD has a fixed silence threshold; only silence exceeding this duration triggers end-of-speech.
Long silence
- Silence exceeding
idle_timeout_secondscloses the connection with code4408. - The actual value is returned in
session.created.limits.idle_timeout_seconds; do not hard-code it. - To keep the connection alive, continue sending PCM16LE audio frames, even when they contain silence (all-zero samples).
For example, 100 ms of silence at 16 kHz mono is new Int16Array(1600).fill(0), sent as a base64 input_audio_buffer.append frame. See Audio Input for frame-size guidance and Connection Lifecycle for close-code details.
Manual flush
input_audio_buffer.flush (and its alias flush) is accepted for compatibility, but the current public session handler does not implement a manual-flush action:
{ "type": "input_audio_buffer.flush" }Do not use this message to force a final result. Continue sending real audio, including silence, so server-side VAD can identify a boundary, and wait for the completed event. If your application needs explicit client-controlled boundaries, use the Segment API for each clip instead. See LiveFinal™ for finalization behavior.
Max speech segment duration
The server-side VAD caps the maximum duration of a single speech segment. When exceeded:
- The current segment is finalized:
input_audio_buffer.speech_stopped→conversation.item.input_audio_transcription.completed. - A new segment starts immediately with
input_audio_buffer.speech_started. - The
reasonfield ofspeech_stoppedexplains why the segment ended (e.g.end_of_speechormax_duration). See Server Events for the full list ofreasonvalues.
Interruption handling
Speaker interrupted
In live scenarios, if a speaker is interrupted:
- Upstream detects the speech boundary.
- Current segment:
input_audio_buffer.speech_stopped→conversation.item.input_audio_transcription.completed. - A new segment starts with
input_audio_buffer.speech_started.
Client strategy
- Each
conversation.item.input_audio_transcription.completedevent is independent and final. - No need to cancel or roll back already-displayed text.
- New segments are ordered by
utterance_index. If two segments overlap in time, useaudio_duration_ms/speech_duration_msto position them on a timeline rather than relying on arrival order.
Recommended client handling
ws.onmessage = (event) => {
if (typeof event.data !== "string") return;
const data = JSON.parse(event.data);
if (data.type === "input_audio_buffer.speech_started") {
showListeningIndicator(data.utterance_index);
} else if (data.type === "input_audio_buffer.speech_stopped") {
showProcessingIndicator(data.utterance_index);
} else if (data.type === "conversation.item.input_audio_transcription.completed") {
appendTranscript(data.utterance_index, data.text);
hideIndicators(data.utterance_index);
}
};Related
- Transcript Lifecycle — state transitions
- Session Configuration — session options
- Connection Lifecycle — idle timeout and reconnect
- Client Messages —
input_audio_buffer.flush - Server Events — event schemas and
reasonvalues