Real-time translation: get translated text alongside transcription while speech is happening.
Realtime translation is available on the public gateway. Declare the target languages once, before sending audio. The target set is fixed for the session; start a new session to change it.
Configure translation
Send session.update right after session.created, before the first audio frame:
{
"type": "session.update",
"language": "zh",
"target_languages": ["en", "ja"]
}language: source language of the audio (optional;autoenables detection)target_languages: array of BCP-47 target tags. Up to 8 languages per session.translate_to: single-language shorthand, equivalent totarget_languages: [value]
Both zh_CN and zh-CN are accepted and normalized. Invalid tags are dropped; if no valid tag remains the gateway returns invalid_target_languages.
The gateway confirms the applied state with a session.updated event:
{
"type": "session.updated",
"request_id": "req_...",
"session_id": "sess_...",
"language": "zh",
"language_source": "client",
"target_languages": ["en", "ja"],
"features": {
"asr_correction": false,
"refinement": false,
"translation": true
}
}Session-fixed features
Translation is a session-level feature. The first accepted target_languages value is frozen before audio. If you send a different set later — or try to enable translation after audio has started — the gateway returns:
{
"type": "error",
"code": "session_features_locked",
"message": "Translation target languages are fixed for the session; start a new session to change them."
}This keeps billing deterministic: every audio second in the session is billed with the same feature set.
Translation output
Each target language produces its own live.translation.completed event. Events arrive independently, so a slow language never blocks the source transcript or another language:
{
"type": "live.translation.completed",
"request_id": "req_...",
"session_id": "sess_...",
"utterance_index": 3,
"language": "en",
"text": "We are shipping the new version next week.",
"source_text": "我们下周发布新版本",
"source_language": "zh",
"latency_ms": 180
}text: translated text for this languagesource_text: the raw ASR line the translation is based onutterance_index: matches theconversation.item.input_audio_transcription.completedevent for the same utterance
Translation runs in parallel with correction. It does not delay the source transcript event.
Source and translated text correspondence
Use utterance_index to pair the source transcript with each translation. The source transcript arrives first; translations follow as they complete.
Switching languages
The source language can change at any time during the session:
{ "type": "session.update", "language": "en" }New utterances after the switch use the new source language. Transcripts already delivered keep their previous language.
The translation target set cannot change mid-session. Start a new session if the target languages change.
Latency
Translation is an additional LLM call per utterance per target language. It runs off the transcript hot path, so the source transcript latency is unchanged. Typical translation latency is a few hundred milliseconds; it depends on audio length, context, and provider load.
Pricing
Realtime translation is billed per target language × processed audio duration. See Pricing for the current rate card.
- 1 target language: base realtime rate + 1 × translation rate
- 2 target languages: base realtime rate + 2 × translation rate
- Translation is only charged when
target_languageswas accepted for the session
Multilingual / code-switching
- Language code: set a concrete
languagesuch asenorzhwhen known;autois also accepted and enables detection. A concrete code is more accurate. - Mixed language: upstream supports code-switching, accuracy depends on model and audio quality
- Chinese aliases:
zh-cn,zh-tw,mandarinetc. are normalized tozh
UI strategy
- Receive
conversation.item.input_audio_transcription.completedand display the source text - Receive
live.translation.completedfor each target language - Match by
utterance_indexand display each translation on its own line - No client-side replacement is needed; each utterance is independent
See Build Live Translation for a detailed guide.
Related
- Session Configuration —
session.updatedetails - Live Translation — translation concepts
- Build Live Translation — UI guide