Realtime Overview

Realtime is a Voice Context Layer mode for live speech. Recorded is the other mode, for full-file processing. See Voice Context Layer for the umbrella model.

The Realtime API provides streaming speech transcription over WebSocket. You send PCM16LE audio frames, the server returns transcription events.

What it does

• Live captions: text appears while the person is still speaking
• Real-time translation: source and translated text together (coming soon for external sessions)
• Multi-language: Whisper-compatible language support
• Server-side VAD: automatic speech segment detection, no client-side splitting needed

Input

• Audio format: PCM16LE / 16kHz / mono
• Transport: JSON text frames (base64) or binary frames (raw PCM)
• Frame size: recommended 100ms frames (~3200 bytes), max 1 MiB

Output

Server pushes JSON events. The primary output is:

• conversation.item.input_audio_transcription.completed — a speech segment has been transcribed

Supporting events:

• input_audio_buffer.speech_started / speech_stopped — speech segment boundaries
• session.created — connection established
• error — error events

Architecture

Your app                   LansonAI Gateway                   Upstream STT
  │                            │                                  │
  ├── WS connect (Bearer) ───→ session.created                   │
  │                            │                                  │
  ├── Send PCM16 frames ──────→ gateway relay (binary PCM) ─────→ VAD + STT
  │                            │                                  │
   │←── input_audio_buffer.speech_started ──────│←── input_audio_buffer.speech_started
   │                            │                                  │
   │←── conversation.item.input_audio_transcription.completed ─│←── conversation.item.input_audio_transcription.completed
  │                            │                                  │
  ├── Close ─────────────────→ meter flush → R2 ledger            │

vs. Segment and Batch

Next steps

• Realtime Quickstart — runnable example
• Connection Lifecycle — connect, timeout, close
• Audio Input — audio format reference