--- summary: "Realtime voice conversations, tool policy, agent voice context, and streaming transcription" read_when: - You want a full-duplex realtime voice model on a call - You are tuning realtime tool policy or agent consult - You are streaming Twilio call audio to a transcription provider title: "Voice call realtime and streaming" sidebarTitle: "Realtime and streaming" --- Full-duplex realtime voice, hangup detection, tool and consult policy, agent voice context, and Twilio Media Streams transcription. Part of the [Voice call plugin](/plugins/voice-call) guide. ## Realtime voice conversations `realtime` selects a full-duplex realtime voice provider for live call audio. It is separate from `streaming`, which only forwards audio to realtime transcription providers. `realtime.enabled` cannot be combined with `streaming.enabled`. Pick one audio mode per call. Runtime behavior: - `realtime.enabled` is supported for Twilio and Telnyx. - `realtime.provider` is optional. If unset, Voice Call selects the first configured realtime voice provider in provider priority order. Providers named in `realtime.providers` are discovered even when another provider is already active; plugin disablement and allow/deny rules still apply. - Bundled realtime voice providers: Google Gemini Live (`google`) and OpenAI (`openai`), registered by their provider plugins. - Provider-owned raw config lives under `realtime.providers.`. - On models that support function tools, Voice Call exposes the built-in `openclaw_end_call` realtime tool. It takes no arguments or call ID; the active voice bridge binds it to the current call. - Voice Call exposes the shared `openclaw_agent_consult` realtime tool by default. GPT-Live uses native delegation to the same call-owned agent consult instead. The realtime model can delegate when the caller asks for deeper reasoning, current information, or normal OpenClaw tools. - `realtime.consultPolicy` optionally adds guidance for when the realtime model should call `openclaw_agent_consult`. - On hosts with the shared context resolver, Voice Call always tells the realtime model that it speaks for an OpenClaw agent that may have other sessions and work. `realtime.agentContext.enabled` is default-off and controls the additional configured identity and profile-file context. Supported older hosts retain the [legacy context behavior](/plugins/voice-call/realtime-and-streaming#agent-voice-context). - `realtime.fastContext.enabled` is default-off. When enabled, Voice Call first searches indexed memory/session context for the consult question and returns authorized snippets to the realtime model within `realtime.fastContext.timeoutMs` before falling back to the full consult agent only if `realtime.fastContext.fallbackToConsult` is true. The active memory plugin authorizes session-transcript hits; plugins without that capability fail closed for session hits while ordinary memory hits remain available. - If `realtime.provider` points at an unregistered provider, or no realtime voice provider is registered at all, Voice Call logs a warning and skips realtime media instead of failing the whole plugin. - `inboundPolicy` must not be `"disabled"` when `realtime.enabled` is true; `validateProviderConfig` rejects that combination. - Consult session keys reuse the stored call session when available, then fall back to the configured `sessionScope` (`per-phone` by default, `per-call` for isolated calls, or `main` for the configured agent's main session). GPT-Live uses agent delegation instead of native function tools. Its current Voice Call bridge cannot invoke `openclaw_end_call` or custom `realtime.tools`. Use an OpenAI GA realtime model or Google Gemini Live when the call needs those controls; selecting GPT-Live does not make them available through delegation. ### GPT-Live Voice Call uses the same Gateway-owned GPT-Live bridge as Discord and Talk. Select `gpt-live-1-codex` with `cove` to use the ChatGPT OAuth route; it tries the routed agent's OpenClaw ChatGPT profile first, then the configured Platform key, API-key profile, and `OPENAI_API_KEY`. Select `gpt-live-1` with `marin` for the public Platform API route. Leaving the model unset preserves Voice Call's provider default. ```json5 { plugins: { entries: { "voice-call": { config: { realtime: { enabled: true, provider: "openai", consultPolicy: "auto", providers: { openai: { model: "gpt-live-1-codex", voice: "cove" }, }, }, }, }, }, }, } ``` The bridge converts carrier G.711 mu-law audio at 8 kHz to and from the model's 24 kHz PCM stream. GPT-Live receives microphone input during playback and owns speech interruption; Voice Call does not add local speech-triggered cancellation. Initial greetings and `voicecall.speak` requests use the same native session context path. Delegated work retains the call's agent, tool policy, and cancellation lifetime. GPT-Live rejects `realtime.consultPolicy: "always"`: it owns delegation and cannot enforce host-triggered transcript consults. Use `"auto"` or `"substantive"` guidance, or choose a model supporting host-controlled turns. `realtime.toolPolicy: "none"` disables the agent consult for native delegation too. The end-call and custom function-tool limitations above still apply. ### Hangup detection Realtime calls normally end when the carrier sends a stream stop event or closes the media WebSocket. If an intermediary does not promptly forward that close, OpenClaw treats 30 seconds without inbound media as a disconnect, waits a 2-second grace period for media to resume, and then ends the call. If the realtime provider ends its session first, OpenClaw also ends the carrier call, including when the provider reports a normal close. This prevents a silent phone connection from remaining open after its voice session has finished. Models supporting function tools can also call `openclaw_end_call` when the caller asks to hang up. The model must speak any final words before calling the tool: a successful call ends the current provider session and phone connection immediately, so no later reply is spoken. If the carrier cannot end the call, the bridge stays connected and the model receives an error it can explain to the caller. Configured `realtime.tools` cannot replace this built-in by name. For inbound Twilio numbers, also configure a Status Callback using `POST` to your public webhook URL with `?type=status` appended, for example `https://voice.example.com/voice/webhook?type=status`. Include the `completed` call event. OpenClaw-created outbound calls configure their callback automatically. The callback provides the fastest teardown signal, while stream close and the inactivity backstop remain independent of it. ### Tool policy `realtime.toolPolicy` controls only the consult run. It never disables `openclaw_end_call` on models that support function tools: | Policy | Behavior | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `safe-read-only` | Expose the consult tool and limit the regular agent to `read`, `web_search`, `web_fetch`, `x_search`, `memory_search`, and `memory_get`. | | `owner` | Expose the consult tool and let the regular agent use the normal agent tool policy. | | `none` | Disable the consult tool and native agent delegation. On models supporting function tools, the built-in end-call tool and custom `realtime.tools` remain available. | `realtime.consultPolicy` guides the realtime model. `always` also enables a host transcript fallback when the provider does not consult, and is unsupported with GPT-Live: | Policy | Guidance | | ------------- | ----------------------------------------------------------------------------------------------- | | `auto` | Keep the default prompt and let the provider decide when to call the consult tool. | | `substantive` | Answer simple conversational glue directly and consult before facts, memory, tools, or context. | | `always` | Consult before every substantive answer. | When a host tool run reports cancellation, the realtime model receives a cancelled result and the phone call stays open. Timeouts and other tool failures remain errors; ending the phone session suppresses pending consult results. ### Agent voice context On hosts with the shared context resolver, every realtime session includes an agent-context paragraph explaining that the voice model speaks for an OpenClaw agent with multiple sessions. It directs questions about other sessions, running work, progress, or priorities to OpenClaw. This paragraph stays present when `realtime.agentContext.enabled` is `false`. Enable `realtime.agentContext` to add the configured agent's identity and selected profile files for ordinary voice turns. `includeIdentity` controls the configured name, emoji, vibe, theme, and creature/persona fields; `includeWorkspaceFiles` controls the files listed in `files`. The shared core resolver loads `IDENTITY.md`, `USER.md`, and `SOUL.md` through the normal bootstrap path, honoring bootstrap hooks and workspace access. Other workspace-relative files use safe workspace reads; missing or unreadable files are skipped. `maxChars` bounds the profile-file block, with a default of 6000 characters, and excludes the agent-context paragraph and configured identity. OpenClaw 2026.9.6 lacks that shared resolver. On this supported host, Voice Call retains its shipped optional context capsule: `enabled: false` omits the capsule; when enabled, identity fields and selected safe workspace-relative files follow their respective controls. `maxChars` bounds the entire optional capsule, including identity, headings, and the truncation marker. The newer multi-session paragraph is unavailable on this path. This compatibility path will retire when the supported host floor includes the shared context resolver. Context is added when the realtime session is created, so it does not add per-turn latency. Calls to `openclaw_agent_consult` still run the full OpenClaw agent and should be used for tool work, current information, memory lookups, or workspace state. ```json5 { plugins: { entries: { "voice-call": { config: { agentId: "main", realtime: { enabled: true, provider: "google", toolPolicy: "safe-read-only", consultPolicy: "substantive", agentContext: { enabled: true, maxChars: 6000, includeIdentity: true, includeWorkspaceFiles: true, files: ["SOUL.md", "IDENTITY.md", "USER.md"], }, }, }, }, }, }, } ``` ### Realtime provider examples Defaults: API key from `realtime.providers.google.apiKey`, `GEMINI_API_KEY`, or `GOOGLE_API_KEY`; model `gemini-3.1-flash-live-preview`; voice `Kore`. `sessionResumption` and `contextWindowCompression` default on for longer, reconnectable calls. Use `silenceDurationMs`, `startSensitivity`, and `endSensitivity` to tune faster turn-taking on telephony audio. ```json5 { plugins: { entries: { "voice-call": { config: { provider: "twilio", inboundPolicy: "allowlist", allowFrom: ["+15550005678"], realtime: { enabled: true, provider: "google", instructions: "Speak briefly. Call openclaw_agent_consult before using deeper tools.", toolPolicy: "safe-read-only", consultPolicy: "substantive", consultThinkingLevel: "low", consultFastMode: true, agentContext: { enabled: true }, providers: { google: { apiKey: "${GEMINI_API_KEY}", model: "gemini-3.1-flash-live-preview", speakerVoice: "Kore", silenceDurationMs: 500, startSensitivity: "high", }, }, }, }, }, }, }, } ``` ```json5 { plugins: { entries: { "voice-call": { config: { realtime: { enabled: true, provider: "openai", providers: { openai: { apiKey: "${OPENAI_API_KEY}" }, }, }, }, }, }, }, } ``` See [Google provider](/providers/google) and [OpenAI provider](/providers/openai) for provider-specific realtime voice options. ## Streaming transcription `streaming` connects Twilio Media Streams to a realtime transcription provider. The classic streaming path requires `provider: "twilio"`; configuration with Telnyx, Plivo, or mock is rejected. Telnyx live audio uses the separately authenticated `realtime.enabled` path instead. Runtime behavior: - `streaming.provider` is optional. If unset, Voice Call selects the first configured realtime transcription provider in provider priority order. Providers named in `streaming.providers` are discovered even when another provider is already active; plugin disablement and allow/deny rules still apply. - Bundled realtime transcription providers: Deepgram (`deepgram`), ElevenLabs (`elevenlabs`), Mistral (`mistral`), OpenAI (`openai`), and xAI (`xai`), registered by their provider plugins. - Provider-owned raw config lives under `streaming.providers.`. - After Twilio sends an accepted stream `start` message, Voice Call registers the stream immediately, queues inbound media through the transcription provider while the provider connects, and starts the initial greeting only after realtime transcription is ready. - If `streaming.provider` points at an unregistered provider, or none is registered, Voice Call logs a warning and skips media streaming instead of failing the whole plugin. ### Streaming provider examples Defaults: API key `streaming.providers.openai.apiKey` or `OPENAI_API_KEY`; model `gpt-4o-transcribe`; `silenceDurationMs: 800`; `vadThreshold: 0.5`. ```json5 { plugins: { entries: { "voice-call": { config: { streaming: { enabled: true, provider: "openai", streamPath: "/voice/stream", providers: { openai: { apiKey: "sk-...", // optional if OPENAI_API_KEY is set model: "gpt-4o-transcribe", silenceDurationMs: 800, vadThreshold: 0.5, }, }, }, }, }, }, }, } ``` Defaults: API key `streaming.providers.xai.apiKey` or `XAI_API_KEY` (falls back to an xAI OAuth auth profile if neither is set); endpoint `wss://api.x.ai/v1/stt`; encoding `mulaw`; sample rate `8000`; `endpointingMs: 800`; `interimResults: true`. ```json5 { plugins: { entries: { "voice-call": { config: { streaming: { enabled: true, provider: "xai", streamPath: "/voice/stream", providers: { xai: { apiKey: "${XAI_API_KEY}", // optional if XAI_API_KEY is set endpointingMs: 800, language: "en", }, }, }, }, }, }, }, } ```