--- title: Transcription description: Learn how to transcribe audio with the AI SDK. --- # Transcription The AI SDK provides the [`transcribe`](/docs/reference/ai-sdk-core/transcribe) function to transcribe audio using a transcription model. ```ts import { transcribe } from 'ai'; import { openai } from '@ai-sdk/openai'; import { readFile } from 'fs/promises'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: await readFile('audio.mp3'), }); ``` The `audio` property can be a `Uint8Array`, `ArrayBuffer`, `Buffer`, `string` (base64 encoded audio data), or a `URL`. To access the generated transcript: ```ts const text = transcript.text; // transcript text e.g. "Hello, world!" const segments = transcript.segments; // array of segments with start and end times, if available const language = transcript.language; // language of the transcript e.g. "en", if available const durationInSeconds = transcript.durationInSeconds; // duration of the transcript in seconds, if available ``` ## Streaming Transcription Streaming transcription is an experimental feature. Use `experimental_streamTranscribe` when you have live raw audio and need transcript updates before the full audio stream is complete. The function uses transcription models with streaming support; provider options configure provider-specific behavior, but the streaming operation is selected by the function itself. ```ts import { openai } from '@ai-sdk/openai'; import { experimental_streamTranscribe as streamTranscribe } from 'ai'; const result = streamTranscribe({ model: openai.transcription('gpt-realtime-whisper'), audio: audioStream, // ReadableStream inputAudioFormat: { type: 'audio/pcm', rate: 24000 }, providerOptions: { openai: { language: 'en', streaming: { delay: 'low', }, }, }, }); for await (const part of result.fullStream) { if (part.type === 'transcript-delta') { process.stdout.write(part.delta); } if (part.type === 'transcript-partial') { console.log('partial:', part.text); } if (part.type === 'transcript-final') { console.log('final:', part.text); } } console.log(await result.text); ``` To access the final transcript metadata: ```ts const text = await result.text; // final transcript text const segments = await result.segments; // final segments with timing, if available const language = await result.language; // language of the transcript, if available const durationInSeconds = await result.durationInSeconds; // duration in seconds, if available ``` The `audio` stream must contain raw audio chunks. `Uint8Array` chunks are raw bytes; `string` chunks are base64-encoded raw bytes. Always set `inputAudioFormat` to match the chunks you send. Streaming transcription requires a provider model instance (e.g. `openai.transcription('gpt-realtime-whisper')`). String model IDs resolve through the global provider (AI Gateway by default), which does not support streaming transcription yet. OpenAI streaming transcription uses `openai.transcription('gpt-realtime-whisper')`. xAI uses the same `xai.transcription()` model for request/response and streaming transcription; `experimental_streamTranscribe` uses xAI's WebSocket STT transport under the hood. ```ts import { xai } from '@ai-sdk/xai'; import { experimental_streamTranscribe as streamTranscribe } from 'ai'; const result = streamTranscribe({ model: xai.transcription(), audio: audioStream, inputAudioFormat: { type: 'audio/pcm', rate: 16000 }, providerOptions: { xai: { language: 'en', keyterm: ['AI SDK', 'Grok'], streaming: { interimResults: true, endpointing: 500, }, }, }, }); ``` Some providers require WebSocket headers for direct streaming STT. In those runtimes, pass a provider-specific `webSocket` implementation when creating the provider. ## Settings ### Provider-Specific settings Transcription models often have provider or model-specific settings which you can set using the `providerOptions` parameter. ```ts highlight="8-12" import { transcribe } from 'ai'; import { openai } from '@ai-sdk/openai'; import { readFile } from 'fs/promises'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: await readFile('audio.mp3'), providerOptions: { openai: { timestampGranularities: ['word'], }, }, }); ``` ### Download Size Limits When `audio` is a URL, the SDK downloads the file with a default **2 GiB** size limit. You can customize this using `createDownload`: ```ts highlight="1,8" import { transcribe, createDownload } from 'ai'; import { openai } from '@ai-sdk/openai'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: new URL('https://example.com/audio.mp3'), download: createDownload({ maxBytes: 50 * 1024 * 1024 }), // 50 MB limit }); ``` You can also provide a fully custom download function: ```ts highlight="6-12" import { transcribe } from 'ai'; import { openai } from '@ai-sdk/openai'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: new URL('https://example.com/audio.mp3'), download: async ({ url }) => { const res = await myAuthenticatedFetch(url); return { data: new Uint8Array(await res.arrayBuffer()), mediaType: res.headers.get('content-type') ?? undefined, }; }, }); ``` If a download exceeds the size limit, a `DownloadError` is thrown: ```ts import { transcribe, DownloadError } from 'ai'; import { openai } from '@ai-sdk/openai'; try { await transcribe({ model: openai.transcription('whisper-1'), audio: new URL('https://example.com/audio.mp3'), }); } catch (error) { if (DownloadError.isInstance(error)) { console.log('Download failed:', error.message); } } ``` ### Abort Signals and Timeouts `transcribe` accepts an optional `abortSignal` parameter of type [`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal) that you can use to abort the transcription process or set a timeout. This is particularly useful when combined with URL downloads to prevent long-running requests: ```ts highlight="8" import { openai } from '@ai-sdk/openai'; import { transcribe } from 'ai'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: new URL('https://example.com/audio.mp3'), abortSignal: AbortSignal.timeout(5000), // Abort after 5 seconds }); ``` ### Custom Headers `transcribe` accepts an optional `headers` parameter of type `Record` that you can use to add custom headers to the transcription request. ```ts highlight="8" import { openai } from '@ai-sdk/openai'; import { transcribe } from 'ai'; import { readFile } from 'fs/promises'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: await readFile('audio.mp3'), headers: { 'X-Custom-Header': 'custom-value' }, }); ``` ### Warnings Warnings (e.g. unsupported parameters) are available on the `warnings` property. ```ts import { openai } from '@ai-sdk/openai'; import { transcribe } from 'ai'; import { readFile } from 'fs/promises'; const transcript = await transcribe({ model: openai.transcription('whisper-1'), audio: await readFile('audio.mp3'), }); const warnings = transcript.warnings; ``` ### Error Handling When `transcribe` cannot generate a valid transcript, it throws a [`AI_NoTranscriptGeneratedError`](/docs/reference/ai-sdk-errors/ai-no-transcript-generated-error). This error can arise for any of the following reasons: - The model failed to generate a response - The model generated a response that could not be parsed The error preserves the following information to help you log the issue: - `responses`: Metadata about the transcription model responses, including timestamp, model, and headers. - `cause`: The cause of the error. You can use this for more detailed error handling. ```ts import { transcribe, NoTranscriptGeneratedError } from 'ai'; import { openai } from '@ai-sdk/openai'; import { readFile } from 'fs/promises'; try { await transcribe({ model: openai.transcription('whisper-1'), audio: await readFile('audio.mp3'), }); } catch (error) { if (NoTranscriptGeneratedError.isInstance(error)) { console.log('NoTranscriptGeneratedError'); console.log('Cause:', error.cause); console.log('Responses:', error.responses); } } ``` ## Transcription Models | Provider | Model | | ------------------------------------------------------------------------------- | ------------------------ | | [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `whisper-1` | | [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-transcribe` | | [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-mini-transcribe` | | [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1` | | [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1_experimental` | | [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3-turbo` | | [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3` | | [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `whisper-1` | | [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `gpt-4o-transcribe` | | [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `gpt-4o-mini-transcribe` | | [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `machine` | | [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `low_cost` | | [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `fusion` | | [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `base` (+ variants) | | [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `enhanced` (+ variants) | | [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova` (+ variants) | | [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova-2` (+ variants) | | [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova-3` (+ variants) | | [Gladia](/providers/ai-sdk-providers/gladia#transcription-models) | `default` | | [AssemblyAI](/providers/ai-sdk-providers/assemblyai#transcription-models) | `universal-3-5-pro` | | [AssemblyAI](/providers/ai-sdk-providers/assemblyai#transcription-models) | `universal-3-pro` | | [Fal](/providers/ai-sdk-providers/fal#transcription-models) | `whisper` | | [Fal](/providers/ai-sdk-providers/fal#transcription-models) | `wizper` | | [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `chirp_2` | | [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `chirp_3` | | [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `telephony` | | [xAI](/providers/ai-sdk-providers/xai#transcription-models) | `default` | | [Cartesia](/providers/ai-sdk-providers/cartesia#transcription-models) | `ink-whisper` | Above are a small subset of the transcription models supported by the AI SDK providers. For more, see the respective provider documentation.