AI Gateway Realtime Quickstart
Start a realtime speech session through AI Gateway.
This quickstart uses openai/gpt-realtime-2. For openai/gpt-live-1, follow the GPT-Live guide instead. GPT-Live requires its own endpoint and session events; changing the model slug in this example does not select the Live protocol.
Paste this prompt into a coding agent with terminal access:
Add a realtime speech session through AI Gateway in the current environment. Use the AI Gateway skill for this task. If it is unavailable, run npx skills add vercel/vercel-plugin --skill ai-gateway, then find and read its SKILL.md before continuing. Reuse the environment's language, framework, package manager, and interface when possible, choose an AI Gateway SDK or API that supports realtime speech, and add only required dependencies. Read AI_GATEWAY_API_KEY from the environment. If it is missing, run npx vercel@latest whoami and pause for login if needed. Determine the team, then run npx vercel@latest --scope <team-slug> ai-gateway api-keys create --name <descriptive-name>. Capture stdout directly into AI_GATEWAY_API_KEY for the request or existing ignored secret storage, and never expose the value. Use openai/gpt-realtime-2, start a session, handle its transcript or audio in the existing interface, run the result, and report the output.
Use Node.js 22.18 or later and a team with available AI Gateway Credits. Export AI_GATEWAY_API_KEY in your current shell. If you need a key, open the Create API Key dialog.
export AI_GATEWAY_API_KEY="your_ai_gateway_api_key"Install the AI Gateway provider and WebSocket client:
pnpm add @ai-sdk/gateway@latest wsCreate realtime.mts:
import { gateway } from '@ai-sdk/gateway';
import WebSocket from 'ws';
const modelId = 'openai/gpt-realtime-2';
const { token, url } = await gateway.experimental_realtime.getToken({
model: modelId,
});
const model = gateway.experimental_realtime(modelId);
const config = model.getWebSocketConfig({ token, url });
const ws = new WebSocket(config.url, config.protocols);
const send = async (event: Parameters<typeof model.serializeClientEvent>[0]) =>
ws.send(JSON.stringify(await model.serializeClientEvent(event)));
ws.on('open', async () => {
await send({
type: 'conversation-item-create',
item: {
type: 'text-message',
role: 'user',
text: 'Say hello in one sentence.',
},
});
await send({ type: 'response-create' });
});
ws.on('message', (data) => {
const parsed = model.parseServerEvent(JSON.parse(data.toString()));
for (const event of Array.isArray(parsed) ? parsed : [parsed]) {
if (event.type === 'audio-transcript-delta') {
process.stdout.write(event.delta);
}
if (event.type === 'response-done') {
console.log();
ws.close();
}
if (event.type === 'error') {
console.error(event.message);
ws.close();
}
}
});Run the script:
node realtime.mts- Build a live browser voice agent with a server-minted token in the Realtime guide
- Configure sessions and limits
- Browse realtime models
Was this helpful?