Python

Python with AI Gateway: OpenAI and Anthropic SDKs

To get started with Python and AI Gateway, you can either call the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API directly, or use the official OpenAI and Anthropic Python SDKs, which are covered below.

You can also use the AI SDK for Python. It's currently in public beta.

Installation

Install your preferred SDK:

pip install openai

pip install anthropic

pip install openai

Quick start

quickstart.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
response = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {'role': 'user', 'content': 'Explain quantum computing in one paragraph.'}
    ]
)
 
print(response.choices[0].message.content)

quickstart.py

import os
import anthropic
 
client = anthropic.Anthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
message = client.messages.create(
    model='anthropic/claude-opus-5',
    max_tokens=1024,
    messages=[
        {'role': 'user', 'content': 'Explain quantum computing in one paragraph.'}
    ]
)
 
for block in message.content:
    if block.type == "text":
        print(block.text)

quickstart.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
response = client.responses.create(
    model='anthropic/claude-opus-5',
    input='Explain quantum computing in one paragraph.',
)
 
print(response.output_text)

Authentication

All SDKs support the same authentication methods. Use an API key for local development or OIDC tokens for Vercel deployments.

auth.py

import os
 
# Option 1: API key (recommended for local development)
api_key = os.getenv('AI_GATEWAY_API_KEY')
 
# Option 2: OIDC token (automatic on Vercel deployments)
api_key = os.getenv('VERCEL_OIDC_TOKEN')
 
# Fallback pattern for code that runs both locally and on Vercel
api_key = os.getenv('AI_GATEWAY_API_KEY') or os.getenv('VERCEL_OIDC_TOKEN')

Streaming

Stream responses for real-time output in chat applications or long-running generations.

streaming.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
stream = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {'role': 'user', 'content': 'Write a short story about a robot.'}
    ],
    stream=True
)
 
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end='', flush=True)

streaming.py

import os
import anthropic
 
client = anthropic.Anthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
with client.messages.stream(
    model='anthropic/claude-opus-5',
    max_tokens=1024,
    messages=[
        {'role': 'user', 'content': 'Write a short story about a robot.'}
    ]
) as stream:
    for text in stream.text_stream:
        print(text, end='', flush=True)

streaming.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
stream = client.responses.create(
    model='anthropic/claude-opus-5',
    input='Write a short story about a robot.',
    stream=True,
)
 
for event in stream:
    if event.type == 'response.output_text.delta':
        print(event.delta, end='', flush=True)

Async support

Both the OpenAI and Anthropic SDKs provide async clients for use with asyncio.

async_client.py

import os
import asyncio
from openai import AsyncOpenAI
 
client = AsyncOpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
async def main():
    response = await client.chat.completions.create(
        model='anthropic/claude-opus-5',
        messages=[
            {'role': 'user', 'content': 'Hello!'}
        ]
    )
    print(response.choices[0].message.content)
 
asyncio.run(main())

async_client.py

import os
import asyncio
import anthropic
 
client = anthropic.AsyncAnthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
async def main():
    message = await client.messages.create(
        model='anthropic/claude-opus-5',
        max_tokens=1024,
        messages=[
            {'role': 'user', 'content': 'Hello!'}
        ]
    )
    for block in message.content:
        if block.type == "text":
            print(block.text)
 
asyncio.run(main())

async_client.py

import os
import asyncio
from openai import AsyncOpenAI
 
client = AsyncOpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
async def main():
    response = await client.responses.create(
        model='anthropic/claude-opus-5',
        input='Hello!',
    )
    print(response.output_text)
 
asyncio.run(main())

Tool calling

Enable models to call functions you define. This example shows a weather tool that the model can invoke.

tools.py

import os
import json
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
tools = [{
    'type': 'function',
    'function': {
        'name': 'get_weather',
        'description': 'Get the current weather for a location',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {
                    'type': 'string',
                    'description': 'City name, e.g. San Francisco'
                }
            },
            'required': ['location']
        }
    }
}]
 
response = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {'role': 'user', 'content': "What's the weather in Tokyo?"}
    ],
    tools=tools
)
 
# Check if the model wants to call a tool
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    print(f"Model wants to call: {tool_call.function.name}")
    print(f"With arguments: {args}")

tools.py

import os
import anthropic
 
client = anthropic.Anthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
tools = [{
    'name': 'get_weather',
    'description': 'Get the current weather for a location',
    'input_schema': {
        'type': 'object',
        'properties': {
            'location': {
                'type': 'string',
                'description': 'City name, e.g. San Francisco'
            }
        },
        'required': ['location']
    }
}]
 
message = client.messages.create(
    model='anthropic/claude-opus-5',
    max_tokens=1024,
    messages=[
        {'role': 'user', 'content': "What's the weather in Tokyo?"}
    ],
    tools=tools
)
 
# Check if the model wants to call a tool
for block in message.content:
    if block.type == 'tool_use':
        print(f"Model wants to call: {block.name}")
        print(f"With arguments: {block.input}")

tools.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
response = client.responses.create(
    model='openai/gpt-6-astra',
    input='What is the weather in Tokyo?',
    tools=[
        {
            'type': 'function',
            'name': 'get_weather',
            'description': 'Get the current weather for a location',
            'parameters': {
                'type': 'object',
                'properties': {
                    'location': {'type': 'string'},
                },
                'required': ['location'],
            },
        },
    ],
)
 
for item in response.output:
    if item.type == 'function_call':
        print(f'Call: {item.name}({item.arguments})')

See Chat Completions tool calls, OpenAI Responses API tool calling, or Anthropic Messages tool calls for more examples.

Structured outputs

Generate responses that conform to a JSON schema for reliable parsing.

structured.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
response = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {'role': 'user', 'content': 'Extract: John is 30 years old and lives in NYC'}
    ],
    response_format={
        'type': 'json_schema',
        'json_schema': {
            'name': 'person',
            'schema': {
                'type': 'object',
                'properties': {
                    'name': {'type': 'string'},
                    'age': {'type': 'integer'},
                    'city': {'type': 'string'}
                },
                'required': ['name', 'age', 'city']
            }
        }
    }
)
 
import json
data = json.loads(response.choices[0].message.content)
print(data)  # {'name': 'John', 'age': 30, 'city': 'NYC'}

See structured outputs for more details.

Images and file input

Each surface takes attachments as an array of content parts in place of a plain string, but the part names differ. Read the file, base64-encode it, and use the shape for your surface:

vision.py

import base64
import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
with open('diagram.png', 'rb') as f:
    image_base64 = base64.b64encode(f.read()).decode()
 
completion = client.chat.completions.create(
    model='anthropic/claude-opus-5',
    messages=[
        {
            'role': 'user',
            'content': [
                {'type': 'text', 'text': 'Describe this image in one sentence.'},
                {
                    'type': 'image_url',
                    'image_url': {'url': f'data:image/png;base64,{image_base64}'}
                }
            ]
        }
    ]
)
 
print(completion.choices[0].message.content)

vision.py

import base64
import os
import anthropic
 
client = anthropic.Anthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
with open('diagram.png', 'rb') as f:
    image_base64 = base64.b64encode(f.read()).decode()
 
message = client.messages.create(
    model='anthropic/claude-opus-5',
    max_tokens=1024,
    messages=[
        {
            'role': 'user',
            'content': [
                {'type': 'text', 'text': 'Describe this image in one sentence.'},
                {
                    'type': 'image',
                    'source': {
                        'type': 'base64',
                        'media_type': 'image/png',
                        'data': image_base64
                    }
                }
            ]
        }
    ]
)
 
for block in message.content:
    if block.type == "text":
        print(block.text)

vision.py

import base64
import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1'
)
 
with open('diagram.png', 'rb') as f:
    image_base64 = base64.b64encode(f.read()).decode()
 
response = client.responses.create(
    model='anthropic/claude-opus-5',
    input=[
        {
            'role': 'user',
            'content': [
                {'type': 'input_text', 'text': 'Describe this image in one sentence.'},
                {
                    'type': 'input_image',
                    'image_url': f'data:image/png;base64,{image_base64}'
                }
            ]
        }
    ]
)
 
print(response.output_text)

PDFs work the same way with a different part type. On Anthropic Messages that is a document part:

pdf.py

import base64
import os
import anthropic
 
client = anthropic.Anthropic(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh'
)
 
with open('report.pdf', 'rb') as f:
    pdf_base64 = base64.b64encode(f.read()).decode()
 
message = client.messages.create(
    model='anthropic/claude-opus-5',
    max_tokens=1024,
    messages=[
        {
            'role': 'user',
            'content': [
                {'type': 'text', 'text': 'What total does this document state?'},
                {
                    'type': 'document',
                    'source': {
                        'type': 'base64',
                        'media_type': 'application/pdf',
                        'data': pdf_base64
                    }
                }
            ]
        }
    ]
)

For the per-surface reference, see Chat Completions file attachments, Responses file attachments, and Anthropic file attachments.

Reasoning

Reasoning models work through a problem before answering. On the Responses API, pass reasoning with an effort level:

reasoning.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
response = client.responses.create(
    model='openai/gpt-6-astra',
    input='Explain the Monty Hall problem step by step.',
    reasoning={'effort': 'medium'},
    max_output_tokens=2048,
)
 
print(response.output_text)

The Chat Completions API takes the same idea through extra_body, since the OpenAI Python client does not have a typed field for it:

reasoning-chat-completions.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.getenv('AI_GATEWAY_API_KEY'),
    base_url='https://ai-gateway.vercel.sh/v1',
)
 
completion = client.chat.completions.create(
    model='openai/gpt-6-astra',
    messages=[{'role': 'user', 'content': 'Explain the Monty Hall problem.'}],
    extra_body={'reasoning': {'effort': 'medium'}},
)
 
print(completion.choices[0].message.reasoning)
print(completion.choices[0].message.content)

Each surface names and shapes this differently, and Anthropic models have their own constraints. See Responses reasoning, Chat Completions reasoning, and Anthropic extended thinking.

Framework integrations

Python frameworks with dedicated AI Gateway support:

Framework	Integration
Pydantic AI	Native `VercelProvider` for type-safe agents
LlamaIndex	`llama-index-llms-vercel-ai-gateway` package
LiteLLM	Use `vercel_ai_gateway/` model prefix
LangChain	Configure via Chat Completions endpoint

See Framework Integrations for the complete list and setup guides.

API reference

For complete API documentation, see:

OpenAI Chat Completions API: Chat completions, embeddings, streaming, tool calls, structured outputs, image inputs, and provider routing
OpenAI Responses API: Streaming, tool calling, structured output, and reasoning
Anthropic Messages API: Streaming, tool calls, extended thinking, web search, and file attachments

Last updated September 8, 2026

Cross-link map: Python with AI Gateway: OpenAI and Anthropic SDKs (/docs/ai-gateway/sdks-and-apis/python)

From the Vercel docs graph (built 2026-10-09T15:41:22.984Z), spanning vercel.com docs + KB, nextjs.org, ai-sdk.dev, and other Vercel documentation sites. Full graph as JSON: https://vercel.com/docs/graph.json

Semantically closest pages

AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere
AI SDK with AI Gateway — Build AI-powered TypeScript applications using the AI SDK with AI Gateway for unified access to 200+ models.
OpenAI Chat Completions API with AI Gateway — Use OpenAI SDKs with the AI Gateway Chat Completions API. Configure the base URL and authentication for chat, streaming,
OpenAI Responses API with AI Gateway — Use the OpenAI Responses API with AI Gateway to generate text, call tools, stream tokens, and more across any supported
OpenAI Chat Completions Images and PDFs with AI Gateway — Send images and PDF documents to a model using the OpenAI Chat Completions API through AI Gateway.

Prerequisites

AI Gateway — Vercel MCP tools for ai gateway.

This page links to (20)

AI Gateway API Keys — Create, view, and delete AI Gateway API keys, and set each key's budget and spend attribution, from the dashboard, CLI,
AI Gateway OIDC Authentication — Authenticate AI Gateway requests from Vercel deployments with OIDC tokens. Configure the AI SDK or send bearer tokens di
AI Gateway Framework Integrations — Connect LangChain, LiteLLM, LlamaIndex, Mastra, Pydantic AI, TanStack AI, and other frameworks to Vercel AI Gateway with
LangChain with AI Gateway — Learn how to integrate Vercel AI Gateway with LangChain to access multiple AI models through a unified interface.
LiteLLM with AI Gateway — Learn how to integrate Vercel AI Gateway with LiteLLM to access multiple AI models through a unified interface.
LlamaIndex with AI Gateway — Learn how to integrate Vercel AI Gateway with LlamaIndex to access multiple AI models through a unified interface.
Pydantic AI with AI Gateway — Learn how to integrate Vercel AI Gateway with Pydantic AI to access multiple AI models through a unified interface.
Anthropic Messages API with AI Gateway — Use the Anthropic Messages API with AI Gateway. Configure authentication and send requests with streaming, tools, images
Anthropic Messages Images and PDFs with AI Gateway — Send images and PDF documents as part of your Anthropic API message requests through AI Gateway.
Anthropic Messages Extended Thinking with AI Gateway — Configure how much Claude thinks before answering, using the Anthropic Messages API thinking parameter through AI Gatewa
Anthropic Messages Tool Calling with AI Gateway — Use function calling with the Anthropic Messages API to allow models to call tools and functions through AI Gateway.
OpenAI Chat Completions API with AI Gateway — Use OpenAI SDKs with the AI Gateway Chat Completions API. Configure the base URL and authentication for chat, streaming,
OpenAI Chat Completions Images and PDFs with AI Gateway — Send images and PDF documents to a model using the OpenAI Chat Completions API through AI Gateway.
OpenAI Chat Completions Reasoning with AI Gateway — Control how much a model thinks before answering with the OpenAI Chat Completions API through AI Gateway.
OpenAI Chat Completions Structured Outputs with AI Gateway — Generate structured JSON responses that conform to a specific schema using the Chat Completions API through AI Gateway.
OpenAI Chat Completions Tool Calling with AI Gateway — Use function calling with the Chat Completions API to enable models to call tools and functions through AI Gateway.
OpenAI Responses API with AI Gateway — Use the OpenAI Responses API with AI Gateway to generate text, call tools, stream tokens, and more across any supported
OpenAI Responses Images and PDFs with AI Gateway — Send images and PDF documents for analysis using the OpenAI Responses API through AI Gateway.
OpenAI Responses Reasoning with AI Gateway — Control how much a model thinks before answering with the OpenAI Responses API through AI Gateway.
OpenAI Responses Tool Calling with AI Gateway — Define tools the model can call with the OpenAI Responses API through AI Gateway.

Pages that link here (1)

By site: vercel-docs (1)

AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere

OpenResponses API

REST API

Was this helpful?