AI Gateway Regional Inference

AI Gateway lets you pick which region to route each request to your model's provider, where available. This may help with your data residency, compliance, and latency requirements. You choose the region per request. AI Gateway routes your data to the provider in a data center in that region, and to the extent the provider stores data, it's configured to store it there. If AI Gateway can't honor the chosen region, the request fails instead of silently running somewhere else.

Set inferenceRegion on a request and the provider runs inference in that region. To the extent it stores anything at rest, it stores it in the same region. One setting covers both:

What varies by provider is whether anything is stored at rest at all. Many providers store nothing for inference, and zero data retention has the provider delete prompts and responses after the request (see where your data goes).

Set providerOptions.gateway.inferenceRegion on the request, and AI Gateway routes inference to that region:

• The default (no inferenceRegion) is global, which means AI Gateway picks whichever region can serve the model, based on availability and latency. It can resolve to a different region on each request, and that region may be outside your users' jurisdiction. Residency is opt-in: pin a region if where inference runs matters to you.
• zone requires geoRegion (us or eu). If you omit it, the request fails.
• You set the region in the request body or config only. There's no HTTP header for it.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'openai/gpt-6-astra',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      inferenceRegion: {
        scope: 'zone',
        geoRegion: 'us',
      },
    },
  },
});
 
console.log(text);

import asyncio
import ai
 
async def main():
    model = ai.get_model("openai/gpt-6-astra")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'openai/gpt-6-astra',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="openai/gpt-6-astra",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
print(response.choices[0].message.content)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "openai/gpt-6-astra",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'openai/gpt-6-astra',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="openai/gpt-6-astra",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "openai/gpt-6-astra",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'openai/gpt-6-astra',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        inferenceRegion: {
          scope: 'zone',
          geoRegion: 'us',
        },
      },
    },
  },
});
 
console.log(response.output_text);

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="openai/gpt-6-astra",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"inferenceRegion": {"scope": "zone", "geoRegion": "us"}}}},
)
 
print(response.output_text)

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "openai/gpt-6-astra",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "inferenceRegion": {
        "scope": "zone",
        "geoRegion": "us"
      }
    }
  }
}'

Most callers set one region and stop. If more than one provider can serve a single request, you can set a default plus per-provider overrides. The top-level value applies to every provider unless you list that provider under providers:

{
  "inferenceRegion": {
    "scope": "zone",
    "geoRegion": "us",           // default for all providers
    "providers": {
      "bedrock": { "scope": "zone", "geoRegion": "eu" },  // Bedrock only: EU
      "anthropic": null                                   // Anthropic: no region pinned
    }
  }
}

This reads as: AI Gateway routes your data to every provider in the US, except it routes Bedrock traffic to the EU and leaves Anthropic unconstrained. AI Gateway ignores unknown provider keys.

AI Gateway routes your data to the provider in the region you pin. To the extent that provider stores your request data at rest, it's configured to store it in that region. Each provider's own terms set what it keeps and for how long, so check the provider's documentation for the specifics.

Abuse and safety monitoring is separate. Regardless of your pinned region or retention settings, a provider handles and retains requests it flags for abuse or safety review under its own policies, which can fall outside your region. Vercel doesn't control that or guarantee it beyond what the provider documents, so check the provider's terms.

To limit retention, add zero data retention (ZDR), where available. With ZDR, the provider deletes your prompts and responses after the request rather than retaining them. Some provider models do not have ZDR. For example, Anthropic's fable-5 is ineligible for ZDR on every provider, so a request with ZDR configured fails rather than route to it.

Not every provider serves every region, so coverage varies by provider and model (see Current limits). Don't assume a request ran where you asked; confirm the resolved region from the response (see Confirming where a request ran).

Pinning a region can raise what a request costs. Providers often price regional inference above their default routing, and one region can cost more than another for the same model. The provider sets that price.

AI Gateway passes the provider's regional price straight through. Choosing a region adds no AI Gateway markup on top of it: you pay the provider's rate for the region you picked.

To see what a model charges per region, open it from the model list and hover the price in the Providers table. The Price by region card shows each region's rate as a percentage difference from the global price, for example EU +10%. A region that costs the same as global reads Same as global.

The https://ai-gateway.vercel.sh/v1/models endpoint returns the same rates as absolute per-token numbers, under pricing.regional:

{
  "pricing": {
    "input": "0.000005",
    "regional": {
      "eu": { "input": "0.0000055" },
      "us": { "input": "0.0000055" }
    }
  }
}

To see what a specific request cost, check the cost reported in the response metadata.

You only take on a region's rate when you set inferenceRegion. Leave it unset (the global default) to route at the provider's standard rate.

Bringing your own key doesn't opt you out of residency. When you use BYOK credentials, AI Gateway applies inferenceRegion the same way it does with system credentials:

• AI Gateway routes your key's requests to in-region endpoints. Your key material is untouched; AI Gateway regionalizes how it's used, calling the provider's in-region endpoint with your key.
• An explicit region overrides a region saved on the credential. If your Vertex credential is saved with a location and the request asks for a different region, the request's region wins. If you don't set a region, the credential's saved location applies where it has one.
• Failure behavior is the same. If no provider can honor the region with your credentials, the request fails with HTTP 400. AI Gateway doesn't fall back to an out-of-region endpoint, even when the credentials are your own.
• Verification is the same. Responses report the resolved region for BYOK requests too, so you can confirm residency from the response metadata.
• Zero data retention skips BYOK keys by default. If you pair a pinned region with ZDR, AI Gateway routes around your own credentials unless you mark a key as ZDR-compliant, since those keys run under your provider agreements rather than Vercel's. See ZDR and BYOK.

Set inferenceRegion and the provider processes your prompts and completions, and stores anything it keeps at rest, only in your chosen region. If it can't, the request fails instead of running elsewhere.

To limit retention, add zero data retention and disallow prompt training, so the provider deletes your data after the request and doesn't train on it. Whether a provider stores anything by default is provider-dependent (see where your data goes).

Region coverage varies by model. To find models that support a region, filter the model list:

• Models available in the US
• Models available in the EU

The /v1/models endpoint returns the same coverage as a regions array per model (for example, ["eu", "us"]); a model with no regions field doesn't support regional routing.

This pins where the provider processes and stores your data. It doesn't pin the network path your request takes to reach the provider.

• If you request no region, AI Gateway routes your data globally. This isn't an error.
• If AI Gateway can't honor the region, the request fails with HTTP 400 (invalid_request_error). AI Gateway doesn't silently fall back to another region.
• There's no automatic downgrade from a preferred region to a broader one.

When you pin a region, every response reports where AI Gateway actually ran inference, so you can verify residency instead of assuming it. The resolved region is in the gateway routing metadata, on the provider attempt that served the request.

With the AI SDK, read it from providerMetadata.gateway. On the OpenAI-compatible endpoints, the same object sits on each choice's message as provider_metadata.gateway:

{
  "gateway": {
    "routing": {
      "finalProvider": "openai",
      "modelAttempts": [
        {
          "providerAttempts": [
            {
              "provider": "openai",
              "inferenceEndpoint": {
                "slug": "geo-us",
                "scope": "zone",
                "geoRegion": "us"
              }
            }
          ]
        }
      ]
    }
  }
}

Read inferenceEndpoint.geoRegion (here us) and compare it against the region you asked for; finalProvider names the provider that served the request. When you don't pin a region, inferenceEndpoint is null, matching the global default, which doesn't pin a region.

If GDPR is the driver for pinning a region, consider your compliance needs: where personal data is processed is one obligation among many. Regional routing answers the location question; it doesn't by itself make a workload GDPR-compliant.

• The regions available today are us and eu, and not every model supports both. Check a model's regions in the model list or from https://ai-gateway.vercel.sh/v1/models.
• Region selection is opt-in; the default is global.
• There's no cross-region fallback: a region that AI Gateway can't honor fails the request.

Pin an inference region on a virtual model to route all of its system-credential traffic to that region. The virtual model's region wins over the request's.