Model Filtering

AI Gateway Model Filtering

Provider filtering, ordering, and sorting controls which providers serve a request. Model filtering instead constrains routing by a capability of the model itself using has in providerOptions.gateway.

Require model capabilities with `has`

Use the has array to restrict routing to provider models that have all of the given capabilities. Because a capability is a property of the model rather than the credential, this filter applies to both system and BYOK credentials.

The following capabilities are supported:

Capability	Description
`free`	Models with free inference
`implicit-caching`	Models that perform automatic (implicit) prompt caching
`reasoning`	Models that support reasoning
`structured-output`	Models that produce schema-constrained output (JSON schema)
`tool-use`	Models that support tool calling
`vision`	Models that accept image input

AI Gateway sets structured-output on a provider model only after a live probe shows that provider produces schema-constrained output, so the same model can carry the capability on some providers and not others. has: ['structured-output'] routes only to the providers that carry it.

Decision requests don't need structured-output in has. AI Gateway adds the requirement automatically to language-model fallbacks, and adding it to the request-wide has would also filter the native decision model.

These examples use AI SDK 7 and the AI SDK for Python beta. Set AI_GATEWAY_API_KEY before running them. See API format differences for setup, request fields, and response handling.

See the AI SDK capability-filtering reference for SDK configuration and usage.

model-filtering.ts

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'deepseek/deepseek-v4-flash',
  prompt: 'Explain quantum computing in two sentences.',
  providerOptions: {
    gateway: {
      has: ['implicit-caching'],
    },
  },
});
 
console.log(text);

model-filtering_ai.py

import asyncio
import ai
 
async def main():
    model = ai.get_model("deepseek/deepseek-v4-flash")
    messages = [ai.user_message("Explain quantum computing in two sentences.")]
    params = ai.InferenceRequestParams(
        extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}}
    )
    async with ai.stream(model, messages, params=params) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)
    print()
 
asyncio.run(main())

model-filtering-chat.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.chat.completions.create({
  model: 'deepseek/deepseek-v4-flash',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  // AI Gateway extension fields are not included in the upstream SDK types.
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
console.log(response.choices[0]?.message.content);

model-filtering_chat.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.chat.completions.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
print(response.choices[0].message.content)

model-filtering-chat.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/chat/completions \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'

model-filtering-messages.ts

import Anthropic from '@anthropic-ai/sdk';
 
const client = new Anthropic({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh',
});
 
const response = await client.messages.create({
  model: 'deepseek/deepseek-v4-flash',
  messages: [
    {
      role: 'user',
      content: 'Explain quantum computing in two sentences.',
    },
  ],
  max_tokens: 1024,
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
for (const block of response.content) {
  if (block.type === 'text') console.log(block.text);
}

model-filtering_messages.py

import os
from anthropic import Anthropic
 
client = Anthropic(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh",
)
 
response = client.messages.create(
    model="deepseek/deepseek-v4-flash",
    messages=[{"role": "user", "content": "Explain quantum computing in two sentences."}],
    max_tokens=1024,
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
for block in response.content:
    if block.type == "text":
        print(block.text)

model-filtering-messages.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/messages \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "messages": [
    {
      "role": "user",
      "content": "Explain quantum computing in two sentences."
    }
  ],
  "max_tokens": 1024,
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'

model-filtering-responses.ts

import OpenAI from 'openai';
 
const client = new OpenAI({
  apiKey: process.env.AI_GATEWAY_API_KEY,
  baseURL: 'https://ai-gateway.vercel.sh/v1',
});
 
const response = await client.responses.create({
  model: 'deepseek/deepseek-v4-flash',
  input: 'Explain quantum computing in two sentences.',
  ...{
    providerOptions: {
      gateway: {
        has: ['implicit-caching'],
      },
    },
  },
});
 
console.log(response.output_text);

model-filtering_responses.py

import os
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["AI_GATEWAY_API_KEY"],
    base_url="https://ai-gateway.vercel.sh/v1",
)
 
response = client.responses.create(
    model="deepseek/deepseek-v4-flash",
    input="Explain quantum computing in two sentences.",
    extra_body={"providerOptions": {"gateway": {"has": ["implicit-caching"]}}},
)
 
print(response.output_text)

model-filtering-responses.sh

curl --fail-with-body https://ai-gateway.vercel.sh/v1/responses \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "deepseek/deepseek-v4-flash",
  "input": "Explain quantum computing in two sentences.",
  "providerOptions": {
    "gateway": {
      "has": [
        "implicit-caching"
      ]
    }
  }
}'

In this example:

Restriction: Only provider models with the implicit-caching capability are eligible for routing and fallbacks. The filter applies to both system and BYOK credentials.
All capabilities required: When you list more than one capability, a model must have every one to be eligible.
Error on mismatch: If no provider model for the requested model has the capabilities, the request fails. Unsupported values are rejected.

has: ['implicit-caching'] ensures you only route to models that cache automatically. To instead let AI Gateway add cache markers for providers that require explicit caching, see Automatic Caching.

Filter by serving weight format

Weight-format conditions route on the weight format the serving provider reports for the model (for example, fp8 or int4 quantized weights). Use quantization:<value> to require a format, or !quantization:<value> to exclude one:

model-filtering.ts

import { generateText } from 'ai';
 
const { text } = await generateText({
  model: 'deepseek/deepseek-v4-flash',
  prompt: 'Summarize this report',
  providerOptions: {
    gateway: {
      has: ['!quantization:fp8'], // Skip providers serving fp8-quantized weights
    },
  },
});

A negated condition also matches providers with no recorded weight format.

Combining with provider filtering

has composes with the provider-level order, only, and sort options. The model-capability filter and the provider filters are both applied, so the request is routed only to providers that satisfy your provider constraints and whose model has the required capabilities.

Quick reference

Option	Type	Description
`has`	`Array<'free' \| 'implicit-caching' \| 'reasoning' \| 'structured-output' \| 'tool-use' \| 'vision' \| 'quantization:<value>' \| '!quantization:<value>'>`	Restrict routing to models that have all of the given capabilities or satisfy the given weight-format conditions

Set has under providerOptions.gateway in every request format shown above. OpenAI and Anthropic Python clients pass that object through extra_body; TypeScript clients spread the Gateway extension into the request. See API format differences.

Capability filtering in virtual models

A virtual model can require capabilities for all of its traffic. Its has list replaces the request's, and every capability in it must hold.

Last updated October 2, 2026

Cross-link map: AI Gateway Model Filtering (/docs/ai-gateway/models-and-providers/model-filtering)

From the Vercel docs graph (built 2026-10-09T15:41:22.984Z), spanning vercel.com docs + KB, nextjs.org, ai-sdk.dev, and other Vercel documentation sites. Full graph as JSON: https://vercel.com/docs/graph.json

Semantically closest pages

AI Gateway Models and Providers — Choose AI Gateway models and providers. Configure routing, fallbacks, timeouts, prompt caching, reasoning, and web searc
AI Gateway Provider Routing and Fallbacks — Configure provider routing, ordering, and fallback behavior in Vercel AI Gateway.
OpenAI Chat Completions Configuration with AI Gateway — Configure provider options, model fallbacks, BYOK credentials, and prompt caching through AI Gateway.
OpenResponses Configuration with AI Gateway — Configure provider routing, fallbacks, and restrictions using the OpenResponses API through AI Gateway.
AI Gateway Model Fallbacks — Configure AI Gateway model fallbacks to try backup models when the primary model is unavailable. Set fallback order and

Prerequisites

AI Gateway — Vercel MCP tools for ai gateway.

This page links to (6)

AI Gateway — Learn how to use the AI Gateway provider with the AI SDK.
AI Gateway Authentication and BYOK — Authenticate AI Gateway requests with API keys or OIDC tokens, and configure bring your own key \(BYOK\) credentials for
Decision — Ask typed questions about shared state and get back structured choices, scores, and boolean probabilities through Vercel
AI Gateway Automatic Prompt Caching — Enable automatic prompt caching and provider-side cache affinity with AI Gateway to reduce costs and latency.
AI Gateway Provider Filtering, Ordering, and Sorting — Control AI Gateway provider routing with order, only, and sort. Set preferences, restrict providers, and rank them by co
AI Gateway SDKs and APIs — Connect to AI Gateway with the AI SDK, Python, REST, or compatible OpenAI, Anthropic Messages, OpenResponses, and Cohere

Pages that link here (1)

By site: vercel-docs (1)

AI Gateway Provider Filtering, Ordering, and Sorting — Control AI Gateway provider routing with order, only, and sort. Set preferences, restrict providers, and rank them by co

Filtering, Ordering & Sorting

Batch Processing

Was this helpful?

AI Gateway Model Filtering

Require model capabilities with has

Filter by serving weight format

Combining with provider filtering

Quick reference

Capability filtering in virtual models

Cross-link map: AI Gateway Model Filtering (/docs/ai-gateway/models-and-providers/model-filtering)

Semantically closest pages

Prerequisites

This page links to (6)

Pages that link here (1)

Cross-link map: AI Gateway Model Filtering (/docs/ai-gateway/models-and-providers/model-filtering)

Semantically closest pages

Prerequisites

This page links to (6)

Pages that link here (1)

Require model capabilities with `has`