Llama 3.1 8B Instruct
Llama 3.1 8B Instruct is a multilingual, instruction-tuned model with a context window of 128K tokens and tool-use capability. It suits cost-effective production deployments that need multilingual coverage and trained tool use. Your use is subject to Meta's Terms & Privacy Policies.
View API reference- Input and output price
- Prices from: Input $0.02, Output $0.05, Per 1M tokens
- 24h uptime
- Loading AI Gateway uptime
import { streamText } from 'ai'
const result = streamText({ model: 'meta/llama-3.1-8b', prompt: 'Why is the sky blue?'})Copy link to headingPlayground
Try out Llama 3.1 8B Instruct by Meta. Usage is billed to your team at API rates. Free users (those who haven't made a payment) get $5 of credits every 30 days.
Llama 3.1 8B Instruct
Llama 3.1 8B Instruct
Copy link to headingProviders
Route requests across multiple providers. Copy a provider slug to set your preference. Visit the docs for more info. Using a provider means you agree to their terms, listed under Legal.
Copy link to headingUptime24 hours
Direct request success rate on AI Gateway and per-provider. Visit the docs for more info.
Copy link to headingThroughput24 hours
P50 throughput on live AI Gateway traffic, in tokens per second (TPS). Visit the docs for more info.
Copy link to headingLatency24 hours
P50 time to first token (TTFT) on live AI Gateway traffic, in milliseconds. View the docs for more info.
Getting started
Call Llama 3.1 8B Instruct through AI Gateway with the AI SDK generateText and streamText functions, or through the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs by changing the base URL. AI Gateway authenticates the request and routes it to an available provider.
Install the AI SDK (pnpm add ai dotenv), create an API key from the API Keys page, and set it as AI_GATEWAY_API_KEY in your environment. Full setup is covered in the text generation quickstart.
import { generateText } from 'ai';import 'dotenv/config';
async function main() { const result = await generateText({ model: 'meta/llama-3.1-8b', prompt: 'Why is the sky blue?', });
console.log(result.text);}
main().catch(console.error);Top-level parameters
The same Llama 3.1 8B Instruct request in each API format AI Gateway supports.
import { generateText } from 'ai';import 'dotenv/config';
async function main() { const result = await generateText({ model: 'meta/llama-3.1-8b', system: 'You are a concise technical assistant.', prompt: 'Summarize the tradeoffs between static generation and SSR.', maxOutputTokens: 1024, temperature: 0.5, });
console.log(result.text);}
main().catch(console.error);Standard parameters like prompt, messages, temperature, and tools work as documented in the AI SDK docs. These are the parameters with model-specific behavior.
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | Model ID in the form creator/model, e.g. meta/llama-3.1-8b. AI Gateway routes the request to an available provider. |
maxOutputTokens | number | No | Hard cap on generated tokens. Llama 3.1 8B Instruct supports up to 16,384 output tokens. |
providerOptions | Record<string, JSONValue> | No | AI Gateway routing options under gateway, plus any provider-native options under the provider’s own namespace — see the table below. |
Input limits
| Input | Formats | Sources | Max count | Max size | Limits |
|---|---|---|---|---|---|
| Text | — | — | — | — | Prompt and response share the 128K-token context window |
Provider options
Set AI Gateway routing options under providerOptions.gateway. For provider-specific options, pass them under the provider’s namespace as documented by the AI SDK.
Learn more in the AI SDK provider docs.
import { generateText } from 'ai';import 'dotenv/config';
async function main() { const result = await generateText({ model: 'meta/llama-3.1-8b', prompt: 'Why is the sky blue?', providerOptions: { gateway: { only: ['bedrock', 'novita'], }, }, });
console.log(result.text);}
main().catch(console.error);These AI Gateway routing options apply to every model. Provider-specific options pass through under the provider’s own namespace (for example providerOptions.anthropic) exactly as documented by the AI SDK.
| Parameter | Type | Required | Description |
|---|---|---|---|
providerOptions.gateway.only | string[] | No | Restrict routing to these provider slugs. Requests fail over only within the listed providers. |
providerOptions.gateway.order | string[] | No | Preferred provider order. Listed providers are tried first; unlisted providers remain available as fallbacks. |
providerOptions.gateway.sort | 'cost' | 'ttft' | 'tps' | No | Rank candidate providers by price, time to first token, or tokens per second instead of the default routing order. |
providerOptions.gateway.zeroDataRetention | boolean | No | Route only to providers with a zero-data-retention policy for this model. |
Routing across providers
AI Gateway serves the same model through multiple providers and fails over automatically. order expresses a preference while keeping every provider eligible; only is a hard allowlist — if none of the listed providers are available the request fails instead of falling back.
Options under a provider's own namespace (for example providerOptions.anthropic) are forwarded to that provider with the request. Providers ignore option namespaces that don't apply to them, so it is safe to set provider options alongside gateway routing options.
Tool calling
Expose tools the model can call. Define each tool’s inputs with a Zod schema.
import { generateText, tool } from 'ai';import { z } from 'zod';import 'dotenv/config';
async function main() { const result = await generateText({ model: 'meta/llama-3.1-8b', prompt: 'What is the weather in San Francisco?', tools: { getWeather: tool({ description: 'Get the current weather for a location', inputSchema: z.object({ location: z.string() }), execute: async ({ location }) => ({ location, temperatureC: 18 }), }), }, });
console.log(result.text);}
main().catch(console.error);Copy link to headingMore models by Meta
| Model |
|---|
| — | $0.01/img | — | | 08/26/2026 | ||||||||||
| 131K | 0.2 s | 100 tps | $0.35/M | $1.50/M | Read$0.04/M | — | +1 |   | 08/10/2026 | |||||
| 1M | 3.8 s | 128 tps | $0.10/M | $0.20/M | Read$0.002/M | — | +2 | | 08/05/2026 | |||||
| 1M | 3.8 s | 203 tps | $1.25/M | $4.25/M | Read$0.15/M | — | +2 | | 07/09/2026 | |||||
| 1M | 3.0 s | 68 tps | $0.10/M | $0.20/M | Read$0.002/M | — | +2 | | — | |||||
| 1M | 3.0 s | 53 tps | $1.25/M | $4.25/M | Read$0.15/M | — | +2 | | — |
Copy link to headingAbout Llama 3.1 8B Instruct
Meta released Llama 3.1 8B Instruct alongside the broader Llama 3.1 family on July 23, 2024, bringing two major upgrades over previous 8B Llama releases: an extended context window of 128K tokens and full multilingual capability across eight languages. Both improvements also apply to the 70B, but the 8B delivers them at substantially lower serving cost and higher throughput. This makes it the practical entry point for most teams evaluating the Llama 3.1 generation.
Tool use is a trained capability in this generation. The 8B can participate in agentic workflows that call external tools, making it suitable for lightweight agent pipelines where the per-call cost of a larger model would be prohibitive. Combined with the context of 128K tokens, the model can maintain substantial conversation history or reference extensive retrieved documents within a single call.
Copy link to headingWhat To Consider When Choosing a Provider
- Configuration: Because the 8B model runs efficiently on modest GPU hardware, providers may offer a wider range of hosting tiers. Decide whether shared or dedicated capacity fits your traffic patterns. Compare $0.02 and $0.05.
- Zero Data Retention: Zero Data Retention is available for this model. It is offered on a per-provider and model basis. See the documentation for details.
- Authentication: AI Gateway authenticates requests using an API key or OIDC token. You do not need to manage provider credentials directly.
Copy link to headingWhen to Use Llama 3.1 8B Instruct
Best for
- Cost-efficient inference: High-throughput applications where per-token economics matter for chatbots, content moderation, and classification at scale
- Multilingual applications: Support across English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai without stepping up to 70B cost
- Lightweight agentic pipelines: Reliable tool-use without the latency and serving cost of a larger model
Consider alternatives when
- Deeper reasoning needed: The task demands the capability of the 70B, particularly for multi-step math or complex coding problems
- Image understanding needed: No Llama 3.1 model supports vision input, so Llama 3.2 11B or 90B are the appropriate choices
- Top instruction following: Instruction following quality needs to be maximized and Llama 3.3 70B's refinements justify the larger scale
Copy link to headingConclusion
Llama 3.1 8B Instruct fills the gap for teams that need open-weight multilingual capability with a context window of 128K tokens at accessible serving costs. Tool-use support makes it a common default for production deployments where per-token efficiency drives architectural decisions.
Copy link to headingFrequently Asked Questions
What benchmarks did Llama 3.1 8B Instruct perform well on?
Llama 3.1 8B Instruct was evaluated across more than 150 benchmark datasets spanning multiple languages. The 8B lines up with closed and open models of a similar parameter count on general knowledge, instruction following, and tool-use tasks.
What tool-use behaviors are supported?
The model supports function calling and structured output generation as trained behaviors, not just prompt-pattern following. It operates within larger agentic systems that orchestrate external API calls or tool invocations.
How does the 8B handle the full context of 128K tokens in practice?
The model holds long documents, conversation histories, or retrieved content in full rather than requiring chunking.
What languages are supported beyond English?
The seven additional languages are German, French, Italian, Portuguese, Hindi, Spanish, and Thai, all with multilingual instruction following and conversational capability.