Skip to content
Docs

Build a web research agent with Workflow SDK

Build a web research agent that searches the web and returns a cited report. Powered by AI SDK and Workflow SDK, it journals every step so progress survives any crash or retries.

Anshuman BhardwajContent Engineer

Build a web research agent that searches the web, extracts findings from each source, and returns a cited report without losing progress when a crash, restart, or deployment interrupts a run that can loop many times. You’ll use AI SDK with Workflow SDK to run a research loop not as one long request but as a sequence of durable, journaled steps, so the search results and findings already recorded survive any interruption.

In this guide, you'll build a Next.js app that hands a user's question to an agent running inside Vercel Workflows, performs web searches across various sources, and extracts findings from each. Along with this, you'll learn to stream the agent's progress to the browser in real time and return a final report with citations.

Deploy the template now, or read on for a deeper look at how it all works.

Durable web research agent with Workflow SDK

Ask a question and the app researches the web, fetches source pages, extracts cited findings, and synthesizes a grounded research brief while streaming each step to the browser in real time.

Deploy Template

This guide covers the core implementation. The AI assistant prompt above covers all the details your coding agent needs, and you can see the full implementation in the template repository.

Copy link to headingQuick start with an AI coding agent

If you are working with an AI coding agent, hand it the project and this prompt:

Agent Prompt

I want to build a durable web research agent using the durable web research agent template. Read the setup instructions at https://agent-resources.dev/durable-web-research-agent-template.md and follow them. They will cover deploying the template, building with the Workflow SDK, how everything works overall, and more.

Copy link to headingVercel Plugin

Turn your agent into a Vercel expert with this plugin. It gives your coding agent current knowledge of the Vercel products this template uses, including Vercel Workflows, AI Gateway, and AI SDK. The plugin is optional; it is not required to use this template or for this guide.

Terminal
npx plugins add vercel/vercel-plugin

Copy link to headingOverview

The request flow looks like this:

API structure
POST /api/research
  -> start(researchWorkflow, [question])
  -> searchWeb step using AI Gateway search
  -> for each source page, extractFindings step
  -> synthesizeReport step
  -> return ResearchReport


GET /api/readable/:runId
  -> read workflow stream
  -> transform workflow chunks into UI chunks
  -> send SSE events to the browser


GET /api/run/:runId
  -> read status and returnValue

Copy link to headingPrerequisites

Before you begin, make sure you have:

Copy link to headingConfigure credentials

For local development, link the Vercel project and pull environment variables:

vercel link
vercel env pull

AI Gateway authenticates requests using Vercel OIDC tokens, which Vercel generates and links to your project automatically.

Copy link to headingCreate the project

Create a new Next.js app with create-next-app :

Terminal
pnpm create next-app@latest durable-web-research-agent --yes
cd durable-web-research-agent

Then install the Workflow and AI SDK packages:

Terminal
pnpm add workflow @ai-sdk/workflow ai zod @vercel/oidc ajv

Copy link to headingConfigure Next.js for Workflow

Wrap your Next.js config with withWorkflow:

next.config.ts
import type { NextConfig } from "next";
import { withWorkflow } from "workflow/next";


const nextConfig: NextConfig = {
  serverExternalPackages: ["@vercel/oidc", "ajv"],
};


export default withWorkflow(nextConfig);

Copy link to headingDefine the data types

Keep the workflow state compact. Store findings, URLs, snippets, and short page text. Avoid writing the entire crawled websites into the workflow event log.

workflows/research-types.ts
export type Finding = {
  claim: string;
  sourceUrl: string;
  snippet: string;
};


export type ResearchReport = {
  sections: Array<{
    heading: string;
    body: string;
  }>;
  citations: Array<{
    claim: string;
    sourceUrl: string;
  }>;
};


export type SearchWebResult = {
  query: string;
  answer: string;
  sources: Array<{
    title?: string;
    url: string;
    text?: string;
  }>;
};


export type SourcePage = {
  title?: string;
  url: string;
  text: string;
  error?: string;
};


export type ExtractFindingsResult = {
  findings: Finding[];
};

Copy link to headingCreate durable research steps

The Workflow SDK gives the app durable control flow and the AI SDK gives the app model calls, AI Gateway tools, and structured outputs. Implement the two branches, as follows:

  • Put generateText(), fetch(), Gateway search tools, and any other side effects inside "use step" functions.
  • Keep the "use workflow" function focused on orchestration, loops over already-returned data, and calls to step functions.
  • Return compact typed objects from steps so Workflow can journal them and replay the run without calling the model again.

The research steps do the work with side effects:

  • searchWeb() calls an AI Gateway search tool.
  • extractFindingsFromPages() calls an AI Gateway model with Output.object.
  • fetchSourcePage() helps when your search tool returns URLs without enough page text.

The example below uses AI Gateway's parallelSearch tool. Search belongs in a step because it is external I/O: it can fail, retry, rate-limit, or return different results over time. Once the step completes, Workflow records the selected sources, and replay can continue from those recorded sources instead of searching again.

parallelSearch returns source-oriented search results with excerpts that are already useful to an LLM. If your project prefers Perplexity-backed search, swap in gateway.tools.perplexitySearch({ maxResults: 5 }).

workflows/research-steps.ts
import { generateText, gateway, Output } from "ai";
import { z } from "zod";
import type {
  ExtractFindingsResult,
  SearchWebResult,
  SourcePage,
} from "./research-types";


const MAX_SOURCE_PAGES = 5;
const MAX_PAGE_CHARS = 12000;


const extractedFindingsSchema = z.object({
  findings: z.array(
    z.object({
      claim: z.string(),
      sourceUrl: z.string().describe("One of the source URLs provided."),
      snippet: z.string(),
    }),
  ),
});


function isRecord(value: unknown): value is Record<string, unknown> {
  return typeof value === "object" && value !== null;
}


function firstString(...values: unknown[]) {
  return values.find((value): value is string => typeof value === "string");
}


function normalizeGatewaySearchOutput(output: unknown): SourcePage[] {
  if (!isRecord(output)) {
    return [];
  }


  const candidates =
    Array.isArray(output.results)
      ? output.results
      : Array.isArray(output.sources)
        ? output.sources
        : [];


  return candidates
    .filter(isRecord)
    .map((result) => {
      const url = firstString(result.url, result.link, result.sourceUrl);
      const text = firstString(
        result.excerpt,
        result.snippet,
        result.text,
        result.content,
      );


      if (!url) {
        return null;
      }


      return {
        title: firstString(result.title),
        url,
        text: (text ?? "").slice(0, MAX_PAGE_CHARS),
      };
    })
    .filter((page): page is SourcePage => page !== null)
    .slice(0, MAX_SOURCE_PAGES);
}


export async function searchWeb({ query }: { query: string }) {
  "use step";


  const result = await generateText({
    model: "openai/gpt-5-nano",
    prompt: `
Find reliable source pages for this research question:


${query}


Return sources that contain concrete technical details, tradeoffs, and
implementation facts. Prefer primary docs and high-signal technical articles.
`.trim(),
    tools: {
      parallel_search: gateway.tools.parallelSearch({
        mode: "agentic",
        maxResults: MAX_SOURCE_PAGES,
        excerpts: {
          maxCharsPerResult: MAX_PAGE_CHARS,
        },
      }),
    },
    toolChoice: { type: "tool", toolName: "parallel_search" },
  });


  const sources = result.toolResults
    .filter((toolResult) => toolResult.toolName === "parallel_search")
    .flatMap((toolResult) => normalizeGatewaySearchOutput(toolResult.output));


  return {
    query,
    answer: result.text,
    sources,
  } satisfies SearchWebResult;
}

Copy link to headingFetch pages when search only returns URLs

Some search tools return URLs and snippets but not enough source text. In that case, add a fetchSourcePage step and call it before extraction. Keep it as a workflow step because page fetches are side effects and can fail or need retries.

workflows/research-steps.ts
function decodeHtmlEntities(value: string) {
  return value
    .replaceAll("&nbsp;", " ")
    .replaceAll("&amp;", "&")
    .replaceAll("&lt;", "<")
    .replaceAll("&gt;", ">")
    .replaceAll("&quot;", "\"")
    .replaceAll("&#39;", "'");
}


function htmlToText(html: string) {
  return decodeHtmlEntities(
    html
      .replace(/<script[\s\S]*?<\/script>/gi, " ")
      .replace(/<style[\s\S]*?<\/style>/gi, " ")
      .replace(/<noscript[\s\S]*?<\/noscript>/gi, " ")
      .replace(/<svg[\s\S]*?<\/svg>/gi, " ")
      .replace(/<[^>]+>/g, " ")
      .replace(/\s+/g, " ")
      .trim(),
  );
}


export async function fetchSourcePage({
  title,
  url,
}: {
  title?: string;
  url: string;
}): Promise<SourcePage> {
  "use step";


  try {
    const response = await fetch(url, {
      headers: {
        accept: "text/html,text/plain;q=0.9,*/*;q=0.8",
        "user-agent": "Mozilla/5.0 (compatible; DurableResearchAgent/1.0)",
      },
    });


    if (!response.ok) {
      return {
        title,
        url,
        text: "",
        error: `Fetch failed with status ${response.status}.`,
      };
    }


    const contentType = response.headers.get("content-type") ?? "";
    const body = await response.text();
    const text = contentType.includes("html")
      ? htmlToText(body)
      : body.replace(/\s+/g, " ").trim();


    return {
      title,
      url,
      text: text.slice(0, MAX_PAGE_CHARS),
    };
  } catch (error) {
    return {
      title,
      url,
      text: "",
      error: error instanceof Error ? error.message : "Unknown fetch error.",
    };
  }
}

Copy link to headingExtract findings with structured output

Extraction is separate from search on purpose: search finds candidate source material and extraction decides which claims are useful enough to keep and turns messy page text into compact, typed research artifacts.

Keeping extraction separate gives you three benefits:

  • The final report can depend on a stable Finding[] instead of raw search prose.
  • The UI can stream findings as soon as each page is processed.
  • If synthesis fails, Workflow reuses the already-extracted findings instead of asking the search model to recreate them.

Use Output.object() so the extraction step returns predictable data.

workflows/research-steps.ts
export async function extractFindingsFromPages({
  pages,
  question,
}: {
  pages: SourcePage[];
  question: string;
}): Promise<ExtractFindingsResult> {
  "use step";


  const readablePages = pages.filter((page) => page.text.trim().length > 0);
  if (readablePages.length === 0) {
    return { findings: [] };
  }


  const allowedUrls = new Set(readablePages.map((page) => page.url));
  const { output } = await generateText({
    model: "anthropic/claude-haiku-4.5",
    output: Output.object({
      schema: extractedFindingsSchema,
    }),
    prompt: `
Question:
${question}


Fetched source pages:
${JSON.stringify(
  readablePages.map((page) => ({
    title: page.title,
    url: page.url,
    text: page.text,
  })),
  null,
  2,
)}


Extract 8 to 12 concise findings that answer the question. Each finding must
use one of the provided page URLs as sourceUrl and include a short snippet from
that page text. Prefer concrete technical differences, tradeoffs, use cases,
ecosystem facts, deployment details, and limitations.
`.trim(),
  });


  return {
    findings: output.findings.filter((finding) =>
      allowedUrls.has(finding.sourceUrl),
    ),
  };
}

The URL filter after generation prevents the model from inventing citations that were not in the fetched source set.

Copy link to headingOrchestrate the workflow

The workflow body should mostly coordinate steps. It can loop over the bounded search results, but it should not perform network requests or model calls directly.

This workflow also writes progress events to getWritable(). Those events look like AI SDK model-call stream parts, so the UI can render tool starts and finishes.

workflows/research-workflow.ts
import type { ModelCallStreamPart } from "@ai-sdk/workflow";
import { getWritable } from "workflow";
import {
  extractFindingsFromPages,
  fetchSourcePage,
  searchWeb,
} from "./research-steps";
import type { Finding } from "./research-types";
import { synthesizeReport } from "./synthesize";


const MAX_SOURCE_PAGES = 5;


type WorkflowStreamPart =
  | ModelCallStreamPart
  | { type: "finish-step" }
  | { type: "start-step" };


function findingKey(finding: Finding) {
  return `${finding.claim}\n${finding.sourceUrl}`;
}


function dedupeFindings(findings: Finding[]) {
  return [
    ...findings.reduce((deduped, finding) => {
      deduped.set(findingKey(finding), finding);
      return deduped;
    }, new Map<string, Finding>()).values(),
  ];
}


async function writeToolCall(
  writable: WritableStream<WorkflowStreamPart>,
  toolCallId: string,
  toolName: string,
  input: unknown,
) {
  "use step";


  const writer = writable.getWriter();
  try {
    await writer.write({
      type: "tool-call",
      toolCallId,
      toolName,
      input,
    } as WorkflowStreamPart);
  } finally {
    writer.releaseLock();
  }
}


async function writeToolResult(
  writable: WritableStream<WorkflowStreamPart>,
  toolCallId: string,
  toolName: string,
  output: unknown,
) {
  "use step";


  const writer = writable.getWriter();
  try {
    await writer.write({
      type: "tool-result",
      toolCallId,
      toolName,
      output,
    } as WorkflowStreamPart);
    await writer.write({ type: "finish-step" });
    await writer.write({ type: "start-step" });
  } finally {
    writer.releaseLock();
  }
}


export async function researchWorkflow(question: string) {
  "use workflow";


  const writable = getWritable<WorkflowStreamPart>();
  const normalizedQuestion = question.trim();
  const findings: Finding[] = [];


  await writeToolCall(writable, "search-0", "searchWeb", {
    query: normalizedQuestion,
    maxSources: MAX_SOURCE_PAGES,
  });
  const searchResult = await searchWeb({ query: normalizedQuestion });
  await writeToolResult(writable, "search-0", "searchWeb", searchResult);


  for (const [index, source] of searchResult.sources
    .slice(0, MAX_SOURCE_PAGES)
    .entries()) {
    const fetchCallId = `fetch-${index}`;
    await writeToolCall(writable, fetchCallId, "fetchPage", source);
    const page = source.text
      ? {
          title: source.title,
          url: source.url,
          text: source.text,
      }
      : await fetchSourcePage(source);


    await writeToolResult(writable, fetchCallId, "fetchPage", page);


    const extractCallId = `extract-${index}`;
    await writeToolCall(writable, extractCallId, "extractFindings", {
      sourceUrl: page.url,
      title: page.title,
      hasText: page.text.trim().length > 0,
    });
    const extracted = await extractFindingsFromPages({
      pages: [page],
      question: normalizedQuestion,
    });
    findings.push(...extracted.findings);
    await writeToolResult(writable, extractCallId, "extractFindings", extracted);
  }


  return synthesizeReport(normalizedQuestion, dedupeFindings(findings));
}

To keep the run predictable even if a search provider returns more results than requested, this adds the following guardrails:

  • the Gateway search tool receives maxResults: 5
  • the workflow slices searchResult.sources to MAX_SOURCE_PAGES

Copy link to headingSynthesize the final report

Synthesis should read only the extracted findings. This keeps the report grounded in durable evidence instead of the model's memory of earlier search text.

That design also makes failure recovery cheaper. If the final report step retries, it receives the same Finding[] from the event log and does not repeat search or extraction. If there are no findings, return an explicit empty-state report instead of asking the model to improvise.

workflows/synthesize.ts
import { generateText, Output } from "ai";
import { z } from "zod";
import type { Finding, ResearchReport } from "./research-types";


const reportSchema = z.object({
  sections: z.array(
    z.object({
      heading: z.string(),
      body: z.string(),
    }),
  ),
  citations: z.array(
    z.object({
      claim: z.string(),
      sourceUrl: z.string().describe("The source URL for the cited claim."),
    }),
  ),
});


export async function synthesizeReport(
  question: string,
  findings: Finding[],
): Promise<ResearchReport> {
  "use step";


  if (findings.length === 0) {
    return {
      sections: [
        {
          heading: "Insufficient sourced findings",
          body:
            "The research run completed without any extracted findings, so a cited report cannot be generated from the workflow state.",
        },
      ],
      citations: [],
    };
  }


  const { output } = await generateText({
    model: "anthropic/claude-sonnet-4.6",
    output: Output.object({
      schema: reportSchema,
    }),
    prompt: `
Question:
${question}


Findings:
${JSON.stringify(findings, null, 2)}


Write a deep, practical research brief that answers the question using only
these findings. For comparison questions, include sections for positioning,
core capabilities, ecosystem and language/runtime fit, agent/orchestration
support, deployment tradeoffs, and a recommendation matrix. Every section must
ground its claims in the findings. Include citations for the most important
claims using the exact sourceUrl values from the findings.
`.trim(),
  });


  return {
    sections: output.sections,
    citations:
      output.citations.length > 0
        ? output.citations
        : findings.slice(0, 10).map((finding) => ({
            claim: finding.claim,
            sourceUrl: finding.sourceUrl,
          })),
  };
}

The deterministic citation fallback matters. Structured output improves shape, but it does not guarantee the model will populate every optional-looking field the way your UI expects.

Copy link to headingStart the workflow from an API route

Start the workflow with a POST request and return the run ID. This route returns immediately. The browser can then subscribe to the readable stream and poll status by runId.

app/api/research/route.ts
import { NextResponse } from "next/server";
import { start } from "workflow/api";
import { researchWorkflow } from "@/workflows/research-workflow";


export async function POST(req: Request) {
  const { question } = (await req.json()) as { question?: string };


  if (!question?.trim()) {
    return NextResponse.json(
      { error: "Question is required." },
      { status: 400 },
    );
  }


  const run = await start(researchWorkflow, [question.trim()]);


  return NextResponse.json({
    runId: run.runId,
  });
}

Copy link to headingStream workflow progress

Workflow streams are read by run ID. The code below converts the workflow stream into AI SDK UI chunks and sends them as Server-Sent Events.

app/api/readable/[runId]/route.ts
import { toUIMessageChunk } from "@ai-sdk/workflow";
import type { UIMessageChunk } from "ai";
import { getRun } from "workflow/api";


type RouteContext = {
  params: Promise<{ runId: string }>;
};


type ModelCallStreamPartWithToolName = {
  type: string;
  toolName?: string;
};


function createResearchUIChunkTransform() {
  return new TransformStream<unknown, UIMessageChunk>({
    start(controller) {
      controller.enqueue({ type: "start" });
      controller.enqueue({ type: "start-step" });
    },
    transform(part, controller) {
      const chunk = toUIMessageChunk(part as never);
      if (!chunk) {
        return;
      }


      if (
        chunk.type === "tool-output-available" &&
        typeof part === "object" &&
        part !== null &&
        "toolName" in part &&
        typeof (part as ModelCallStreamPartWithToolName).toolName === "string"
      ) {
        controller.enqueue({
          ...chunk,
          toolName: (part as ModelCallStreamPartWithToolName).toolName,
        } as UIMessageChunk);
        return;
      }


      controller.enqueue(chunk);
    },
    flush(controller) {
      controller.enqueue({ type: "finish-step" });
      controller.enqueue({ type: "finish" });
    },
  });
}


export async function GET(_request: Request, { params }: RouteContext) {
  const { runId } = await params;


  let run;
  try {
    run = await getRun(runId);
    await run.status;
  } catch {
    return Response.json(
      { error: `Run ${runId} was not found.` },
      { status: 404 },
    );
  }


  const readable = (
    run.getReadable() as unknown as ReadableStream<unknown>
  ).pipeThrough(createResearchUIChunkTransform());


  const encoder = new TextEncoder();
  const stream = readable.pipeThrough(
    new TransformStream<unknown, Uint8Array>({
      transform(chunk, controller) {
        const data = typeof chunk === "string" ? chunk : JSON.stringify(chunk);
        controller.enqueue(encoder.encode(`data: ${data}\n\n`));
      },
    }),
  );


  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream; charset=utf-8",
      "Cache-Control": "no-cache, no-transform",
      Connection: "keep-alive",
      "X-Accel-Buffering": "no",
    },
  });
}

The custom transform preserves toolName on output chunks. The UI needs that metadata to decide which tool outputs contain findings.

Copy link to headingRead status and the final report

The final report is the workflow return value.

app/api/run/[runId]/route.ts
import { NextResponse } from "next/server";
import { getRun } from "workflow/api";


type RouteContext = {
  params: Promise<{ runId: string }>;
};


function toIso(value: Date | null | undefined) {
  return value ? value.toISOString() : null;
}


export async function GET(_request: Request, { params }: RouteContext) {
  const { runId } = await params;


  try {
    const run = await getRun(runId);


    const [status, workflowName, createdAt, startedAt, completedAt, returnValue] =
      await Promise.all([
        run.status,
        run.workflowName,
        run.createdAt,
        run.startedAt,
        run.completedAt,
        run.returnValue.catch(() => null),
      ]);


    return NextResponse.json({
      runId,
      status,
      workflowName,
      createdAt: toIso(createdAt),
      startedAt: toIso(startedAt),
      completedAt: toIso(completedAt),
      returnValue,
    });
  } catch {
    return NextResponse.json(
      { error: `Run ${runId} was not found.` },
      { status: 404 },
    );
  }
}

Copy link to headingConnect a client

The backend is intentionally UI-agnostic. A React console, a CLI, or an internal dashboard can all use the same three contracts:

EndpointClient responsibility
POST /api/researchSend { question }, receive { runId }
GET /api/readable/:runIdSubscribe to streamed UI chunks
GET /api/run/:runIdPoll status and read the final returnValue

The stream route already converts workflow events into AI SDK UI chunks. A client does not need to know how Workflow stores the run. It only needs to interpret these events:

  • tool-input-available means a workflow phase started, such as searchWeb, fetchPage, or extractFindings.
  • tool-output-available means that phase finished.
  • tool-output-available with toolName: "extractFindings" contains { findings: Finding[] }.
  • finish means the readable stream ended.

Dedupe streamed findings by toolCallId and finding index. A reconnect or retry can replay chunks that the browser has already seen.

You now have a web research agent ready to answer your questions. The complete template includes a polished frontend with real-time streaming for findings and workflow steps. To get started, clone this template’s GitHub repository, configure credentials, and run the local app:

Terminal
pnpm install
pnpm dev

Open http://localhost:3000 and submit:

Prompt
When to use Vercel Workflows?

Copy link to headingTroubleshooting

Copy link to headingSearch returns no source pages

Inspect result.toolResults from the Gateway search call. Gateway tools can differ in output shape. Normalize results, sources, excerpts, and URLs before passing data to extraction.

Copy link to headingSearch provider access is restricted

Use a Gateway-executed search tool that is enabled for your team. If parallelSearch is not available, try perplexitySearch, or configure provider access in AI Gateway.

Copy link to headingThe final report has no citations

Make sure synthesis receives extracted findings, not raw search text. Add the deterministic citation fallback shown above so the UI always has source URLs when findings exist.

Copy link to headingLive findings stay empty

Check that the stream route preserves toolName on tool-output-available chunks and that the UI reads findings from extractFindings outputs.

Copy link to headingSteps rerun on every replay

Move side effects out of the workflow body and into "use step" functions. Native fetch, provider calls, timestamps, random values, and writable stream writes should be inside steps.

Copy link to headingResources

Copy link to headingFAQ

Do I need provider-specific API keys for search or model calls?

No. The guide uses AI Gateway model strings and Gateway search tools, so authentication goes through the linked Vercel project and OIDC. You do not need to add OpenAI, Anthropic, or Perplexity keys to the app.

Do I need a database to store findings?

Not for this example. Findings are step outputs in the workflow run, so they are durable for the life of that run. Add a database only if you want long-term storage, cross-run search, or product analytics.

What happens if the workflow crashes halfway through a research run?

Completed steps are already recorded in the Workflow event log. When the app resumes, Workflow can continue from the last completed step instead of starting the whole research run over.

Why use a deterministic workflow pipeline instead of an autonomous agent loop?

A pipeline gives the backend a predictable budget and failure model. The model still performs search, extraction, and synthesis, but Workflow owns the order of operations: search once, process up to five sources, extract findings, then write the report.

Why is extraction separate from search?

Search finds candidate sources; extraction turns source text into durable findings. Keeping them separate lets the UI stream findings as they are recorded, and lets retries reuse extracted findings without repeating search.

Related documentation

More Vercel Workflows guides

Ready to deploy?

Start deployingTalk to an expert