For an AI agent to read web pages and PDFs as Markdown, web_fetch from Cloudflare’s Agents SDK is enough; for JavaScript apps or sessions with login, you still need a browser. The tool arrived in agents@0.28.0 on October 9, 2026, and at yoDEV we deployed it in a real Worker to check what it actually returns.
yoDEV Tested · October 13, 2026
We deployedweb_fetchfromagents@0.28.0in a real Worker with a Workers AI binding and pointed it at seven URLs. The results we report as ours come from that test.
The same version also brought a browser tool. So the useful question isn’t just how to use web_fetch, but when it’s enough and when your agent needs a real browser. Our test answers that with real results, not documentation.
What is Cloudflare’s Agents SDK?
The Agents SDK is Cloudflare’s framework for building and deploying AI agents on Workers. It’s published on npm as agents and developed in the public GitHub repository cloudflare/agents. It brings together the pieces an agent needs: state, scheduled tasks, WebSockets, MCP clients, and as of now, ready-to-use tools like web_fetch and browser, with adapters for AI SDK, TanStack AI, and Cloudflare’s Pi harness.
What does web_fetch do?
web_fetch takes a URL and returns its content as text the model can read, along with the final URL, HTTP status code, and content type. According to the official documentation:
| Input | What web_fetch does |
|---|---|
| HTML page | Converts it to Markdown with Workers AI Markdown Conversion |
| Site compatible with Markdown for Agents | Requests Markdown first and skips conversion |
| PDFs and compatible Office documents | Converts them to Markdown with Workers AI |
| JSON | Formats it for readability |
| Long content | Returns a 20,000-character window by default, with an offset to continue |
| Any request | Applies your allowlist of hosts, redirect controls, byte limit, and timeout |
It makes simple GET requests: it doesn’t run JavaScript, doesn’t send cookies, and doesn’t log in.
What does web_fetch return? We tested it in a real Worker
On October 13, 2026, we deployed a temporary Worker in yoDEV’s Cloudflare account with agents@0.28.0, the fetchWeb() function, and a Workers AI binding, and pointed it at seven URLs. No model and no Durable Object: just the tool’s output. When done, we deleted the Worker.
| Case | Result | Time |
|---|---|---|
| Cloudflare documentation page | 200 · text/markdown → markdown: the site delivered Markdown directly, no conversion · 13,245 characters |
136 ms |
| arXiv summary page (HTML) | 200 · text/html → markdown via Workers AI · 8,347 characters |
692 ms |
| “Attention Is All You Need” PDF | 200 · application/pdf → markdown · 39,813 characters |
~2.0 s |
Same PDF, second window (offset=1480) |
chars 1480–2601 of 39813 |
~1.6 s |
| Excalidraw (JavaScript app) | 200 OK, only 270 characters: title, description, and image, no app content | 159 ms |
example.com (not in allowlist) |
Error web_fetch_disallowed_url, retryable: false |
77 ms |
localhost |
Error web_fetch_disallowed_url |
74 ms |
Each time corresponds to a single execution from one location. Take them as reference, not benchmark.
Three findings matter more than the numbers:
- A JavaScript app doesn’t fail: it returns almost nothing. Excalidraw responded 200 with a title and description. An agent that only checks the status code would believe it read the page. The documentation describes a
web_fetch_blockederror for anti-bot challenges, but the empty shell of a client-rendered app produces no error. - The output from a PDF is useful, not flawless. The Markdown starts with a PDF metadata block (creator, producer, dates) before
### Page 1, and in the first lines some words get stuck together (“NeedAshish Vaswani”). It works for a model to read and summarize it; not to publish as-is. - Each
offsetre-downloads and converts the document. The second PDF window cost another 1.6 seconds of conversion, as the documentation warns. It also cut at a line break (2,601 instead of the requested limit), which avoids splitting sentences in half.
How do you install web_fetch with Cloudflare Workers AI?
You need Agents SDK 0.28.0 or higher and a Workers AI binding. These steps come from Cloudflare’s documentation.
Update the SDK in your Worker project:
npm i agents@latest
Add the Workers AI binding to wrangler.jsonc, which web_fetch uses to convert HTML and documents:
{
"ai": {
"binding": "AI",
"remote": true
}
}
In local development, conversions run remotely against your Cloudflare account. Markdown, JSON, and plain text don’t go through Workers AI.
Then add the tool to your existing AI SDK call:
import { streamText } from "ai";
import { webFetchTool } from "agents/webfetch/ai-sdk";
const tools = {
web_fetch: webFetchTool({ binding: env.AI }),
};
const result = streamText({ model, messages, tools });
The SDK also includes adapters in agents/webfetch/tanstack-ai and agents/webfetch/pi.
How do you extract text from a PDF or web page without calling a model?
Use fetchWeb() from agents/webfetch. It returns the full page already converted, and windowWebFetchPage() together with renderWebFetchPage() produce the same windowed result that adapters deliver to the model. Doesn’t need Durable Object or browser. This is the core of the Worker we tested:
import {
fetchWeb,
renderWebFetchPage,
windowWebFetchPage,
} from "agents/webfetch";
const page = await fetchWeb(
{ url },
{ binding: env.AI, allowedHosts: ["developers.cloudflare.com", "arxiv.org"] },
);
const output = windowWebFetchPage(page, { offset: 0, pageChars: 20_000 });
return new Response(renderWebFetchPage(output));
This is what the model receives (from our test with the PDF):
web_fetch: https://arxiv.org/pdf/1706.03762 · 200 · application/pdf → markdown · chars 0–1480 of 39813 · continue with offset=1480
<untrusted_web_content>
...
</untrusted_web_content>
If you’re looking to convert PDF to Markdown on your own machine, without Workers, check out MarkItDown, Microsoft’s tool that converts any document into context for your AI.
How do you read a long PDF in parts?
Pass the offset from the previous result to get the next window. If you write your own handler, save the WebFetchPage that fetchWeb() returned and call windowWebFetchPage() again with the new offset: that way you read the next part without downloading or converting the document again. In our test, skipping that step meant paying for PDF conversion twice.
How do you limit which sites your agent can read?
Define the request policy when creating the tool. The model only chooses url, offset, and format; you choose the limits:
| Option | Default value | Effect |
|---|---|---|
allowedHosts |
Any public host | Hosts the tool can read; *.example.com covers subdomains, not example.com |
blockedHosts |
None | Hosts always rejected, checked first |
pageChars |
20000 | Maximum characters per result |
maxBytes |
5000000 | Maximum response size |
timeoutMs |
30000 | Total timeout, including redirects and conversion |
maxRedirects |
5 | Redirects it follows (0 to 20) |
allowPrivateHosts |
false | Allows local and private hosts, for local development |
The built-in source also rejects schemes other than HTTP, credentials in the URL, URLs longer than 2,000 characters, and redirects from HTTPS to HTTP, and re-validates each redirect destination. One limit worth knowing: it blocks literally-written private IPs and local hostnames, but doesn’t resolve DNS to check what a public host points to. In an agent reading arbitrary URLs, allowedHosts is the control that truly reduces the attack surface.
web_fetch or browser: which does your agent need?
Use web_fetch when content comes in the HTML or file the server sends: documentation, blogs, PDFs, Office documents, JSON APIs. In our test it read all that between milliseconds and a couple of seconds, without launching any browser.
Use a browser (Cloudflare’s browser tool, which arrived in the same 0.28.0 version) when the agent needs:
- Content rendered with JavaScript. Our Excalidraw test returned 200 with no content.
- A logged-in session.
web_fetchdoesn’t send cookies or credentials. - Interaction or screenshots. Click, fill forms, see how the page looks.
- Pages behind an anti-bot challenge, which
web_fetchreports asweb_fetch_blocked.
A practical rule: try web_fetch first, and if the result is suspiciously short for the page (like 270 characters for a full app), switch to the browser. If you want a Chrome headless as an HTTP service, we already covered Cloudflare’s Browser Run.
What limits does web_fetch have?
- Prompt injection. Content arrives wrapped in
<untrusted_web_content>, which marks it as data, not instructions. Cloudflare’s documentation clarifies that doesn’t guarantee protection against prompt injection. - Cost. HTML and document conversion runs on Workers AI and charges to your account. At the time we published this note, we hadn’t verified the price of Markdown Conversion; check it before processing large batches of PDFs.
- Unsupported content. Doesn’t support images, audio, or video, and
format: "raw"can’t return PDF. - A young API. As of October 13, 2026,
web_fetchrequires Agents SDK 0.28.0 or higher, and the package is still in 0.x versions. Expect changes in imports and options; check the documentation before updating.
How about you? What kind of content does your agent need to read: documentation, PDFs, or web apps?
- With a simple fetch or a tool like web_fetch
- With a headless browser (Playwright, Puppeteer, Browser Run)
- I haven’t given it web access yet
- I prefer another option (tell us which)
Comment below or, if this article reached you by email, reply directly to the email: your reply gets published here.
Related: MarkItDown: Microsoft’s Tool That Converts Any Document Into Context for Your AI