Claude Code web fetch 403: cause and fix

Claude Code web fetch 403 means the website refused the request, not that Claude Code is broken. We pointed WebFetch at three public pages (an npm package, a Medium post, a G2 reviews page) and all three came back 403 with no page text. FATSEARCH read the same three pages, two of them with a free plain request.

We make FATSEARCH. The numbers below are our own runs, dated. Where another reader is the better pick, we say so.

By FATSEARCH · data as of 2026-10-19 · reviewed 2026-10-19 · View as Markdown

What WebFetch returns on a 403

This is the whole tool result Claude got for the npm page react (Claude Code 2.1.280). Medium and G2 returned the same text, word for word:

The server returned HTTP 403 Forbidden.

The response body was not retrieved. If this URL requires authentication, use an authenticated tool (e.g. `gh` for GitHub, or an MCP-provided fetch tool) instead of WebFetch.

That’s 92 tokens and no page. The message hints that the URL may need a login. These three don’t: they’re public pages.

Why it happens

The site runs bot protection. A plain request from the same machine got the same 403: Cloudflare’s “Just a moment…” challenge on npm (cf-mitigated: challenge), a Cloudflare block page on Medium, DataDome on G2 (x-datadome: protected).
Changing the user agent doesn’t help. The same three URLs with a desktop Chrome user agent:

2026-10-19T18:01:41Z	chrome-ua	403 cf-mitigated=challenge	https://www.npmjs.com/package/react
2026-10-19T18:01:41Z	chrome-ua	403 cf-mitigated=	https://medium.com/@liranyoffe/reverse-engineering-claude-code-web-tools-1409249316c3
2026-10-19T18:03:17Z	chrome-ua	403 cf-mitigated=	https://www.g2.com/products/firecrawl/reviews

WebFetch says who it is. Claude Code’s tools reference says it sends a User-Agent beginning with Claude-User. Anthropic lists Claude-User as the agent for fetches a user asked for, and site owners can block it in robots.txt (Anthropic’s crawler page). None of these three sites names Claude-User in robots.txt, so this was a bot wall, not an opt-out. If a site does opt out, leave it alone.
WebFetch has no second route. It makes the request, gets refused, and tells Claude to use an authenticated tool or an MCP fetch tool instead.

The fix: an MCP reader with a fallback

FATSEARCH is a CLI and MCP server that reads pages for agents. It tries a free plain request first; when the site refuses, it retries through your own Oxylabs or Bright Data key. Same three pages, FATSEARCH 0.1.3, a couple of minutes before the WebFetch run.

npm, free plain request, 345 ms. The whole reply is 733 tokens (Anthropic’s count), and the version number is in its first lines:

= j1.u1 react · www.npmjs.com · 8 sections · 1 photo · 824tok · plain 345ms spent $0.0000
u1#0 info | React is a JavaScript library for building user interfaces.. Latest version: 19.3.1, last…
m1 photo png | (no alt) → https://static-production.npmjs.com/a12c728e90758ccd16976b394b964317.png
[u1#1] # (top)
⚠️
npm tokens that bypass 2FA are being restricted — account changes (Aug 2026) and direct publishing (Jan 2027). Learn how to prepare → [l1]
×
[u1#2] # react
19.3.1 • Public • Published 5 days ago

Medium, also the free request, 1,479 ms:

= j2.u2 Reverse Engineering Claude Code Web Tools · medium.com · 34 sections · 1 photo · 5.4ktok · plain 1479ms spent $0.0000

G2 is the hard one. The plain request hit a captcha wall, an Oxylabs try timed out, Bright Data got the page. That took 48,121 ms and cost $0.0015 on Bright Data; FATSEARCH marks the cut-short Oxylabs tries as unverified, since they may or may not be billed:

= j6.u6 The G2 on Firecrawl · www.g2.com · 17 sections · 12 photos · 3.6ktok · plain:captcha_wall✗ oxy:provider_timeout✗≤$0.0010 (unverified) bd✓ oxyjs:abandoned✗≤$0.0015 (unverified) 48121ms spent $0.0015
u6#0 info | author Matías C. · published 2026-10-06T13:58:48-05:00 · modified 2026-10-06T13:58:49-05:00 · rating 4.6 (6 reviews)

Add it to Claude Code (from the installation guide), then save a provider key once if you have one:

claude mcp add --scope user fats -- fats mcp
fats setup   # optional: saves an Oxylabs or Bright Data key from your environment

Without a key FATSEARCH still makes the free request, which was enough for npm and Medium here. Step by step with the checks: web search MCP for Claude Code.

This isn’t a one-off. In our run of 38 hard pages (social, news, shops, travel), Claude’s own web tools had the answer in the text on 17/38 pages, FATSEARCH on 35/38 (method). For the hosted readers on the same pages, see Firecrawl alternatives.

Claude Code web fetch failed: the other errors

A 403 isn’t the only way WebFetch fails. From Claude Code’s error reference and tools reference:

“Unable to verify if domain … is safe to fetch”. Before each fetch, WebFetch sends the hostname to api.anthropic.com for a safety check. If your network blocks that, allowlist api.anthropic.com or set skipWebFetchPreflight: true in settings.
“The safety check … is rate-limited”. The check itself got a 429. Don’t retry in a loop; one later try is fine. The same setting skips the check.
localhost or a bare hostname. WebFetch refuses hosts without a dot before making any request. Use curl through Bash.
A redirect to another host comes back as a text result naming the target; Claude fetches it in a second call.
Long or slow pages. WebFetch reads up to 100,000 characters per call and gives up on a page that hasn’t downloaded within five minutes (CLAUDE_CODE_WEBFETCH_DEADLINE_MS).

For pages that load, WebFetch is fine and needs no setup. Keep it, and add a reader for the pages that refuse it. Every reader we measured on one page: web scraping MCP.

Questions

What does error code 403 mean in Claude?

It's the website's answer, not Claude's: 403 Forbidden means the server refused the request. Claude Code's WebFetch then reports that it didn't retrieve the page. In our test npm, Medium and G2 refused it, and refused a plain request with a desktop Chrome user agent too. That is bot protection on the site, not a Claude setting.

How to fix a 403 Forbidden code?

Check robots.txt first: if the site opts out of Claude-User, respect it. For a public page behind a bot wall, use a reader that can retry through an unblocking provider. FATSEARCH tries a free request first; of our three pages only G2 needed the paid route (Bright Data, $0.0015).

How to enable web fetch in Claude?

In Claude Code, WebFetch is on by default. In the default permission mode it asks before a fetch, unless a rule already allows the domain or it's one of the preapproved documentation sites; "Yes, and don't ask again" saves a WebFetch(domain:…) allow rule. If the tool is missing, check that CLAUDE_CODE_DISABLE_WEB_FETCH isn't set, and on a Team or Enterprise account run /status: WebFetch waits for your organization's policy to load.

Claude Code web fetch 403