# FATSEARCH benchmarks

## Are we better?

| Job | Better? | The numbers (FATSEARCH vs the best rival) |
| --- | --- | --- |
| Reading pages inside an agent (Claude Code) | Yes on money and tokens, about the same right answers, slower on blocked pages | right answers 50 vs 51 · $15.06 vs $53.48 per 1,000 tasks · 5,026 vs 38,129 tokens per task · slowest runs 43.8 s vs 21.5 s (FATSEARCH vs Firecrawl MCP, 51 tasks) |
| Hard pages (social, shops, news) | Yes: more answers (35 vs 24), first answers faster, slower on walled pages | answer found 35 vs 24 · answered within 1 s 12 vs 0 · on the 24 pages both answered, median 23.7 s vs 4.9 s (FATSEARCH 0.2.0 on 2026-10-21 vs Firecrawl on 2026-10-16, 38 pages) |
| Web search quality | Results coming | no numbers yet |

> Money = Claude Sonnet at list price + the tool's own fee. Hard pages and one page: model input only, Claude Sonnet 5.5 at $2/M input tokens (checked 2026-10-19); each reader's fee is listed with its set.
>
> We make FATSEARCH. The numbers below are our own runs, dated. Where another reader is the better pick, we say so.
>
> Method and raw runs: https://fatsearch.com/docs/benchmarks/ (synced from https://github.com/fatsearch/fatsearch/blob/v0.2.3/docs/benchmarks.md).

## FATSEARCH 0.2.0: 35 of 38 hard pages answered, 12 within the first second

Answer in the text: FATSEARCH 0.2.0 found 35 of 38 on 2026-10-21. The table below is the 2026-10-16 run: FATSEARCH 0.1.0 found 35, next best Firecrawl 24. A refusal never counts, however fast.

### Pages answered over time

| Answered within | FATSEARCH 0.2.0 · 2026-10-21 | FATSEARCH 0.1.0 · 2026-10-16 | Firecrawl · 2026-10-16 | Tavily · 2026-10-16 | Jina Reader (no key) · 2026-10-16 | Jina Reader (no key) · 2026-10-20 |
| --- | --- | --- | --- | --- | --- | --- |
| 0.5 s | 4/38 | 3/38 | 0/38 | 4/38 | 1/38 | 0/38 |
| 1 s | 12/38 | 12/38 | 0/38 | 9/38 | 1/38 | 0/38 |
| 2 s | 17/38 | 16/38 | 1/38 | 10/38 | 3/38 | 0/38 |
| 3 s | 17/38 | 16/38 | 4/38 | 11/38 | 3/38 | 4/38 |
| 5 s | 17/38 | 16/38 | 13/38 | 14/38 | 5/38 | 7/38 |
| 10 s | 19/38 | 17/38 | 22/38 | 17/38 | 6/38 | 16/38 |
| 15 s | 19/38 | 20/38 | 23/38 | 17/38 | 6/38 | 16/38 |
| 20 s | 19/38 | 27/38 | 24/38 | 18/38 | 8/38 | 17/38 |
| 30 s | 24/38 | 30/38 | 24/38 | 18/38 | 8/38 | 17/38 |
| 45 s | 33/38 | 34/38 | 24/38 | 19/38 | 8/38 | 17/38 |
| 60 s | 35/38 | 35/38 | 24/38 | 19/38 | 8/38 | 17/38 |
| 90 s | 35/38 | 35/38 | 24/38 | 19/38 | 8/38 | 17/38 |

> 38 pages: 2 pages from one government site were dropped: on 2026-10-20 it blocked every reader we ran, and no hosted reader ever read it.
>
> A page counts once a reader hands back the answer in its text. A refusal, a wall or an empty page never counts, however fast it comes back.
>
> Rivals were measured on 2026-10-16 and FATSEARCH 0.2.0 on 2026-10-21: different days, and pages and providers change in between.
>
> FATSEARCH 0.1.0 (2026-10-16, dashed): 35 answered, 12 within 1 s, 27 within 20 s. 0.2.0 is slower: fewer answers within 1 s or 20 s.
>
> **0.2.0 is slower than 0.1.3 on this run** (19 vs 28 pages within 20 s; 13 pages slower, 1 faster, one new miss on TikTok). On most of the slower pages the paid call that won came back later. An A/B run of the 0.1.3 binary ten minutes after 0.2.0 (same script, keys and pages) was as slow: 33/38 answered, 12/20/21/33 within 1/10/20/60 s. Bright Data's winning call took a median 23.8 s then, against 13.9 s the day before. So this is the providers' hour, not a 0.2.0 change (`measure/2026-10-20-hard-pages-0.2.0`, sub-folder `ab-0.1.3`).
>
> On the 24 pages both answered, Firecrawl’s median was 4.9 s, FATSEARCH 0.2.0 23.7 s. FATSEARCH tries the free request first and moves to a paid provider only when a wall blocks it: it was faster on the 10 pages its free request read and slower on the 14 behind a wall. Measured on 0.2.0 with the cache off.
>
> On the 18 pages both answered, Tavily’s median was 1.0 s, FATSEARCH 0.2.0 23.7 s. On the 8 pages both answered, Jina Reader’s median was 3.5 s, FATSEARCH 0.2.0 12.1 s. On the 17 pages both answered, Jina Reader’s median was 5.8 s on its 2026-10-20 re-run, FATSEARCH 0.2.0 23.4 s.
>
> Source: measure/2026-10-20-hard-pages-0.2.0 (per-page times: FATSEARCH 0.2.0 2026-10-21; FATSEARCH 0.1.0 and the rivals 2026-10-16).

| Reader | Answer in the text (of 38) | Model bill per 1,000 questions | Tokens per question | Reader fee (its words) |
| --- | --- | --- | --- | --- |
| FATSEARCH | 35 (agent answered right 26/38) | $5.20 | 2.6k | — |
| Firecrawl | 24 (agent answered right 24/38) | $5.60 | 2.8k | Free: 1,000 credits a month (one scraped page = one credit). Hobby: $16 a month billed yearly ($19 monthly) for 5,000 credits. Standard: $83 a month billed yearly for 100,000. |
| Tavily | 19 | — | — | Free: 1,000 API credits a month. Pay as you go: $0.008 a credit. A basic extract costs 1 credit per 5 successful URLs (advanced: 2). |
| Jina Reader (no key) | 8 | — | — | Free without a key at 20 requests a minute. With a key it is billed by output tokens; a new key comes with 10M free tokens. |
| Built-in fetch | 17 (page read · agent answered right 17/38) | $16.60 | 8.3k | — |

> The same 38 hard pages (2026-10-16) through three hosted readers and Claude's built-in tools, one run each.
>
> "Built-in" is Claude's own web search and web fetch tools.
>
> In the agent runs, FATSEARCH answers "pending" after 5 seconds while a slow page is still loading. The model often gave up there, which cost it more points than any page it failed to read.
>
> With FATSEARCH the tool runs the command line; with Firecrawl it returns Firecrawl's saved copy, cut to 1,000 tokens.
>
> Agent rows: the 2026-10-16 run, FATSEARCH 0.1.0 (`measure/2026-10-16-hard-pages-speed`).
>
> Firecrawl declined 9 pages outright as sites it does not support (social networks, a news site, a recipe site, an airline).
>
> Tavily tried and came back empty on 16.
>
> Jina's keyless tier refused 21 domains for anonymous use on 2026-10-16 and only 3 on 2026-10-20, so its number is about access, not extraction.
>
> Four answers have gone stale since the questions were written; FATSEARCH 0.1.0 read all 34 that have not.
>
> Source: https://fatsearch.com/docs/benchmarks/#other-readers (2026-10-16).

## FATSEARCH: about the same right answers, 7.6× fewer tokens, 3.6× cheaper

The same 51 web tasks in Claude Code, one run each. A typical task takes about as long (FATSEARCH 5.3 s, Firecrawl 6.7 s); the slowest runs take FATSEARCH 43.8 s, Firecrawl 21.5 s.

| Reader | Right answers (of 51) | Cost per 1,000 tasks | Tokens per question | Median / slowest tenth | Reader fee (its words) |
| --- | --- | --- | --- | --- | --- |
| FATSEARCH | 50 | $15.06 | 5,026 | 5.3 s / 43.8 s | — |
| Firecrawl MCP | 51 | $53.48 | 38,129 | 6.7 s / 21.5 s | — |

> Claude Code 2.1.280, claude-sonnet-5, headless, built-in web tools off; Firecrawl's hosted MCP with an API key vs the FATSEARCH MCP with an Oxylabs key. Right = the agent's final answer matched the known one. Every right answer was also in what the tool returned. We ran FATSEARCH twice more on the same tasks to check stability: 99/102 right.
>
> Slowest runs = the slowest tenth (p90). FATSEARCH is slow on blocked pages: it waits for the paid render, and a known retry bug, which we are fixing, makes the agent wait again (5 runs over a minute, the worst 191 s).
>
> Firecrawl's server cache already held most pages; FATSEARCH started empty, so Firecrawl's times are a best case.
>
> Tokens = Claude Code's own count per task: input + cache + output. The Firecrawl MCP adds 10,977 tokens to every model call (16 tools and its instructions vs FATSEARCH's 1 tool).
>
> $ = Claude at list price, as Claude Code reports it, + the tool: Firecrawl credits at the Hobby plan's $16 per 5,000 credits; FATSEARCH = Oxylabs' price as `fats` prints it.
>
> Source: measure/2026-10-20-claude-code-firecrawl-vs-fats/results.json (2026-10-19).

## When the agent has to search first

Measuring, results soon.

## Over a day: Oxylabs 12–16, Bright Data 8–14 of 19 pages served

| Run (EEST) | Oxylabs key only | p50 | Bright Data key only | p50 |
| --- | --- | --- | --- | --- |
| 03:33–03:53 | 13/19 | 46 s | 14/19 | 21 s |
| 04:37–04:53 | 12/19 | 36.3 s | 8/19 | 17.1 s |
| 06:50–07:08 | 14/19 | 22.2 s | 13/19 | 17.5 s |
| 07:47–08:05 | 13/19 | 30.6 s | 11/19 | 12.9 s |
| 09:15–09:40 | 16/19 | 30.5 s | 8/19 | 23.6 s |
| 11:10–11:26 | 15/19 | 30.3 s | 12/19 | 28.4 s |
| 12:13–12:40 | 14/19 | 30.7 s | 12/19 | 20.9 s |
| 14:05–14:35 | 14/19 | 42.7 s | 10/19 | 25.5 s |
| 17:26–17:45 | 16/19 | 27.2 s | 9/19 | 37.7 s |

Oxylabs key only: 12–16 of 19. Bright Data key only: 8–14 of 19.

**Provider weather.** The same build, run a minute apart, has moved by up to 3 pages.

p50 = median time to the first answer over all 19 pages, served or not.

Three earlier runs on the same 19 pages fell in the same bands: 2026-10-14 18:15–18:37 (Oxylabs 16 and 15 in two passes, Bright Data 14), 2026-10-14 21:19–21:40 (Oxylabs only, 16) and 2026-10-15 01:59–02:40 (Oxylabs 14, Bright Data 10). Both keys together served 14 of the 20 in the one run made with both (2026-10-14). Before the any-one-key ladder, one key got 10 and 12 of 20.

Spend for one 19-page pass: $0.017–0.028 billed per key mode at list prices, plus up to $0.013 in calls `fats` cut, printed as unverified.

Source: https://fatsearch.com/docs/benchmarks/#results-hard-pages-time-to-an-answer (2026-10-15).

## One page: FATSEARCH sends 1,384 tokens, the others 2× to 85× more

One question about Wikipedia “Rust (programming language)” (https://en.wikipedia.org/wiki/Rust_(programming_language)), every reader we measured, default settings. Tokens: Anthropic's count_tokens. Bill = model input only, at $2/M tokens (Claude Sonnet 5.5, checked 2026-10-19).

| Reader | Call | Tokens | Answer | vs FATSEARCH | Bill per 1,000 reads | Measured |
| --- | --- | --- | --- | --- | --- | --- |
| FATSEARCH | outline, then one filter, then one section (plain HTTP, $0) | 1,384 | found | baseline | $2.77 | 2026-10-17 |
| Firecrawl | v2 /scrape, markdown, default settings | 112,517 | found | 81× | $225 | 2026-10-17 |
| Bright Data | Web Unlocker, markdown | 103,625 | found | 75× | $207 | 2026-10-17 |
| Oxylabs | Web Scraper API /v1/scrape, markdown, no JS | 102,609 | found | 74× | $205 | 2026-10-17 |
| Tavily | /extract, basic depth | 97,504 | found | 70× | $195 | 2026-10-17 |
| Jina Reader | r.jina.ai, no key | 94,415 | found | 68× | $189 | 2026-10-17 |
| SearXNG + mcp-searxng | web_url_read, default (whole page as Markdown) | 117,088 | found | 85× | $234 | 2026-10-19 |
| SearXNG + mcp-searxng (other call) | Its headings first, then two sections picked from them (readHeadings, section) | 3,037 | found | 2× | $6.07 | 2026-10-19 |
| Firecrawl self-hosted | v2.11.162 from the self-host guide, /v2/scrape, markdown, default settings | 107,332 | found | 78× | $215 | 2026-10-19 |
| Firecrawl self-hosted (other call) | Through firecrawl-mcp (firecrawl_scrape), the tool text an agent gets | 113,609 | found | 82× | $227 | 2026-10-19 |
| Crawl4AI | Docker server 0.9.5, MCP md tool, default filter fit | 74,896 | found | 54× | $150 | 2026-10-19 |
| Crawl4AI (other call) | With the bm25 filter and our question as the query | 36,307 | found | 26× | $72.61 | 2026-10-19 |
| open-webSearch | fetchWebContent, default maxChars of 30,000 (cut, and it says truncated) | 11,609 | found | 8× | $23.22 | 2026-10-19 |
| open-webSearch (other call) | With maxChars 200000 (its maximum), the whole page | 34,881 | found | 25× | $69.76 | 2026-10-19 |
| mcp-server-fetch | fetch tool, default max_length of 5,000 characters: one call, the top of the page, cut with a hint to read on from start_index 5000 | 2,373 | not in text | 2× | $4.75 | 2026-10-19 |
| mcp-server-fetch (other call) | With max_length 999999 (its maximum), the whole page in one call | 76,147 | found | 55× | $152 | 2026-10-19 |

Source: https://fatsearch.com/docs/benchmarks/#one-page-six-readers; raw captures and counts: measure/2026-10-17-wikipedia-rust, measure/2026-10-19-wikipedia-rust-oss.

## Easy pages: 7 pages, the slowest 732 ms

All seven answered from the free plain request, $0.

> Seven pages fetched in parallel in one session, both keys set, 2026-10-15 17:26–17:45: example.com, a Wikipedia article, the Hacker News front page, a docs.rs crate page, an arXiv PDF, gov.uk, and a host that does not exist. All seven answered from the free plain request, $0.

| Page | Time |
| --- | --- |
| host that does not exist (`! no_host`) | 101 ms |
| example.com | 120 ms |
| gov.uk | 144 ms |
| docs.rs crate page | 210 ms |
| arXiv PDF | 442 ms |
| Wikipedia article | 483 ms |
| Hacker News front page | 732 ms |

Source: https://fatsearch.com/docs/benchmarks/#results-easy-pages (2026-10-15 17:26–17:45).

## A 17.4 MB binary that starts in 4.07–4.13 ms

- **17.4 MB** Release binary, symbols stripped (17,405,936 bytes · macOS arm64 · 0.2.0)
- **4.07–4.13 ms** Start-up, p50 (p90 4.30–4.57 ms · /usr/bin/true: 1.02–1.08 ms)
- **5.1 ms** A cached read (FATSEARCH 0.1.0)
- **8.09 s** Hard wall for one run (timeout:8, both keys)
- **10 ms** Refusing a 53 MB page (default 16 MiB cap · 27 MB RSS)
- **1.01×** Real tokens per estimated token (36 of the 42 reads within ±10%)

Source: https://fatsearch.com/docs/benchmarks/#binary-and-startup (2026-10-16).

## Where FATSEARCH loses

- Instagram posts (shells or "Post isn't available" pages)
- The YouTube results page on Bright Data (the results sit in a script block, see the outline's islands)
- Renders cut at 60 s (TikTok, the Joomla forum)
- Bright Data relaying a site's 502
- Reddit's captcha when it lands on both the plain request and the provider

- Slower on walled pages. On the 24 pages both answered, Firecrawl’s median was 4.9 s, FATSEARCH 0.2.0 23.7 s. FATSEARCH tries the free request first and moves to a paid provider only when a wall blocks it: it was faster on the 10 pages its free request read and slower on the 14 behind a wall. Measured on 0.2.0 with the cache off. Rivals were measured on 2026-10-16 and FATSEARCH 0.2.0 on 2026-10-21: different days, and pages and providers change in between.

- In the agent runs, FATSEARCH answers "pending" after 5 seconds while a slow page is still loading. The model often gave up there, which cost it more points than any page it failed to read.

By FATSEARCH · https://fatsearch.com/benchmarks/