Googlebot’s 2 MB Crawl Limit: Timeline, Real-World Data, and What to Check Now

In February 2026, Google revised its crawler documentation three times within just nine days—adding a bit more confusion with each version. The headline that made the rounds in the SEO scene: Googlebot now only reads the first 2 MB of an HTML file instead of the previously documented 15 MB. Mathematically, that’s an 86.7% drop, if you want to frame it dramatically. And that is exactly what some people did on LinkedIn.

Chris Long (Nectiv) made the topic go viral on LinkedIn. Industry voices like Barry Schwartz, Jamie Indigo, and Steve Toth joined the discussion. Between fearmongering and downplaying, the truth—as always—lies somewhere in the middle. The change simply does not affect the vast majority of websites. But those who are affected won’t even notice: Google does not issue any warning when content is truncated at the 2 MB limit.

In this article, you will find the complete timeline of all three documentation changes, the core distinction between fetching and indexing, our own measurement data, five concrete testing methods with step-by-step instructions, and a sober assessment of what this means for your SEO and AI visibility.

The timeline is crucial because Google adjusted the documentation not once, but three times in quick succession. And the final change puts the first one into perspective.

Update 1: February 3, 2026

Google updates the official Googlebot documentation and the general crawler documentation. The key new passage on the Googlebot page: “When crawling for Google Search, Googlebot crawls the first 2MB of a supported file type, and the first 64MB of a PDF file.” At the same time, it clarifies that CSS and JavaScript files are loaded separately and are each subject to their own 2 MB limit. The limit applies to uncompressed data.

As Barry Schwartz reported on Search Engine Roundtable, Google explained: While restructuring the documentation, they documented Googlebot-specific limits more precisely. The information had previously been placed in a less logical location.

Update 2: Approx. February 6–8, 2026

John Mueller (Google) reacts to the community’s uncertainty with clarifications on Bluesky and Reddit. His main statements: It is a documentation clarification, not a change in behavior. The 2 MB is specific to Googlebot and web search. He recommends Dave Smart’s Tame-the-Bots tool for testing and considers the issue to be “extremely rare” in practice.

Update 3: February 11, 2026 – The Crucial Softening

Google adjusts the general crawler documentation a third time. The new wording now reads: “a Google crawler like Googlebot may have a smaller size limit (for example, 2MB).”

The “for example” is remarkable. It no longer positions the 2 MB as an absolute limit, but as an example value for a possible limit. The previous version had formulated the 2 MB much more concretely. Whether Google is intentionally leaving itself some wiggle room here or whether the wordings come from different documentation teams remains an open question.

Most reports have not cleanly carved out this point, even though it is central to understanding the issue. Google’s crawling infrastructure operates in two separate phases—and a different limit applies to each phase.

Phase 1 – Fetching: Google’s crawlers download up to 15 MB of a file from your server. This is the general limit for the entire Google infrastructure. Your server delivers the file, Google saves it. Nothing has changed regarding this limit.

Phase 2 – Indexing (Processing for Search): For Google Web Search, Googlebot only processes the first 2 MB of the downloaded HTML. Anything beyond that is ignored for ranking purposes. It exists on the server, but Google doesn’t use it for the search results.

For example, if your HTML file is 5 MB, Google may download it completely—but will only evaluate the first 2 MB for ranking. The rest is entirely invisible to Web Search.

Dave Smart (Tame the Bots) frames it pragmatically: “At the risk of overselling how much of a real world issue this is – it really isn’t for 99.99% of sites I’d imagine.”

The various numbers have caused considerable confusion in the SEO community. John Mueller clarified on Bluesky why this is the case: Google runs many different crawlers, and Googlebot is just one of them. His comment: “Google has a lot of crawlers, which is why we split it. It’s extremely rare that sites run into issues in this regard, 2 MB of HTML is quite a bit.”

Mueller emphasized on Reddit: “None of these recently changed, we just wanted to document them in more detail.” And as the quickest practical check, he recommends an approach without any tools: Simply Google a unique text quote from far down on a page. If Google finds it, everything is indexed.

How big is the limit actually in practice? According to the HTTP Archive Web Almanac 2025, the median HTML file size on mobile pages is a mere 33 KB. Even at the 90th percentile, it’s only around 151 KB—as John Mueller himself confirmed on Reddit: “This means 90% of the pages out there have less than 151kb HTML.”

To put that into perspective: 2 MB equals about 2 million characters. That easily fills a 400-page novel—squeezed onto a single web page. To reach the limit, a page would have to be about 60 times larger than average.

The analysis by Seobility, based on 44.5 million crawled pages, confirms this picture: Only 0.82% of all analyzed pages exceed 2 MB.

SEO consultant Nikki Pilkington formulated one of the sharpest warnings about this—and she makes a point that every website owner should remember: “If someone contacts you about your website’s HTML file size being a problem, ask them what your current HTML file size is.” Only the HTML, not the total page size including images and scripts. Anyone who can’t answer that is selling fear instead of help.

Instead of just talking about international statistics, I actually measured the uncompressed HTML file size of German websites—including the pages of seo-kreativ.de.

seo-kreativ.de Self-Test

Our longest article—with several tables, code examples, and detailed explanations—is 327 KB uncompressed. That’s 16% of the 2 MB limit. Even tripling the length would still put it in the safe zone. The compression rate consistently sits at around 23–25% of the original size, which shows why you must always check the uncompressed values.

Comparison: How Big Are Well-Known Websites Really?

For comparison, I measured the uncompressed HTML size of some of the world’s largest websites in incognito mode using Chrome DevTools. The result: Not a single page exceeds the 2 MB limit.

Even YouTube—one of the most complex web applications out there—comes in at around 1 MB on search result pages, remaining well below the limit. Netflix, as an SPA, delivers a lean shell of just about 500 KB; the actual content is loaded dynamically via JavaScript. And seo-kreativ.de? Even our longest article uses just 16% of the available budget.

This confirms John Mueller’s assessment: The 2 MB limit is simply irrelevant for the vast majority of websites. Those who are affected usually have a fundamental problem with inline code or bloated HTML—not with too much content.

Although the number of affected pages is small, the mechanism deserves attention. The team at Spotibo conducted real indexing tests, gaining insights that go beyond the sheer documentation.

What the Tests Show

Spotibo created test pages with 3 MB and 16 MB of HTML and submitted them to Google for indexing. The 3 MB file was cut off after exactly 2 MB. In the indexed source code, the text breaks off mid-word, followed only by the closing </html> tag. Everything beyond the 2 MB boundary was silently ignored.

The tricky part: The status in Google Search Console still showed “URL is on Google” and “Page is indexed”—completely without any warning or error message. There was no indication whatsoever that content had been truncated.

The 16 MB file wasn’t even processed by Google. When attempting to trigger indexing, an error message appeared.

Collateral Damage: When Structured Data Breaks Mid-Code

One aspect that has received little attention in coverage so far: Truncation doesn’t just affect body text, but also machine-readable data. And this is where it gets really treacherous.

Many websites place their structured data (JSON-LD) at the end of the HTML document—often right before the closing </body> tag. This is technically correct and a common best practice. But on a page that exceeds the 2 MB limit, exactly this area gets cut off.

The problem: JSON-LD is strictly syntactic. If Google chops the code mid-way, closing brackets, quotation marks, or commas will be missing. This invalidates the entire structured data markup—not just the truncated part, but the entire block. Imagine an FAQ schema with 30 questions where Google stops reading after question 22: The missing closing bracket makes the entire JSON invalid, and Google discards all 30 FAQ entries, not just the last eight.

The same applies to Product schema with many variants, review aggregations, or BreadcrumbList markup. Every time the JSON-LD block spans the 2 MB boundary, you risk completely losing your Rich Results—without Search Console reporting a schema error, because the URL Inspection Tool sees the complete code.

Who Can Actually Be Affected

It’s not just poorly maintained hobby sites. Complex websites with extensive filters, tracking setups, or dynamic content can hit the limit. An overview of typical risk groups:

E-commerce sites with client-side filtering are at the top of the list. Shops on Magento or Shopify that write large JSON objects for product filters directly into the HTML are a typical risk group—especially those with many product variants.

Single-Page Applications (SPAs) with inline bundles form the second category. If JavaScript and CSS are not outsourced to external files but end up completely within the HTML document, it adds up quickly.

Base64-encoded images in HTML are an often underestimated factor. Embedding images directly into the markup via Data URLs instead of referencing them as external files massively bloats the HTML document—a single high-res image can account for hundreds of kilobytes.

Page builders like Elementor or WPBakery generate significant additional HTML code for complex layouts, which accumulates further with every element.

Infinite scroll implementations that load more and more content into the DOM as you scroll can dynamically cross the line—even if the initial HTML code is below the limit.

There are five reliable methods to check the HTML file size of your pages. Here they are with concrete instructions—because the details matter.

Method 1: Chrome DevTools (My preferred method)

Open the developer tools with F12. Switch to the “Network” tab. Completely reload the page (Ctrl+Shift+R). Filter by “Doc”—this will only show the HTML document.

Click on your page’s entry and look at the “Size” column. You will see two values there: the compressed transfer size and the uncompressed resource size. Only the uncompressed size (resource) counts toward the 2 MB limit. The GZIP/Brotli-compressed transfer size is irrelevant for the assessment—it helps save transfer time, but does not protect against the indexing limit.

Method 2: The Mueller Test (Simplest method)

John Mueller uses this approach himself and recommended it on Bluesky. The idea: Search Google for a unique text quote located as far down your page as possible.

Open your page in the browser, scroll all the way to the bottom, and copy a unique sentence—ideally something that only appears on this one page. Then search on Google: site:yourdomain.com "your exact quote from way down". If Google shows the page as a result, your content is fully indexed. If Google doesn’t find the text, even though the page is generally indexed, it might have been truncated at the 2 MB limit.

The advantage: This test checks exactly what you actually care about—is the content indexed? No detours via file size calculations.

Method 3: Tame the Bots (Recommended by Mueller)

Dave Smart updated his Fetch and Render Tool on February 6, adding a “Cap text to 2MB” checkbox. John Mueller recommended the tool on Bluesky that very same day: “If you’re curious about the 2MB Googlebot HTML fetch limit, here’s a way to check.”

How to use it: Enter the URL, activate the “Cap text to 2MB” checkbox, select the Googlebot Mobile user agent, and start the fetch. The tool shows you what your page looks like when truncated at 2 MB—including rendered HTML, initial source code, and a screenshot.

Method 4: Screaming Frog (For bulk checks)

For a site-wide check, Screaming Frog is the tool of choice. Pay attention to the correct column: “Size” shows the uncompressed file size—this is the relevant metric. “Transferred”, on the other hand, shows the compressed size and is not indicative of the limit. Sort by “Size” descending and mark anything over 1 MB for closer inspection. You should treat pages over 1.5 MB as a warning sign.

Method 5: Compare Response HTML vs. Rendered HTML

For a deeper analysis—especially relevant for AI visibility—it pays to compare Response HTML (what your server delivers directly) with Rendered HTML (what is in the DOM after JavaScript execution). If there is a large difference between the two, your page may be perceived quite differently by systems that do not render JavaScript. Since most AI crawlers execute JavaScript only to a limited extent or not at all, this comparison reveals your potential AI visibility gap.

Bonus: The Truncation Cross-Check in 4 Steps

You don’t just want to know if your HTML is large—you want to know if Google actually indexed everything. This systematic check combines file size and indexing:

Step 1 – Measure size: Check the uncompressed HTML size via Chrome DevTools or Screaming Frog (Method 1 or 4). If it’s under 500 KB, you can stop here—you are safe.

Step 2 – Set anchors: Identify a unique text element as far down in the source code as possible. This can be a sentence in the last paragraph, an FAQ answer at the end, or an anchor point in the footer content. Copy this text exactly.

Step 3 – Perform Mueller test: Search Google: site:yourdomain.com "your exact text from the bottom of the page". If Google finds it, the page is fully indexed. If Google doesn’t find it, check in the next step whether it’s due to truncation.

Step 4 – Cross-check with text from the top: Repeat the search with a unique text from the top of the page. If Google finds the top text but not the bottom text, you have a strong indicator of truncation. If Google doesn’t find the top text either, the page might not be indexed at all—in which case the problem lies elsewhere.

This cross-check takes five minutes and is more reliable than any pure size measurement because it tests the actual indexing outcome.

If you regularly monitor your Crawl Budget anyway, add HTML file size and the Response vs. Rendered comparison as permanent checkpoints in your technical SEO audit.

Advanced: Proving Truncation on Your Own Domain

Most sources on this topic cite Spotibo’s test findings—and they are solid. But as a technical SEO, you shouldn’t have to rely on third-party sources. You can reproduce the truncation on your own domain. Here is how:

Create a test page on your domain (e.g., /truncation-test/) and fill it with 2.5 MB of generated Lorem Ipsum text. Place a unique marker string at the beginning, after exactly 1 MB, after exactly 2 MB, and at the end—for example MARKER-START-001, MARKER-1MB-002, MARKER-2MB-003, and MARKER-END-004. Submit the URL for indexing via Search Console and wait 3–7 days.

Then check all four markers via site:yourdomain.com "MARKER-START-001". The hypothesis: Google finds the markers before and at 2 MB, but not the marker after it and at the end. With this, you have empirical proof on your own domain—your own data instead of third-party sources.

If you have identified pages over or near 2 MB, the measures are manageable in most cases.

Your Checklist

The 2 MB Budget: Where Are Your Bytes Going?

It helps to look at the 2 MB not as an abstract limit, but as a budget that various components of your HTML consume. Based on our own measurements, a typical distribution pattern emerges:

For a lean website like seo-kreativ.de (327 KB largest article), about 55% is taken up by inline CSS and inline JS. For JavaScript-heavy SPAs, this share can be significantly higher, though the actual initial HTML size varies greatly depending on the rendering strategy. This shows: The most effective step is almost always the same—move inline code to external files. This shrinks the HTML budget to the actual content, and even for long articles, that rarely exceeds 200–300 KB.

A Sensible Target Value

Ideally, keep your HTML files under 500 KB. Not because Google demands it, but because it provides a comfortable buffer, improves performance, and gives you headroom for growing content. Set up your monitoring: Warning level at 1.5 MB, critical at 2 MB. This way, you’ll notice creeping bloat before it becomes a problem.

Order Matters: Packing Your HTML Source Code Like an Emergency Kit

If Google cuts off at 2 MB, the total size alone isn’t what matters—it’s what is in those first 2 MB. Think of your HTML source code like an emergency go-bag: Whatever goes in first is guaranteed to survive. Whatever is at the bottom might be lost in an emergency.

A well-thought-out source code order looks like this:

Priority 1 – Must be in the first 100 KB: <head> with meta tags and canonical, JSON-LD (structured data), <h1> and introduction, critical CSS (if not external).

Priority 2 – Belongs in the first half (up to ~1 MB): Main content with all H2/H3 headings, the most important internal links, above-the-fold elements.

Priority 3 – Can go in the second half (1–2 MB): Secondary content (related posts, sidebar), comments, less important internal links, non-critical UI elements.

Must never endanger the limit: Inline CSS and inline JS (belongs in external files), Base64 images (should be referenced as external URLs), page builder wrapper divs (reduce or optimize markup).

The crucial difference from a generic “make page faster” recommendation: This isn’t about load times, but indexing safety. Even if your page doesn’t hit the limit, the correct source code order ensures that in the event of an unexpected size explosion (new plugin, additional tracking, content update), the ranking-relevant elements stay protected.

Besides the classic Googlebot, there is another reason to keep HTML lean: AI crawlers and LLM-based search systems. ChatGPT, Perplexity, Google AI Overviews, and other systems rely on efficiently fetching and processing web content.

Many of these systems render JavaScript much worse than Googlebot—or not at all. They prefer working with raw HTML. This means: AI crawlers are typically more sensitive to bloated HTML than Googlebot. If your content only becomes visible through JavaScript execution, these systems might only see a fraction of your content.

Therefore, comparing Response HTML (what the server delivers directly) with Rendered HTML (what is created after JavaScript execution) is becoming an increasingly important audit criterion. It reveals how big your AI visibility gap actually is.

In my assessment, Google’s documentation update fits into a larger picture: In a world where AI systems use the web as a data source, technical efficiency is shifting from a nice-to-have to a must-have. This applies not only to classic Google Search but to overall visibility in SEO, AIO, GEO, and LLMO.

Those who keep their HTML code lean are making things easier not just for Googlebot—but for all systems that process web content. For an overview of how the entire Google algorithm behaves from crawling to ranking, check out our comprehensive guide.

Was the crawl limit really lowered from 15 MB to 2 MB?

It’s more complicated than the headlines suggest. There are now two clearly documented phases: Fetching (downloading) with the unchanged 15 MB limit, and Indexing (processing for search) with the 2 MB limit for Googlebot. Google calls it a documentation clarification, not a change in behavior. The update from February 11, 2026, additionally frames the 2 MB as an example (“for example, 2MB”), which further softens its strictness.

Does the 2 MB limit apply to the compressed or uncompressed file size?

To the uncompressed file size. This is a common misunderstanding: Even if your server transmits HTML compressed via GZIP or Brotli, the original size of the file is what counts for the limit. For comparison: The seo-kreativ.de homepage transfers at only 67 KB, but is 286 KB uncompressed. In Chrome DevTools, you can spot the difference by looking at “transferred” (compressed) vs. “resource” (uncompressed).

Can I rely on the URL Inspection Tool?

No—and this is one of the most important points of this topic. The URL Inspection Tool uses the “Google-InspectionTool” crawler with the 15 MB fetching limit. It shows you the full source code, even if Googlebot truncated it at 2 MB during indexing. Spotibo proved this with concrete test files: Search Console displayed “Page is indexed” without warning, even though the content was cut at 2 MB.

What happens to content beyond the 2 MB?

Google cuts off the HTML code after 2 MB and only indexes the part before it. This happens without any warning. Text, internal links, and structured data located past the 2 MB mark are ignored for ranking purposes—but continue to exist on the server and are accessible to other Google crawlers (e.g., Image, Video) with the 15 MB limit.

Do images and videos count towards the 2 MB limit?

Externally referenced images and videos do not count—they are loaded separately and have their own limits. External CSS and JavaScript files also each have their own 2 MB budget. But beware: Base64-encoded images (Data URLs), inline styles, and inline scripts do count towards the HTML budget and can inflate it considerably. Offloading inline code to external files is therefore the most effective single optimization step.

The sober bottom line: The 2 MB limit is irrelevant for over 99% of all websites. But the mechanism behind it—silent truncation without warning, misleading URL Inspection Tool—deserves significantly more attention than the raw number itself.

What surprised me the most while researching this article wasn’t the limit itself, but the fact that Google issues no warning in Search Console when content gets cut off. The page sits there labeled “indexed”, while at the end of the day, content, links, and structured data are quietly ignored.

The three documentation changes in nine days show that Google itself struggled with the messaging. The softening to “for example, 2MB” on February 11 suggests that the boundary is less rigid than initially assumed—which, however, doesn’t change the fact that in Spotibo’s tests, content was truncated at exactly 2 MB.

For practice: Test your most important pages with the Mueller test or Tame the Bots. Keep your HTML under 500 KB where possible. Strictly move inline code into external files. And adopt the comparison of Response HTML vs. Rendered HTML as a new audit criterion—because this comparison will become increasingly crucial as AI visibility grows.