Google’s crawl budget docs: more than the announced polish

On 22 July 2026 Google reworked its crawl budget documentation. The crawling docs changelog carries a two-sentence entry about it, and it promises language work: „to improve clarity, terminology consistency, and flow”. You read that and you move on.

I first read about it in Search Engine Journal. The article picks out three wording changes and draws a writing lesson from them. The angle convinced me, the selection made me curious: three examples show a pattern, they do not show the extent. So I went looking myself, put the previous day’s archived version next to today’s and compared both texts sentence by sentence.

There was more. Three examples turned into ten changed passages, and three of them carry new factual information. It does not appear in the changelog entry, and so far it is missing from my own crawl budget guide as well. More on that later.

The comparison is reproducible because both versions are public and both addresses appear further down. Which change counts as wording and which as substance is my classification. The exact phrasings sit next to it for you to read. Here is the overview, sorted by type of change.

First: hostload. The crawl capacity limit now officially carries a second name. The word is not entirely new on the page, because the previous day’s version already knew it as an error message: „you’re getting Hostload exceeded in the URL inspection tool”. What is new is that Google uses it as a name for the limit itself. More important than the name is the new definition anyway. Until now the limit read like a ceiling for simultaneous connections, now it limits „the total amount of time your server spends holding connections open for Google”. What is meant is the total connection time, calculated from number and duration. A server that responds slowly burns through the budget faster than a quick one on this reading, and that was previously a guess at best.

Second: 429 and 5xx are now spelled out. The old version spoke generally of „server errors” and the new one names „server errors (5xx HTTP status codes) or rate-limiting signals (such as HTTP 429)”. Anyone running an aggressive rate-limiting rule on the server or in the web application firewall is throttling their own crawl budget on this wording. That consequence was not readable before.

Third: the limit applies across crawlers. New in the text: „the crawl capacity limit is shared across all crawlers. This means that high demand from one crawler can reduce the capacity available for others.” Google gives no examples. My reading: if AdsBot or the shopping crawler fetch a lot, less is left for Googlebot. Of the three factual additions I consider this the one with the most practical consequences.

On top of that, a section became concrete. „Make your pages efficient to load” was already there the day before, word for word and without a single actionable step underneath. Now there are two: optimise response times and resources, and support HTTP caching via 304 Not Modified. The 304 is the real addition here, because that status code did not appear on the old page a single time. What follows from it in practice is in my guide on how to optimize your crawl budget. How crawling and indexing interlock in the first place, I explain in the basics of crawling and indexing.

The rewrite is announced, just not where I looked first. The July section of the Search Central changelog carries entries for 1, 7, 10, 14, 24 and 29 July. For 22 July there is nothing. As of 5 August 2026.

The reason is a move. On 18 December 2025 Google moved several crawling documents to a dedicated infrastructure site, „Optimize your crawl budget” among them. That site keeps its own change log at developers.google.com/crawling/docs/changelog, with its own RSS feed. That is where the 22 July entry sits, and where the move itself is documented too.

Then there is language. Call the same change log with ?hl=de and July 2026 is missing completely. The most recent German entry is from 4 May 2026, from a time when the rewrite did not exist yet. October 2025 is missing as well, so this is not a one-off slip. I fetched both versions at the same moment on 5 August 2026: eighteen entries in the English one, not a single July entry in the German one.

The documentation itself is not affected, and that difference matters. The German version of the crawl budget page carries the complete rewrite, including 429, 304 and the shared capacity limit. Only hostload is called „Hostlast” there, a word you will not meet anywhere else. What lags behind is the change log alone, which is precisely the page whose only job is to tell you about changes.

Anyone subscribed to only one of the two lists misses the other half. And even reading the right entry does not tell you everything: it promises clarity, consistent terminology and better flow. About 429, about 304 and about the shared capacity limit it says nothing. A changelog describes what the author intended. What actually ended up in the text is only in the text.

How a detail change like this plays out in crawling is something I wrote up when Googlebot’s 2 MB crawl limit changed. In practice this means two things: both feeds belong in your reader, and the two or three documents your most important articles stand on belong in a monitor of your own on top of that.

1. Replace vague promises with the actual condition

„Responds quickly for a while” sounds helpful and says nothing. How quickly? For how long? The new version names the condition instead: consistent responses, stable or improving response times, latency and Time to First Byte included. That is verifiable. Search your own texts for phrasings like „as a rule” or „relatively fast” and replace each one with the condition you can actually evidence. If you find no evidence, cut the statement. Making it vaguer is not an alternative.

2. Cut words that say the same thing twice

„Simultaneous parallel connections” is a pair that repeats itself, because simultaneous and parallel mean the same thing. Such pairs cost nothing to write and slow down every single reading. English classics are „free gift”, „advance planning” and „past history”. While writing they rarely stand out, while reading aloud they do immediately.

3. Describe what a system does, not what it wants

„Google’s crawlers might decide that it’s not worth the time” became „Google’s crawlers might not explore”. A crawler decides nothing and finds nothing beneath its effort, it fetches a URL or it does not. The anthropomorphism seems vivid and in truth shifts the focus from observable behaviour to an assumed intention. That is exactly where SEO texts start attributing motives to Google.

4. One thought per sentence

The old version packed the crawl budget definition and the hostname rule into one sentence, now there are three and the hostname rule stands on its own. The criterion is not sentence length, the new text has plenty of long sentences. What matters is whether a sentence carries one thought or two that pull against each other. A subordinate clause that takes back the main statement belongs on its own.

5. Use the same term for the same thing

Twice the old version carried a term that the page itself did not define: once „serving limit” for the capacity limit and once „Index Coverage report” for a tool that Search Console has long called „Page indexing”. Both are corrected now. The second case is the more instructive one, because here Google’s own documentation carried a name that Google’s own product had already dropped. If it happens there, it happens in your articles all the more. Search for tool names, menu paths and feature labels. They age quietly.

Before I write down rules here, a look at my own files is in order. My guide on crawl budget was last updated on 25 April 2026, a good three months before the rewrite. It says: „Crawl Rate Limit (crawling capacity): this is the technical limit for how many requests Googlebot can send to your server.”

That is two problems in one sentence. „Crawl Rate Limit” was already called „crawl capacity limit” in Google’s documentation before the rewrite, and the definition by number of requests is exactly the model the new version has just replaced. It is about connection time and requests are only one factor in it. About hostload, 429 and 304 my article so far says nothing at all.

That is uncomfortable to write and exactly the reason this article exists. An article rarely goes stale because someone wrote it badly, but because the source moved underneath it and I did not check. The guide gets its update in the next few days. Anyone who wants to check whether their own text still says anything of its own will find the self-test for interchangeable content here on the blog.

The usual reflex for „update the article” is a new section at the bottom and a fresh year in the title. Google did the opposite: vague promises became concrete, a word pair disappeared, an anthropomorphism gave way to a description of behaviour and two terms were unified. By my count the page grew from around 1,220 to around 1,360 words in the process. That is eleven percent more text for three additional facts, and not a single new heading.

The second lesson sits in the changelog entry. Google did say something, and promised clarity, consistent terminology and flow. That three facts had been added was not in there. A change notice tells you reliably that something happened. For the what, you are relying on the self-assessment of whoever wrote the change.