How it works
Check access and verify the crawler using the official method when needed, not its User-Agent alone.
Example
Example: A server log entry is checked against official IP verification rules.
Common mistakes
Any client can spoof the Googlebot name in a request.
How can you check this in practice?
Verify the IP against officially published ranges or reverse DNS followed by forward DNS resolving to the same IP. User-Agent alone does not authenticate identity. Inspect permissions for the specific crawler and actual host.
Distinctions and limitations
Search Googlebot is not every Google system. Different crawlers and product tokens serve different roles. Allowed robots.txt access does not prove a request occurred, and an actual request does not establish indexing.
Official sources
Editorial responsibility: Martin Zhelyazkov · Founder of Z-Consult
Updated: 05/11/2026
For a correction or question: office@z-consult.bg