What the checker does
Five requests go to your home page, each with a different user agent. For every response we record the status code, the final URL after redirects, the number of words on the page and any sign of a bot challenge. We recognise challenge and block pages from Cloudflare, Sucuri, Imperva, Akamai and AWS WAF, such as “Just a moment...” screens, CAPTCHA forms and 403 pages.
The five visitors we imitate:
- Desktop browser (Chrome)
- Phone (mobile Chrome)
- Googlebot
- Mediapartners-Google, the AdSense crawler
- AdsBot-Google
How to read the result
Compare the rows with each other rather than reading any one of them alone.
- All five return 200 with similar word counts: crawlers get the same page as people. That is what you want.
- Bots get 403, 429 or 503 while the browser gets 200: a firewall, rate limit or security plugin treats the bot user agent as a threat.
- Bots get 200 but far fewer words: usually a challenge page or an empty shell. Check which vendor we detected.
- Different final URLs: a redirect sends bots or phones somewhere else. Showing them different content can look like cloaking.
- Everything fails, the browser included: the site itself is down, or the domain or SSL certificate has a problem.
Why our Googlebot may be blocked when the real one is not
A good firewall does not trust the user agent string. It verifies the real Googlebot with a reverse DNS lookup: the IP address must resolve to a googlebot.com or google.com hostname that points back to the same IP. Our requests come from our own servers, so a well-configured WAF may block our imitation Googlebot while letting Google in. That is correct behaviour.
So treat a block here as a strong lead, not a verdict. Confirm it in Google Search Console: open URL Inspection, enter your home page and click Test live URL. If Google gets the page, your firewall verifies bots properly. If the live test fails too, the block is real.
Fixing the “Site down or unavailable” rejection
“Site down or unavailable” is a status shown in AdSense, not a policy. It means the AdSense crawler could not load your site when it tried. Google's checklist for unreachable sites asks whether the site is live, whether it needs a password, whether the SSL certificate is valid with HTTP redirecting to HTTPS, and whether robots.txt blocks the crawler.
Work through the likely causes below, then request a new review from the Sites page in AdSense:
- Cloudflare: open Security, Events and filter by user agent to see what was challenged. Review custom rules, rate limits and country blocks that apply to all traffic. Google crawls mostly from US IP addresses, so blocking the United States blocks Google.
- Wordfence, Sucuri and similar plugins: allow verified search engine crawlers and avoid blocking whole countries.
- Hosting: shared hosts sometimes throttle bots with 429 or 503. Ask support to allowlist the crawler IP ranges Google publishes.
- Password or maintenance mode: switch off coming-soon plugins and HTTP authentication before you apply.
- SSL: renew an expired certificate and redirect http:// to https:// with a 301.
Why access comes first
If the reviewer cannot load your page, nothing else in your application gets a chance. Access is the first thing to check, and once you know which layer blocks crawlers, the fix is usually quick.
The full Approvalens scan tests access across up to 5,000 pages and adds robots.txt, ads.txt, content and policy checks to the same report. Source: https://support.google.com/adsense/answer/12176698
FAQ
What does “Site down or unavailable” mean in AdSense?
AdSense could not reach your site when it tried to review it. Common causes are a firewall or bot challenge, a robots.txt block on Mediapartners-Google, a password or maintenance page, an invalid SSL certificate and a wrong URL entered in AdSense.
Does Cloudflare block Googlebot?
Not by design. Cloudflare recognises verified bots such as Googlebot. Blocks usually come from custom firewall rules, rate limits, country blocks or aggressive bot settings. Check Security, Events for requests with Google user agents.
Your tool says Googlebot is blocked but Search Console says it is fine. Which is right?
Search Console. Its live test comes from real Google IP addresses that your firewall can verify. We only imitate Googlebot's user agent, so a strict WAF may block us and still let Google in.
Is it cloaking if bots see a different page?
Under Google's spam policies, cloaking means showing search engines different content from users in order to manipulate rankings. A challenge page is usually an accident, not cloaking, but it still stops the review, so fix it.