113 guides
Guides
Practical guides to AdSense approval, written from Google's own documentation and the patterns we see in scans.
Guides
Start here
New to AdSense? These three cover what Google asks for and what to do if you're turned down.
- AdSense Requirements in 2026: What Google Actually Asks ForThe real AdSense requirements for 2026, from age and ID checks to content, privacy policy and EEA consent, separated from the myths about posts and traffic.Read the guide
- AdSense Approval Checklist: What to Check Before You ApplyA practical AdSense pre-application checklist: crawler access, content depth, privacy policy, ads.txt, navigation and consent, each tied to Google's own docs.Read the guide
- AdSense Rejected? A Step-by-Step Plan Before You ReapplyWhy AdSense rejects sites (low value content, site down, policy, not ready), how to read your exact reason, fix it and request a new review the right way.Read the guide
0113 guides
Rejection messages
What each AdSense rejection message means and what to change before you reapply.
"You Have a Duplicate AdSense Account" Rejection: Find and Close the Old Account
AdSense said you have a duplicate AdSense account? How to find the other account, which one to close, and what to do after closing it.
3 min read · Nov 12, 2026"Issues With Your Traffic Sources" AdSense Rejection: What It Means and How to Fix It
AdSense said issues with your traffic sources? Which sources Google bans, how to audit your own traffic, and what to do before you confirm the fix.
5 min read · Nov 12, 2026"Unsupported Language" AdSense Rejection: Supported Languages and What to Do
AdSense said unsupported language? The 50 languages Google supports, common ones that aren't on the list, multilingual sites, and how to reapply.
3 min read · Nov 12, 2026"Earnings at Risk" and "ads.txt Not Found" in AdSense: What to Do
What the AdSense ads.txt warning means, how Not found differs from Unauthorized, why it stays after you upload the file, and how to check it.
8 min read · Oct 5, 2026"Getting Ready" or "Site Isn't Ready to Show Ads": What Your AdSense Status Means
What Getting ready, Needs attention and Ready mean on the AdSense Sites page, how long Getting ready normally lasts, and what to check when it seems stuck.
6 min read · Oct 5, 2026AdSense Site Verification Failed: Fix the Code Snippet, Meta Tag or ads.txt
AdSense couldn't verify your site? How the code snippet, meta tag and ads.txt methods work, why each fails, and how to check before you retry.
7 min read · Oct 5, 2026"Valuable Inventory: No Content" in AdSense: What It Means and How to Fix It
AdSense found screens without publisher content. Learn the two very different causes, how to see what the crawler actually received, and what to fix first.
8 min read · Oct 5, 2026"Valuable Inventory: Scraped Content" in AdSense: What Counts and How to Fix It
Why AdSense flags scraped content, where Google draws the line between copying, rewriting, translating and curating, and how to clean up.
7 min read · Oct 5, 2026"Valuable Inventory: Under Construction" in AdSense: How to Fix It
AdSense thinks your site isn't finished. Find the placeholders, empty sections, demo content and dead links behind it and clean them up.
6 min read · Oct 5, 2026"Policy Violations" AdSense Rejection: Find and Fix the Cause
AdSense says your site has policy violations but won't name the page. How to audit content, scripts, widgets and comments, fix them and reapply.
7 min read · Oct 5, 2026AdSense Rejected? A Step-by-Step Plan Before You Reapply
Why AdSense rejects sites (low value content, site down, policy, not ready), how to read your exact reason, fix it and request a new review the right way.
6 min read · Nov 4, 2026"Site Down or Unavailable" in AdSense: Causes and Fixes
Why AdSense says your site is down or unavailable when it loads fine for you, how to test it the way Google's crawler sees it, and how to fix each cause.
8 min read · Oct 5, 2026"Low Value Content" AdSense Rejection: What It Means and How to Fix It
Why AdSense says low value content, the page patterns behind it, how to audit your site in an afternoon, and what to fix before the next review.
11 min read · Oct 5, 2026
0243 guides
Fixes
Step-by-step fixes for the problems a scan finds, with platform-specific instructions.
"Screens With Replicated Content" AdSense Issue: How to Fix
Policy center says Google-served ads on screens with replicated content. What Google counts as replicated, how to find the pages, fix them and request a review.
7 min read · Nov 12, 2026AdSense and CLS: Stop Ads From Shifting Your Layout
Why ad slots push content down, how to reserve space with min-height per screen size, what anchor and vignette ads do, and how to measure CLS in the field.
9 min read · Oct 7, 2026AdSense Accidental Clicks: Ad Placement Rules That Get Flagged
Ad placements AdSense treats as encouraging accidental clicks: download buttons, navigation, games, mobile sticky ads, labels and wording, and how to fix them.
10 min read · Oct 7, 2026AdSense "Ad Serving Limited": What It Means, How Long It Lasts
What the AdSense ad serving limit means, the two reasons Google gives, how long it usually lasts, what to check and what makes it worse.
9 min read · Oct 7, 2026AdSense Ads Not Showing After Approval: What to Check, in Order
Approved but no ads? Check site status, ads.txt, the code, your ad blocker, unfilled units, consent and the Policy center, in the order that finds it fastest.
9 min read · Oct 7, 2026AdSense Copyrighted Material Violation: Find It and Fix It
Why AdSense flags copyrighted material, where it hides on small sites (images, lyrics, APKs, streams, copied posts), DMCA and Lumen notices, and how to fix it.
8 min read · Oct 7, 2026Back Button Hijacking: Google's Spam Policy and How to Fix It
Google made back button hijacking a spam violation from 15 June 2026. What counts, which scripts do it, how to find them in DevTools, and how to fix it.
8 min read · Oct 7, 2026cloud-init Status Error or Degraded: Read the Logs, Fix or Ignore
What cloud-init status error and degraded done mean, which logs to read, common causes (YAML, network, mirrors), when it's harmless, how to re-run safely.
7 min read · Oct 7, 2026Crawled - Currently Not Indexed: What It Means and What to Fix
What "Crawled - currently not indexed" means in Search Console, which URLs to ignore, which to merge or remove, and how to get real articles indexed.
9 min read · Oct 7, 2026Port 631 Open on Your Server? Remove CUPS or Bind It to Localhost
Why a VPS has CUPS and cups-browsed listening on port 631, what the 2024 cups-browsed vulnerabilities allowed, and how to remove it or keep it local and verify.
6 min read · Oct 7, 2026Discovered - Currently Not Indexed: Why Google Hasn't Crawled It
"Discovered - currently not indexed" means Google knows the URL but hasn't crawled it. Check server capacity, then internal links, sitemaps and URL clutter.
9 min read · Oct 7, 2026Fail2ban on Ubuntu for SSH and WordPress wp-login.php
Install fail2ban on Ubuntu, use jail.local, check the sshd jail, add a wp-login.php jail for nginx or Apache logs, unban yourself, and know what it can't stop.
7 min read · Oct 7, 2026"Screens Without Publisher-Content" AdSense Issue: How to Fix
Policy center says Google-served ads on screens without publisher-content. Which pages trigger it, how to fix them, and how to request a review.
10 min read · Oct 7, 2026Hacked Site: Safe Browsing Warning, Cleanup and AdSense Review
Signs your site was hacked, the cleanup order that stops reinfection, and the exact review steps in Search Console and the AdSense Policy center.
11 min read · Oct 7, 2026Is My Website Down? How to Tell and What to Check First
Down for everyone or just you? A fast order of checks: another network, DNS, TLS, HTTP status, Cloudflare 52x and 508 errors, and what to send your host.
12 min read · Oct 7, 2026Mixed Content After Moving to HTTPS: Find and Fix It
Why an https:// page still loads http:// files, what browsers upgrade or block, and how to fix it in WordPress, with CSP or Cloudflare without breaking things.
8 min read · Oct 7, 2026Facebook or WhatsApp Shows the Wrong Image? Fix og:image
Why Facebook, WhatsApp, X and LinkedIn show the wrong link preview image or none: og:image mistakes, blocked crawlers, file size, and how to clear each cache.
9 min read · Oct 7, 2026Secure SSH and UFW on a New Ubuntu VPS Without Locking Yourself Out
SSH keys, password login off (and the sshd_config.d trap on Ubuntu cloud images), then UFW in the right order. Plus why Docker ports ignore UFW.
8 min read · Oct 7, 2026Security Headers for Blogs With Ads: What to Set, What Breaks
HSTS, nosniff, X-Frame-Options, Referrer-Policy, Permissions-Policy and CSP for an AdSense blog: safe values, nginx/Apache/Cloudflare configs.
10 min read · Oct 7, 2026SSL Certificate Expired: Fix Certbot Renewal That Failed
Let's Encrypt certificate expired or certbot renew fails? Find the cause with a dry run, fix port 80, DNS, Cloudflare and webroot errors, then verify.
11 min read · Oct 7, 2026"No Space Left on Device" on a VPS: Find It and Free It Safely
Your VPS disk is full. How to find what filled it with df, du and lsof, and how to free space one item at a time: journals, logs, kernels, Docker, backups.
10 min read · Oct 7, 2026Broken Links and Soft 404s: Find Them, Fix Them, Stop Them
Find broken internal links, make missing pages return a real 404 instead of redirecting home, and fix them in WordPress, Blogger and Next.js.
9 min read · Oct 5, 2026Canonical Tag Problems: Wrong, Missing, Doubled or http://
Why canonical tags point to the home page, another domain or a dead URL, how to check a page in a minute, and the fixes for WordPress, Blogger and Next.js.
8 min read · Oct 5, 2026Duplicate Titles and Meta Descriptions: Find the Cause, Fix It
Why posts end up with the same title or meta description, how Google builds title links and snippets, and fixes for Yoast, Rank Math, Blogger and Next.js.
9 min read · Oct 5, 2026H1 and Heading Structure: What Matters and How to Fix It
What Google says about H1s and heading order, why themes break heading structure, and how to fix it in WordPress, Blogger and Next.js without the myths.
10 min read · Oct 5, 2026Noindex by Mistake: Find It in Tags, Headers and Settings
A stray noindex hides good pages from Google. Find it in the HTML, the HTTP headers or a CMS setting, remove it safely, and know when noindex is right.
9 min read · Oct 5, 2026Orphan Pages and Internal Links: A Repair Workflow
Find articles nothing links to, decide what to keep, add in-text links with clear anchors and build hub pages. Notes for WordPress, Blogger and Next.js.
9 min read · Oct 5, 2026Redirect Chains and Loops: Find Every Hop, Cut It to One
Why redirect chains and loops hurt AdSense review, how to trace each hop with curl or DevTools, and one-hop fixes for Apache, Nginx, Cloudflare, Next.js.
10 min read · Oct 5, 2026Slow Server Response (TTFB): Measure It, Then Fix the Cause
What time to first byte measures, why our number may differ from yours, how to test it with curl, and the cache, hosting and Next.js fixes to try first.
9 min read · Oct 5, 2026Structured Data Errors: Broken, Incomplete and Duplicate JSON-LD
Fix broken JSON-LD, half-filled Article markup and duplicate schema from two plugins, with Google's actual property rules and the tools to test it.
10 min read · Oct 5, 2026XML Sitemap Errors: Dead URLs, Bad Files and Fake lastmod
Fix sitemap problems that waste crawling: unreadable files, redirects and noindex pages in the list, tag-page bloat and lastmod dates that never change.
9 min read · Oct 5, 2026About and Contact Pages for AdSense: What They Need
Google only requires a privacy policy, but About and contact pages still shape AdSense review. What to put on each, with examples, and which pages are optional.
6 min read · Oct 7, 2026ads.txt for AdSense: Setup, Exact Format and Common Errors
How to create ads.txt for AdSense: the exact Google line, where the file must live, how to add it on WordPress or Blogger, and the errors that break it.
6 min read · Oct 7, 2026AdSense CMP Rules for EEA, UK and Swiss Visitors (TCF v2.3)
Who needs a Google-certified CMP for AdSense, what TCF v2.3 changed on 1 March 2026, and how to check your consent banner before ads start serving.
8 min read · Oct 7, 2026AdSense Privacy Policy Requirements, With Sample Wording
What Google requires in a privacy policy for AdSense sites: cookie and third-party disclosures, opt-out links, other ad networks and EEA consent, with examples.
7 min read · Oct 7, 2026Author Pages and E-E-A-T: Showing Who Writes Your Content
How bylines and author pages support E-E-A-T, what Google's own guidance asks for, how to mark up authors, and why invented experts backfire on AdSense sites.
6 min read · Oct 7, 2026Duplicate Content and AdSense: What Actually Gets You Rejected
The difference between harmless technical duplicates and the copied, scraped or spun content that AdSense rejects, with checks and fixes for both kinds.
6 min read · Oct 7, 2026Keyword Cannibalization: Find and Fix Competing Pages
When several pages on your site target the same query, they split signals and look like filler to reviewers. How to spot cannibalization and merge or refocus.
6 min read · Oct 7, 2026Mobile Usability and Speed for AdSense: What Matters
Speed isn't an AdSense approval rule, but a site that won't load, jumps around or buries content under mobile ads breaks rules that are. Fixes and targets.
6 min read · Oct 7, 2026Programmatic SEO and AdSense: Where Templates Cross the Line
Programmatic pages aren't banned; scaled template pages without real data are. How Google's policies apply and how to prepare a template-heavy site for review.
6 min read · Oct 7, 2026robots.txt and the AdSense Crawler: Rules That Matter
How Mediapartners-Google reads robots.txt, why User-agent: * does not block it, safe examples for WordPress and AI bots, and the mistakes that cost approvals.
5 min read · Oct 7, 2026Site Navigation for AdSense: Menus Reviewers Can Follow
AdSense wants sites that are easy to get around. What that means in practice: crawlable menus, no orphan pages, no broken links and no misleading buttons.
7 min read · Oct 7, 2026Thin Content: How to Find It and Fix It Before AdSense Review
What counts as thin content for Google and AdSense, why word count isn't the measure, and a practical improve, merge, redirect or delete process for weak pages.
7 min read · Oct 7, 2026
0319 guides
Questions
Straight answers to the questions people ask before they apply.
AdSense on Subdomains: Adding Sites and Where ads.txt Goes
Can you add a subdomain to AdSense? What the Sites page accepts, why ads.txt belongs on the root domain, when to use subdomain=, and free subdomains.
7 min read · Oct 7, 2026AdSense PIN Not Received? Thresholds, Timing and What to Do
Google posts the PIN once you earn $10, by untracked mail that takes about 3 weeks. When to request a new one, why it may not have been sent, the 4-month limit.
8 min read · Oct 7, 2026AdSense Seller Information and sellers.json: Transparent or Not
What AdSense seller information visibility does, where the setting is, transparent vs confidential, the business domain, and how sellers.json ties to ads.txt.
7 min read · Oct 7, 2026AdSense US Tax Info and W-8BEN for Non-US Publishers
What AdSense's United States tax info step asks if you live outside the US, which form you get, treaty claims, and what is withheld if you skip it.
8 min read · Oct 7, 2026AdSense Auto Ads vs Manual Ad Units: Which to Use, and When to Mix
How Auto ads place ads, what manual units give you, the settings that matter after the 2026 ad load change, and the layout shift and density trade-offs.
9 min read · Oct 7, 2026SPF, DKIM and DMARC Explained for Small Site Owners
Contact form or newsletter mail landing in spam? SPF, DKIM and DMARC in plain words, Gmail and Yahoo's sender rules, one SPF record, and a safe DMARC rollout.
9 min read · Oct 7, 2026Title Tag Length: Pixels, Not Characters (and What Google Says)
Google sets no title length limit and cuts title links to fit the screen. Why about 600 px matters more than 60 characters, and how to write titles that fit.
8 min read · Oct 7, 2026AdSense for a New Website: A Checklist From Domain to Application
Building a new site for AdSense? A phase-by-phase checklist from choosing the domain to Request review: what Google requires and what can wait.
7 min read · Oct 5, 2026Does AdSense Require Traffic? What Google Actually Checks
There's no minimum traffic for AdSense approval. What Google checks instead: where visitors come from, invalid clicks, and where low traffic slows you.
5 min read · Oct 5, 2026How Long Does AdSense Approval Take? Every Step, With Google's Timings
Google says the review takes a few days, sometimes 2-4 weeks. Every step from sign-up to first ads, why yours may take longer, what to do while you wait.
6 min read · Oct 5, 2026How Many Words per Post for AdSense? What Actually Counts
Google has no preferred word count. What "enough text" means for AdSense, why long posts still fail, and how to tell complete from padded.
6 min read · Oct 5, 2026AdSense Alternatives in 2026: Real Entry Requirements
Ezoic, Mediavine, Journey, Raptive, Monumetric, Media.net and more: what each ad network actually requires to join in 2026, and which fit a small or new site.
6 min read · Oct 7, 2026AdSense Approval Checklist: What to Check Before You Apply
A practical AdSense pre-application checklist: crawler access, content depth, privacy policy, ads.txt, navigation and consent, each tied to Google's own docs.
7 min read · Oct 7, 2026The AdSense Domain Age Myth: Is There a 6-Month Rule?
There is no current minimum domain age for AdSense. Where the 6-month rule came from, the real 6-month deadline in your account, and why young sites still fail.
6 min read · Oct 7, 2026AdSense Requirements in 2026: What Google Actually Asks For
The real AdSense requirements for 2026, from age and ID checks to content, privacy policy and EEA consent, separated from the myths about posts and traffic.
6 min read · Oct 7, 2026Can You Get AdSense With AI-Generated Content? What Google Allows
AdSense doesn't ban AI-written content; it bans unreviewed, mass-produced pages. What Google says, what gets rejected, and a workflow that works.
8 min read · Oct 5, 2026Expired Domains and AdSense: Risks to Check Before You Buy
Can a dropped or auctioned domain get AdSense approval? What Google's expired domain abuse policy says, and the history checks to run before you build on one.
6 min read · Oct 7, 2026Google Page Quality Explained: Lowest to Highest Ratings
How Google's Search Quality Rater Guidelines grade pages from Lowest to Highest, what raters look at, and what the scale means for a site applying for AdSense.
6 min read · Oct 7, 2026How Many Posts Do You Need for AdSense Approval?
Google sets no minimum post count. Where the 20 to 30 posts advice comes from, what reviewers need instead, and how to find your own number.
6 min read · Oct 5, 2026
0413 guides
Platforms
Setup notes and traps for the platform your site runs on.
How to Add ads.txt in Next.js (App Router and Pages Router)
Add ads.txt to a Next.js site: a static file in /public or a Route Handler, a proxy matcher that leaves it alone, and how to check it on the live domain.
5 min read · Oct 5, 2026How to Add ads.txt on Blogger (and When You Don't Need To)
When Blogger needs a custom ads.txt, the steps in Settings → Monetization, what to paste, and where Google looks on blogspot and custom domains.
4 min read · Oct 5, 2026ads.txt on Cloudflare Pages and Workers: Setup and the SPA Trap
How to serve ads.txt from Cloudflare Pages or Workers static assets, why a missing file can return your home page with status 200, and how to test it.
5 min read · Oct 5, 2026How to Add ads.txt on WordPress: File, Plugin or WordPress.com
Three ways to add ads.txt to WordPress, why a physical file silently overrides plugins, the subfolder trap, WordPress.com, and how to test it.
5 min read · Oct 5, 2026AdSense on Next.js and React Sites: Rendering, Scripts, ads.txt
Getting a Next.js or React site approved for AdSense: server-rendered content, the AdSense script in the App Router, ads.txt in /public, proxy traps.
9 min read · Oct 5, 2026AdSense on Shopify: theme.liquid, ads.txt Redirect and Store Content
AdSense on Shopify: the code in theme.liquid, an ads.txt workaround with Files and URL redirects, private mode, and why product pages aren't enough.
7 min read · Oct 5, 2026AdSense on Squarespace: Plans, Code Injection and the ads.txt Gap
Can Squarespace run AdSense? Which plans allow Code Injection, why trial sites can't be reviewed, the missing ads.txt option, crawler settings and consent.
6 min read · Oct 5, 2026AdSense on Wix: Plan, App, ads.txt and What Gets Rejected
Getting a Wix site approved for AdSense: the plan and domain you need, the Wix AdSense app, ads.txt in Marketing Integrations, image-heavy pages.
7 min read · Oct 5, 2026Fix www vs non-www in Cloudflare: One 301 Redirect Rule
Redirect www to the root domain (or back) in Cloudflare: the proxied DNS record, one Redirect Rule, Bulk Redirects for Pages, and a 4-URL test.
5 min read · Oct 5, 2026Redirect www to non-www (or Back) in cPanel: Two Safe Ways
Fix www vs non-www on cPanel: the Redirects screen with the right www option, a tested .htaccess rule that also forces HTTPS, SSL for both names.
5 min read · Oct 5, 2026AdSense on Blogger: Earnings Tab, Settings and Common Rejections
AdSense on Blogger: signing up from the Earnings tab, the settings to check first, custom domains, ads.txt and the template habits behind rejections.
8 min read · Oct 5, 2026AdSense on WordPress: Setup, Plugins and Rejection Traps
Getting WordPress ready for AdSense: where the code goes, Site Kit, WordPress.com plans, settings that hide your site, and thin auto-made pages.
7 min read · Oct 5, 2026Is Cloudflare Blocking AdSense? How to Check and Fix It
How Cloudflare bot protection, challenges, country rules and rate limits can block the AdSense crawler, how to confirm it in Security Events and what to fix.
6 min read · Oct 7, 2026
0511 guides
Niches
Topics AdSense restricts or refuses, and what a focused site looks like.
Affiliate Sites and AdSense: Getting Approved With Affiliate Links
AdSense allows affiliate links but rejects thin affiliate pages: what Google calls thin affiliation, how to disclose links, what makes a review worth reading.
8 min read · Oct 7, 2026YMYL Sites and AdSense: Health, Finance and Legal Topics
What YMYL means in Google's rater guidelines, which AdSense policies touch health and money sites, and what to show so a reviewer can trust your pages.
11 min read · Oct 7, 2026AdSense for APK Download Sites: The Policies Every Page Touches
Why APK and mod APK sites rarely fit AdSense: copyright and DRM bypass, malware, copied store listings, download buttons by ads. And what works.
5 min read · Oct 5, 2026Can a Calculator or Online Tool Site Get AdSense? What Google Checks
AdSense for calculator and online tool sites: the policies that apply, the text each tool page needs, safe ad placement, and a real example.
7 min read · Oct 5, 2026Can a Lyrics Site Get AdSense? Copyright, Copies and Alternatives
Why lyrics sites struggle with AdSense: lyrics are copyrighted, every site has the same text, profanity limits ads. What a song site needs instead.
5 min read · Oct 5, 2026AdSense for News Sites: Agency Copy, Feeds, Bylines and Policies
What gets news sites rejected by AdSense: republished agency copy and scraped feeds, missing bylines and ownership, clickbait, graphic images.
5 min read · Oct 5, 2026AdSense for Recipe Sites: Original Recipes, Photos and Ad Layout
What AdSense looks for on a food blog: why reworded recipes are replicated content, your own photos, recipe schema, health claims, ad layout.
5 min read · Oct 5, 2026AdSense for Wallpaper Sites: Why Image Galleries Get Rejected
Wallpaper and image sites vs AdSense: pages that are mostly images, collected and copyrighted pictures, suggestive images, download buttons by ads.
5 min read · Oct 5, 2026AdSense Prohibited Content: What Google Won't Monetise
The content categories Google Publisher Policies prohibit, how they differ from restricted content, and how a single section can sink an AdSense review.
8 min read · Oct 7, 2026AdSense Restricted Content: Allowed, but With Fewer Ads
Restricted isn't prohibited. What Google Publisher Restrictions cover, from alcohol and gambling to weapons, and how a restriction label affects ads and review.
6 min read · Oct 7, 2026Niche Focus for AdSense: Does Your Site Need One Topic?
Google has no single-niche rule for AdSense, but it asks if a site has a primary purpose. How to judge focus, find off-topic pages and fix a scattered site.
6 min read · Oct 7, 2026
0614 guides
AI visibility
Whether ChatGPT, Claude, Perplexity and other AI crawlers can reach your pages, and how to allow or block them on purpose.
How to Allow AI Crawlers in robots.txt (ChatGPT, Claude, Perplexity)
The exact robots.txt tokens for OpenAI, Anthropic, Perplexity, Google and Apple, how to allow AI search but block training, and mistakes that hide you.
6 min read · Oct 7, 2026Applebot-Extended Explained: Opting Out of Apple AI Training
Applebot-Extended controls whether Apple trains its models on your content; it doesn't crawl or affect Siri and Spotlight search. Usage and nosnippet.
4 min read · Oct 7, 2026Can ChatGPT See My Website? A 5-Minute Check
Whether ChatGPT can find and read your site depends on OAI-SearchBot, ChatGPT-User, your firewall and your HTML. How to check each one and fix what blocks it.
5 min read · Oct 7, 2026ClaudeBot, Claude-User and Claude-SearchBot Explained
Anthropic runs three bots: one for training, one for Claude's search, one for user requests. What each does, what blocking costs and robots.txt examples.
5 min read · Oct 7, 2026Is Cloudflare Blocking AI Bots on Your Site? How to Check
Cloudflare's AI bot policies, AI Crawl Control and managed robots.txt can block ChatGPT, Claude and Perplexity. Where each lives and how to allow search.
7 min read · Oct 7, 2026AI Bots Blocked by Your Host, Firewall or Plugin? Where to Look
robots.txt allows AI crawlers but they still get 403s? Where to find blocks in WordPress plugins, nginx, Apache, Vercel and Netlify, and how to fix them.
6 min read · Oct 7, 2026Google-Extended: Does Blocking It Affect Google Search?
Google-Extended controls Gemini training and grounding, not Search. What it covers, what it doesn't (AI Overviews) and how to block it safely.
5 min read · Oct 7, 2026GPTBot vs OAI-SearchBot vs ChatGPT-User: Which to Allow
OpenAI runs separate bots for training, ChatGPT search, user requests and ads. What each does, which obey robots.txt, and three robots.txt setups.
5 min read · Oct 7, 2026Can AI Crawlers Read JavaScript? Making Content Visible to ChatGPT
If your text only appears after JavaScript runs, some AI crawlers may see an empty page. What is documented, how to test your HTML and how to fix it.
5 min read · Oct 7, 2026The noai Meta Tag: Does It Stop AI Training? What Actually Works
Where noai and noimageai came from, which crawlers document support for them (none of the big ones), and the opt-outs OpenAI, Google, Apple and Bing honour.
5 min read · Oct 7, 2026PerplexityBot Not Crawling Your Site? Causes and Fixes
Why Perplexity may not cite your pages: robots.txt, WAF and CDN blocks, IP allowlists and JavaScript-only content, and how to check each one.
5 min read · Oct 7, 2026robots.txt Group Precedence: Named AI Bots vs User-agent: *
Why a GPTBot group makes GPTBot ignore your * rules, how duplicate groups merge, Applebot's Googlebot fallback and other robots.txt surprises for AI bots.
5 min read · Oct 7, 2026Should I Block AI Training Bots? Trade-offs for Publishers
Blocking GPTBot, ClaudeBot, Google-Extended and other training bots: what it changes, what it doesn't, how it relates to AI search, and AdSense.
5 min read · Oct 7, 2026What Is llms.txt? Format, Examples and Whether You Need One
llms.txt is a proposed Markdown file that gives AI agents a short map of your site. The format, a working example, common errors and what it does not do.
6 min read · Oct 7, 2026
Free scan
Check your own site
The free scan reads the first 50 pages and shows your score and every problem it finds.