Check that your CDN and firewall allow search crawlers

Allowing crawlers in robots.txt isn't enough if your CDN, firewall or bot protection blocks them first. Google's guidance for AI features says to confirm your hosting and CDN allow crawling. Check bot settings for Googlebot, Bingbot and AI search crawlers, and test key pages with URL Inspection and your server logs.Google Search CentralTactic credited to Google Search Central. See every tactic from them.