Is your Shopify store blocking the AI crawlers? A checklist

Check 1: Find your robots.txt file and read it

Robots.txt is a small text file on your site. It tells bots which pages they may visit. Every Shopify store has one. Type your store's address and add /robots.txt at the end, for example yourstore.com/robots.txt.

You will see lines like "User-agent" (which bot the rule is for) and "Disallow" (pages that bot should skip). Shopify's default file already blocks pages like the cart and checkout. That is normal and fine. Those pages are not meant to show up in answers anyway.

Good looks like this: no rule that tells the important AI bots to stay away from your whole site, your products or your collections. A line like "Disallow: /" under one of those bots is the big warning sign.

• Open yourstore.com/robots.txt in a browser
• Look for "User-agent" lines that name AI bots
• Watch for "Disallow: /" or "Disallow: /products" under them

Check 2: Make sure the bots that decide answers can get in

Not every AI bot does the same job. Three of them decide whether your store can show up in answers at all. Each company describes these on its own crawler page.

Googlebot controls Google Search, and that includes AI Overviews and AI Mode. OAI-SearchBot decides whether a site shows in ChatGPT search. PerplexityBot decides whether a site shows in Perplexity's answers. Blocking any of these is serious. Google said in July 2026 that it sends "billions of clicks to websites every week through AI features in Search".

Good looks like this: all three bots can reach your product pages, collection pages, home page and guides.

• Googlebot: Google Search, AI Overviews and AI Mode (critical)
• OAI-SearchBot: ChatGPT search (critical)
• PerplexityBot: Perplexity answers (critical)
• Claude-SearchBot and Claude-User: whether Claude can find and quote you (medium)

Check 3: Don't panic about the training bots

Some AI bots only collect pages to train future AI models. GPTBot (from OpenAI) and ClaudeBot (from Anthropic) are training only. Blocking them does not hide your store from ChatGPT or Claude answers. Plenty of store owners worry about this, but it is a separate choice.

Google-Extended is similar. It covers Gemini training and grounding, which means checking facts for Gemini answers. It does not control Google Search or AI Overviews. So blocking Google-Extended won't remove you from Google's AI answers.

Good looks like this: you know which of these bots you allow and why. Whether to let training bots in is your call. Just don't mix them up with the search bots in Check 2.

Check 4: Test three pages, not just your home page

A bot can be allowed on your home page but blocked from your products. Rules often apply to some folders and not others. So test one page of each kind: a product page, your home page and a guide or blog post.

The free AI Crawler Checker from GEO reads your robots.txt and shows which AI bots are let in, and which blocks really hide you from AI answers. Paste in each of the three addresses and compare the results.

Good looks like this: Googlebot, OAI-SearchBot and PerplexityBot are allowed on all three pages. Product pages matter most, because that is where shoppers land when an AI assistant names a product.

• A product page, for example yourstore.com/products/your-best-seller
• Your home page
• A guide or blog post, for example yourstore.com/blogs/news/your-guide

Check 5: Look beyond robots.txt at your firewall settings

Robots.txt is a polite request, not a lock. It also doesn't show everything. Some services that sit in front of a website can block bots on their own. A firewall or content delivery service, such as Cloudflare, may have a setting that blocks AI bots. That block won't appear in your robots.txt file.

Two other bots work differently again. ChatGPT-User and Perplexity-User fetch a page when a person asks about it. Both companies say robots.txt may not apply to them. So robots.txt is not the whole story either way.

Good looks like this: if you use Cloudflare or a similar service in front of your Shopify store, you have checked its bot settings and know what they block.

Check 6: Make sure your key facts load without extra scripts

Being allowed in is only half of it. The bot also has to be able to read the page. Vercel, a web hosting company, wrote in June 2025 that most AI crawlers read pages but do not run JavaScript. JavaScript is the code that makes some parts of a page appear after it loads.

If your price, sizes, ingredients or reviews only appear after a script runs, a bot may miss them. Some apps add content this way. One simple test: view the page source in your browser (right-click, then View page source) and search for your product's price or a key fact.

Good looks like this: the product name, price, main details and description are all in the page source as plain text. When SteelSeries and its agency NoGood worked on AI search, fixing pages that crawlers could not read was part of the job, according to a 2026 case study published by The Drum.

Check 7: Fix it in Shopify, then test again

Shopify lets you change robots.txt through a theme file called robots.txt.liquid. In Shopify admin, go to Online Store, then Themes, then Edit code, and look in the Templates folder. If the file isn't there, your store is using Shopify's default rules.

If you or an old app added a rule blocking Googlebot, OAI-SearchBot or PerplexityBot, remove it. If you are not comfortable editing code, ask your theme developer. A small typo can block more than you meant.

Good looks like this: after the change, you run the same three pages through the checker again and all three search bots are allowed. Then put a reminder in your calendar to check again every few months, and after any theme change or new app.