StoreCheck Visibility · Guides
In short: The four most common technical reasons for invisibility in AI answer engines: your bot is blocked at the edge (even with a permissive robots.txt), robots.txt explicitly disallows search bots, your pages look empty without JS, or structured data is missing. These are measurable and fixable — but know this: technical accessibility is the prerequisite for being cited, not the guarantee.
| AI eğitim botları politikası | %96 |
| Yapısal veri (JSON-LD) | %24 |
| AI arama botları robots.txt'de engelli | %6 |
| AI/arama botlarına edge engeli | %2 |
Method: 209 completed self-serve checks (anonymized — no domains). Rows show the share of sites where this check came back warn/fail. Data: 2026-09-23. Cite via the open research dataset →
robots.txt is your site's declaration — it says "I don't want these bots". But the real gate lives elsewhere: a WAF, bot-protection service or CDN layer can block bot requests independently of robots.txt. The owner looks at robots.txt and assumes "we're open"; in reality OAI-SearchBot gets a 403.
This distinction matters because the diagnosis differs: you see the declaration by opening the file, but the real block only shows when you send a request as the bot itself. Verify yourself: curl -s -o /dev/null -w "%{http_code}" -A "OAI-SearchBot" https://yourdomain.com/ — if it returns 403/503 instead of 200, the problem is at the edge, not in robots.txt.
Bots come in three classes with different consequences:
Rather than "block everything" or "allow everything", keeping search bots open is usually the right balance.
Most AI crawlers don't run JavaScript — if your page doesn't carry real text in the first HTML, the bot sees an empty shell. Check the source (Ctrl+U): is your content there as text?
The second layer is structured data: JSON-LD markup for Product, Organization, FAQ lets the machine answer "what is this, whose is it, what does it sell" with confidence. If the source has no application/ld+json, the answer engine struggles to classify the page.
Answer engines don't evaluate a site in isolation — they look for independent sources mentioning you: directories, reviews, forums, news. If you're technically flawless but nothing on the web verifies you, the model says "I don't know this business". This layer can't be measured from outside and isn't fixed by one scan — it takes original data, customer reviews and industry mentions.
Also, each engine's source ecosystem differs: appearing on ChatGPT doesn't mean appearing on Perplexity — measurement must be done engine by engine.
No — anyone promising that is selling marketing. What can be done is removing blockers and increasing citability; the decision belongs to the engine.
Not by itself — llms.txt is a bridge introducing your site to agents; if there's a block or content can't be read, it changes nothing. On a clean site it's good practice.
Then the problem isn't technical — it's the authority/evidence layer: independent sources talk about your competitor more. Compare your forum, directory and review presence.
This page is technical guidance; no ranking or AI citation guarantee is claimed.