Search Overflow

AEO · Answer engine optimisation

Eligibility comes before visibility.

Is this page eligible to be used in an AI answer at all?

Before an assistant can cite you it has to fetch you, render you, and find something worth quoting. Most sites that cannot be cited are not badly written. They are blocked.

39checks across 15 named AI crawlers

How this usually goes wrong.

You blocked the crawler that decides citations
Blocking OAI-SearchBot removes you from ChatGPT Search. Blocking GPTBot only opts you out of training and costs you nothing. In robots.txt the two rules look identical.
The answer exists only in the browser
An assistant that runs no JavaScript sees what your server sent. We count the words that survive that and tell you the number.
A snippet cap nobody set on purpose
A site can be fully crawlable and still be unquotable, because nosnippet or a max-snippet cap quietly limits how much of the page can be used.

What happens after we find it.

Missing structured data, incomplete organization and product schema, and social card markup are derivable today. Indexing directives are deliberately not: a noindex is as often intentional as it is a mistake, so we explain the finding and leave that call to you.

Access is not eligibility, and eligibility is not inclusion. Unblocking a crawler makes a page reachable. Whether it is then cited stays the engine’s decision.

Questions, answered plainly.

Is blocking AI bots bad for me?

It depends entirely on which ones. A handful decide whether you can be cited in an answer; the rest only collect training data, and blocking those is a legitimate choice with no citation cost. We separate the two rather than counting how many bots you block.

Does the free scan cover all of these?

No. The free scan matches your robots.txt against six crawlers and reads one page. The rest of the catalogue runs on a full crawl of the site.