Whether search and answer-engine crawlers can fetch, index, and discover the audited page.
Methodology 3.0.0-preview
A readiness score with visible rules and honest limits.
The audit examines the initial server response for one public URL, plus that host's robots.txt and sitemap. It is designed to identify technical work—not manufacture an “AI visibility” claim from evidence the scan cannot observe.
Run the auditScoring model
Pass and fail drive the score. Evidence quality stays visible.
Only applicable pass/fail checks earn or lose weighted points. Weighted not-tested checks reduce coverage instead of being silently counted as failures; zero-point advisories remain outside the math. Confidence combines evidence coverage and evidence quality. A numeric score is withheld below 70% coverage or 60% confidence.
Whether the page presents a clear topic, useful hierarchy, and extractable answers in its initial HTML.
Whether machine-readable markup identifies the organization, page subject, and commercial offering.
The checks
One policy powers the audit and this page.
The final audited URL returns a successful HTTP response.
HTML and HTTP robots directives do not contain noindex.
The page declares one valid, same-host canonical URL.
robots.txt permits Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, and PerplexityBot.
A sitemap is declared in robots.txt or found at /sitemap.xml.
Advisory: publish and maintain intentional robots.txt controls even though a missing file does not block crawlers.
Advisory: training, model-control, and user-initiated agents are reported separately and never change the score.
The initial HTML contains a concise, descriptive title.
The initial HTML contains a substantive meta description.
The page contains one non-empty H1.
Question-led headings, definition lists, or FAQ schema pair questions with substantive answers.
The initial HTML exposes at least one normal internal link.
The page pairs useful body copy with multiple descriptive subheadings.
At least one JSON-LD block parses and declares a schema.org type.
JSON-LD identifies an organization, person, article, product, or service entity.
JSON-LD uses @id or sameAs to connect the entity to a stable identity.
Crawler taxonomy
Search crawlers and data-use controls are not the same thing.
Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, and PerplexityBot affect the discovery-access check. Training, model-control, and user-initiated agents are shown for policy awareness only. Blocking one of those advisory agents is not scored as a search failure.
Google Search crawling, including content eligible for Google's AI search features.
Bing search discovery and indexing.
Surfaces websites in ChatGPT search results.
Search crawling used to improve relevance for Claude users.
Search indexing used to answer Perplexity queries.
OpenAI model-training control; it is not the ChatGPT search crawler.
Anthropic model-training control; it is separate from Claude-SearchBot.
A standalone control token for Gemini training and grounding; it does not control Google Search inclusion.
A fetch initiated by a ChatGPT user or agent action.
A fetch initiated by a Claude user request.
A missing robots.txt file means no crawler restriction was discovered; it does not mean “all bots blocked.” HTTP 429, authentication responses, server errors, and fetch failures produce not-tested evidence instead of a guessed result.
Limitations
What this audit does not claim
- This is a one-page, technical-readiness scan of the initial HTML response; it does not render client-side JavaScript.
- It does not measure rankings, answer-engine citations, referral traffic, conversions, or actual AI visibility.
- Crawler access is a point-in-time robots.txt interpretation, not proof that a provider has crawled or indexed the page.
- Structured data presence is checked syntactically and semantically at a high level; provider-specific eligibility is not guaranteed.
Data handling
The public scan does not require contact details.
The server processes the submitted public URL to produce the score and top findings. Contact details are requested only when you choose to unlock the detailed roadmap; that submission and audit summary are sent to eComStrategics through Formspree so Dan can follow up. Standard hosting and service logs may still apply. See the privacy notice.
Primary documentation