AI Discovery Index

AI Discovery Index is a live dashboard that measures how quickly search engines, AI crawlers and assistant fetches find newly published webpages. It records the first verified visit from each bot or live assistant, reports medians and interquartile ranges, and plots coverage and trend curves for 7-, 30-, 90-day and all-time windows. The dataset is maintained by an AI search agency as part of its AI SEO and AEO tooling, and is used to demonstrate AI visibility performance and offer services like AI audits, crawler audits and referral tracking. Tracking began 6 June 2026, the data was last updated 20 Oct 2026, and the index notes 330 pages published since inception (219 excluded because URLs were already live).

The findings show that AI crawlers typically reach new pages very quickly: median time to first crawl by any bot is 14 hours, and median time to first AI crawler is also 14 hours, with 72-77% of new pages visited by an AI crawler within 24 hours. Bingbot and ClaudeBot are often first (Bing ~47-51% of first crawls; Claude ~32-49%), both with ~14h medians. Googlebot/GoogleOther and OAI/GPT bots are slower (medians around 38h or longer), while Amazonbot and Applebot often take multiple days. Live assistant fetches (e.g., ChatGPT-user) are more variable and generally slower, with medians ranging from under a day up to several days and lower coverage, indicating that crawler discovery is rapid but assistant-level consumption lags and differs by platform.