Research Hub
Monthly studies of how popular websites in every EU country treat AI crawlers and accessibility. Totals only, no website named, free to reuse under CC BY 4.0.
- AI crawlers · monthly Live
AI crawler blocking on EU websites
How popular national websites in each EU country treat AI search and training crawlers in robots.txt, and how many publish llms.txt.
Key findings
- 10% of the measured websites block at least one AI search or answer crawler in robots.txt.
- 17% block at least one AI training crawler.
- 82% block no AI crawler at all.
4,645 websites measured · list of 08/2026
Updated 17/09/2026
- Accessibility · monthly Live
Website accessibility in the EU
Five common accessibility barriers on the homepages of popular national websites in each EU country.
Key findings
- 81% of the measured homepages have at least one of the five barriers.
- The most common barrier is “Links without text”, found on 57% of homepages.
- A homepage has 22.7 barriers on average.
3,754 websites measured · list of 08/2026
Updated 17/09/2026
- Original Research
How EU Public- and Private-Sector Homepages Score on Accessibility
In April 2026 we ran the same comprehensive audit on the homepages of 394 EU public- and private-sector organisations. The overall median was 64/100, and EU institutions had a median of 67/100. Same method for every site, one-time snapshot.
April 2026 · Accessibility snapshot
How the studies work
Each month we take up to 200 popular websites per EU country — national domains on HTTPS, one per registered domain — from the public Chrome UX Report list of each country’s top 10,000 sites. That is a sample of up to 5,400 websites in 27 countries.
Our crawler, RankProofBot, reads each site’s robots.txt and obeys it, then fetches /llms.txt and the homepage. It counts what it finds and keeps no per-site data: results are totals per country and for the EU. Sites that block us, do not answer or build their homepage only with JavaScript are reported as excluded, not counted as results.
These are deliberately narrow checks of one page per site. The accessibility study looks for five barriers in the raw HTML, which is not an accessibility audit; the AI crawler study reads robots.txt rules, not what crawlers actually do.
Each study page has the full table per country, CSV and JSON downloads and ready-to-quote findings under a CC BY 4.0 licence. If you spot an error, tell us through the contact page and we will correct it.
CC BY 4.0 — attribution required.