# 2026-05-16
Another day, another peek behind the curtain of AI crawler activity on our site. Today, we saw a total of 576 visits, with 7 active bots diligently indexing our content. It's always fascinating to see how these automated visitors interact with AEO Scanner.
Let's dive into the bot breakdown. Here's a quick look at the top 5 bots by visit count:
| Bot Name | Visits |
| Bingbot | 531 |
| ClaudeBot | 28 |
| YandexBot | 11 |
| DuckDuckBot | 2 |
| OAI-SearchBot | 2 |
Bingbot continues to be the workhorse, accounting for the vast majority of our bot traffic with 531 visits and a remarkable 527 unique paths crawled. That's a lot of indexing! However, we did see some interesting activity beyond the usual suspects. ClaudeBot, while significantly behind Bingbot, showed a good number of visits and unique paths, focusing on sitemap files and the robots.txt file. This suggests ClaudeBot is actively trying to understand our site's structure.
Looking at the top paths, it's clear that the robots.txt file is a hot item, with 12 visits from 5 different bots. Similarly, the sitemap-news.xml file also attracted a lot of attention (12 visits by 2 bots), indicating that crawlers prioritize these resources for understanding our content. We also noticed some errors. Specifically, Bingbot reported a 503 error on several /aeo/shops/llms.txt files (lanteh-com-tw, sfang-org-tw, sites-google-com), and a 404 error on zenisis.ai. This tells us the AI is actively trying to assess various LLM implementations related to our AEO data, but there are some accessibility problems to address in certain shops. It's a good reminder to always review the shops' LLM setup.
For a concrete actionable GEO/AEO tip based on today's observations: Since Bingbot is the most active, and given the errors, prioritize ensuring all /aeo/shops/llms.txt files are accessible and properly formatted, especially those related to the sites reporting errors. This is crucial for successful indexing by the most active crawler. Also, keep an eye on how the sitemap files are performing and make sure to update them regularly to assist crawlers in finding new content.
That's all for today. Remember, understanding your site's AI traffic is key to optimizing for search and ensuring your content is effectively indexed. Keep an eye on your own site's AI crawler activity with AEO Scanner!