Well, another day, another deep dive into the fascinating world of AI crawler activity on AEO Scanner! Today, we observed a total of 584 visits, with a strong showing from our bot friends – 8 in total. Let's dig into the details to see what the digital denizens have been up to.
First off, here's a quick look at the top 5 bots that were the most active on our site today:
| Bot Name | Visits | Unique Paths |
| Bingbot | 505 | 496 |
| ClaudeBot | 36 | 6 |
| Googlebot | 13 | 2 |
| AhrefsBot | 11 | 10 |
| YandexBot | 10 | 8 |
As you can see, Bingbot is still the clear leader, accounting for the vast majority of bot traffic with a whopping 505 visits. This is pretty typical, but still something to keep an eye on. ClaudeBot, with 36 visits, shows a significant level of activity too, and it is definitely worth our attention.
One interesting pattern we noticed today is the continued focus on sitemap files. Both /sitemap.xml and /sitemap-news.xml were heavily visited by multiple bots. In fact, these paths were among the top paths accessed overall, with 28 and 20 visits, respectively. This suggests these crawlers are actively trying to understand our site's structure and recent content updates. This level of interest in sitemaps is not unusual, but it's a good reminder that they are crucial for effective SEO.
Also, a couple of errors popped up today. Bingbot encountered a 503 error on /aeo/shops/minarc-com-tw/llms.txt and a 404 error on /aeo/shops/w-i-com-tw/llms.txt. This could indicate that some LLM-related content or shop data is not correctly configured or available. If these shops are relevant to our AEO data, we need to investigate and fix these issues promptly.
Here’s a concrete AEO tip based on today’s observations: Since Bingbot is so active and is the primary source of 503 and 404 errors, we should prioritize the quality and correctness of LLM-related data for shops that are targeted by Bingbot. We should review the content or the structure of the data and verify its availability, ensuring the AI crawlers can access our information without problems. This will help Bingbot crawl our site more effectively and ensure that our data is being indexed correctly.
That's all for today's AI crawler report! Remember to keep an eye on your own site's AI traffic with AEO Scanner. Stay informed, stay ahead, and keep those bots happy!