Well, another day, another deep dive into the fascinating world of AI crawler activity! Today, April 17th, we observed a total of 878 visits to the AEO Scanner, with 7 bots making their presence known. It's always a bit of a thrill to see who's poking around.
Here's a quick look at the top 5 bots and their visit counts:
| Bot Name | Visits |
| Bytespider | 807 |
| Googlebot | 54 |
| Bingbot | 9 |
| AhrefsBot | 2 |
| ClaudeBot | 2 |
Bytespider, as usual, is the undisputed champion, accounting for the vast majority of our traffic. We also had a visit from Googlebot with 54 visits, which is substantial. Interesting to note the presence of AhrefsBot and ClaudeBot, too.
One particularly interesting pattern today is the focus of Googlebot on the /pixel path, with 25 visits to that particular endpoint. It's a clear indication that Google is keeping a close eye on our image optimization and potentially tracking pixel data. Also, the appearance of the new bot, AhrefsBot, is something we will keep an eye on.
From an AEO perspective, today's data emphasizes the importance of ensuring the reliability of our shop feeds, specifically the /aeo/shops/ endpoints, since the Bytespider made multiple attempts at those pages. We saw a number of 503 errors on the /aeo/shops/ endpoints, which might indicate temporary server issues or rate limiting by the AI. This is a crucial area for us to monitor and optimize for smoother bot access. A concrete action item for us is to check the server logs and see the status code from where the errors come. Maybe there is a pattern to the timing of those failures.
Overall, it was a busy day in the world of AI crawlers! Remember, you can keep tabs on your own site's AI traffic with AEO Scanner. Stay informed, stay ahead!