Another day, another deep dive into the fascinating world of AI crawler activity on AEO Scanner (aeo.codecity.com.tw). Today, we observed a total of 587 visits, with 6 bots actively crawling our site. Let’s break down the key trends and see what we can learn.
Here’s a snapshot of the top 5 bots, based on their visit counts:
| Bot Name | Visits |
| Bingbot | 540 |
| YandexBot | 40 |
| Bytespider | 4 |
| DuckDuckBot | 1 |
| Googlebot | 1 |
Bingbot continues to dominate the landscape, accounting for a staggering 540 visits. This is expected, but still worth noting. YandexBot is a distant second, clocking in with 40 visits. We saw Bytespider crawl our site, showing 4 visits. Both DuckDuckBot and Googlebot each visited our site only 1 time. It's interesting to see these different bots and how they interact with our site, or what their crawls reveal about it.
A few interesting patterns emerged today. First, it seems that there were issues related to the path /aeo/shops/flexoplate-com-tw/llms.txt. Bytespider encountered a 503 error trying to access this file. It is worth investigating what this is, as it could be caused by either the bot misinterpreting something, or problems in the setup of our site. Second, /robots.txt had 13 visits from 5 bots. This emphasizes its core role and the importance of having the file set up properly.
From a GEO/AEO perspective, understanding these bot behaviors helps us tailor our content and site structure. Given the heavy presence of Bingbot, it’s a good reminder to ensure our site's XML sitemap is properly formatted and accessible, as Bingbot frequently uses this file. Making sure that our robots.txt is not overly restrictive is also paramount.
Remember to stay informed by monitoring your own site's AI traffic with AEO Scanner. Happy optimizing!