Bot traffic in Google Analytics are often miscategorized as Direct traffic, making it hard to filter out unwanted data from sources like web crawlers, scrapers, and spam. While eliminating bot traffic isn't feasible, focusing on users who spend more than a few seconds on your site helps you evaluate meaningful interactions and reduce the impact of bots.
Where does bot traffic come from?
Bot traffic comes from a variety of sources… too many to list, but below are some of the most common sources:
- WEBSITE WIDGETS that crawl your site looking for data, i.e. pricing data for dealers. This is often crawled off websites and pushed to you in a software.
- WEB SPIDERS AND CRAWLERS who are checking your site for issues/broken links and errors.
- INVENTORY URL SCRAPERS from websites such as 3rd party providers will grab your VDP URLs so that your inventory on their website links to the right page on your site.
- SPAMMERS that are trying to collect phone numbers and email address to sell and distribute.
How do I eliminate bot traffic from GA4?
Unfortunately, you can't eliminate bot traffic from GA4. GA4 automatically filters out known bots and spiders from your property, and updates traffic numbers as their database updates. These traffic sources are identified by Google research as well as by Google referencing the International Spiders and Bots list, maintained by the Interactive Advertising Bureau.
Sometimes internal or 3rd party tools will masquerade as a bot or spider, even though they will not be flagged by Google’s systems. For systems like this, it is recommended that you reach out to these partners and see if it is possible to get their IP address(es) so you can set up an IP exclusion within your property. IP exclusions will allow you to label this traffic as “internal” so that it can be more easily filtered in reports.
If I can't eliminate Bot Traffic, how do I best evaluate my direct traffic?
- Focus on users and not sessions.
- Identify people who spend time on your site, i.e., users who have spent more than 15 seconds on your website. Most bots come, get their required information and leave in a few seconds, so focusing on time on site will help evaluate quality traffic.
How to Reduce Bot Traffic
Although it’s a continuous effort, refining your filters this way offers the most practical solution for reducing the impact of bot traffic on your GA data. As mentioned above, if it is possible to label this traffic as “internal” (using IP exclusion) then that can go a long way in ensuring your reports remain clean of this data.