The Atlantic recently reported that over 60% of web traffic is non-human. While such a high percentage is questionable, at least according to comScore, the fact remains that many websites are plagued by non-human traffic (NHT).
What is NHT?
There are several NHT sources, with some traffic being good, some traffic being spammy, and some traffic being outright malicious. The good traffic consists of search engine bots that crawl your website’s index, for example. Spammy traffic might consist of scrapers that come along to duplicate your content or spammers that leave irrelevant comments. Malicious traffic might consist of hackers that steal website data or install malware.
Regardless of the nature of the NHT, the issue posed with all of it is that it skews your Google Analytics data. Thus, your site visit and unique visitor numbers as well as bounce rates become inflated. You also see a decreased number of pages per visit and decreased visit duration and goal conversion.
Finally, if you are paying for advertising that is charged through a CPM model, bots cost you money. Many publishers are very aware of bot traffic and how it inflates the advertiser’s budget- yet do very little to alert the advertiser of this issue. That’s another reason why you must learn how to spot NHT.
How do you know if you have NHT?
There are some tell-tale ways to check if you are getting NHT on your website. On your Google Analytics page, go to Audience, then Technology, then Browser & OS. If Mozilla Compatible Agent pops up on your list of browsers, there’s a high likelihood you’re seeing bot traffic. Although the Mozilla Compatible Agent is sometimes used by mobile apps browsers, it is quite often used by bots like the SiteConfidence, YahooNews and Ask.com bots.
Other likely bots include these agents: Microsoft Corp, Google Inc., Yahoo! Inc., Inktomi Corp, and StumbleUpon Inc.
Bot traffic can also sometimes be recognized by its novelty and non-involvement; visits will be new, have a one page view, 100% bounce rate and zero time duration. A good percentage of bot traffic will be using old browser versions.
How do you kick bots out?
If you go into your Admin area on Google Analytics, you will find a Filters area. Click on that, then on New Filter. This will open up the following area, which you can use to set a custom filter that will exclude a given ISP Organization. For the filter pattern, input the following string:
^(microsoft corp(oration)?|inktomi corporation|yahoo! inc\.|google inc\.|stumbleupon inc\.)$|gomez
This will remove the majority of bots from your metrics.
Simpler strings may be created that list the likely offender(s) as follows: yahoo|microsoft corp$|inktomi
Hit Save and you’re done.
What about NHT that leaves comment spam?
Of course, getting rid of ISP bots that crawl your pages is just the start. Many blogs are also plagued by spam comments which take time and effort to constantly delete. Many of these comments, which contain links to various product or service sites, are left by SEO spammers.
Fortunately, if you have a WordPress-based website, you can go to the Plugins panel and locate the Akismet plugin, which uses an algorithm and database to differentiate spam and legitimate comments. Keep in mind that the plugin costs $5/month if your website is earning money through affiliate products or advertising. Thus, just by activating the plugin, you won’t have it actually working unless you sign up for the payment plan.
Once you’ve paid, you’ll receive your API key from Akismet.com. After inputting that key, you’ll be able to use the Akismet menu to look at “caught” comments. If by some chance those comments aren’t spam, you can mark them as such and put them back into your comments section. You can also mark comments that have snuck through Akismet’s algorithm as spam- this helps the plugin adjust its algorithm to catch those comments in the future.
Other useful plugins include the following:
Growmap Anti Spambot Plugin (GASP) is free and enables you to add a captcha form to your comment area. Blog visitors must check a box to confirm they are human before being allowed to leave a comment.
CommentLuv is not exactly a plugin, but it does enable blog visitors to include a link to their latest or preferred post with their comment. This “humanizes” the commenting process and cuts down on spammers or even visitors who would otherwise leave a link to an irrelevant post.
What about malicious NHT?
Malicious NHT often includes malware and spyware installs, website hijacking and data (e.g., credit card number) theft. Luckily, there are several effective and free WordPress plugins that prevent a majority of attacks to your website. These plugins include the following:
All In One WordPress Security provides three levels of protection via its “basic,” “intermediate” and “advanced” settings. The basic setting won’t interfere with your site features yet still protect your website. Conversely, the advanced setting offers the most protection but may disable some site features.
iThemes Security helps stop automated attacks and strengthens passwords and user credentials. The plugin also tracks vulnerable plugins and outdated software.
Bulletproof Security provides a number of firewalls, login security and monitoring, site backup and tracking options, code error monitoring and much more.
Finally, it goes (almost) without saying that you should regularly create a backup copy of your database just in case an attack or other issue comes up and completely disables your website or blog.
How do you battle NHT and other spam issues or attacks? Please leave a comment below.