Imagine if I told you that every fifth person you met would try and rob you. That, without fail, 20 per cent of all people through the door of a shop would be there to steal, take another customer’s details or snoop around for a competitor.

Does this sound far fetched? It’s everyday life on the Internet.

Around 40 per cent of all Internet traffic is driven by automated software, otherwise known as bots, according to our latest Bad Bot Research Report for 2017. Of this, roughly half of all bot traffic is good for you – it includes website indexing by the likes of Google and Microsoft that helps your customers find you when they search, for example.

However, that leaves 20 per cent of all Internet traffic rated as “bad.” For large companies and popular internet sites, it’s even worse – for the top 10,000 sites ranked by Alexa worldwide, around 36.4 per cent of all traffic is malicious.

For those running businesses, there are four typical areas where bots will be targeted:

1.     Proprietary content / pricing information – this covers any content that you may create for your customers, or for your own company’s benefit. For major publishers, this may be the output from journalists that customers pay their subscriptions for; for other businesses, it can include their insight and expertise captured in white papers and blog posts. This information can be stolen and then posted elsewhere on the web.

2.     Log-in pages – for any company that sells online, knowing your customers are who they say they are is essential. This is normally based on creating a combination of an email address and a password. However, the log-in pages used for registering are a natural target for bots. Either bots will try to break in to legitimate customer accounts, or they will attempt to create fake accounts for use in fraud.

3.     Web forms – these are used to gather comments, enter information and generally allow customers to contact companies. However, they are often targets for bots as well. Anyone who has ever had to deal with comment spam will recognise this issue.

4.     Payment processing – it shouldn’t be a surprise that bots are targeted at financial gain. For any company doing business on the Internet, security around how payments are processed should be a priority, but bots are often not an obvious threat.

What to look for

The challenge here is not that these attacks are taking place. Instead, it’s spotting when a problem is developing and stopping it while letting legitimate users carry on.

Bots are designed to look just like regular users, and they don’t cause the big site-wide issues that other IT security problems do. Whereas a Denial of Service attack will stop a site completely, a bot attack will carry on potentially undetected.

One tell-tale sign that bots are active on your site will be an increase in bank chargebacks to resolve customer complaints. These will be caused by fraud transactions where the bank involved will try to claw back money involved. If these start to grow rapidly with no distinct cause, then bots may be involved.

Similarly, around a third of all sites will have evidence in the form of spam accounts or web forms. Other signs include higher numbers of failed log-in attempts than normal, accounts being locked more often than normal, or increased complaints regarding accounts being locked.

To stop these kinds of attacks, it’s worth looking at where you do business and how many potential customers come from outside your target markets. If you don’t normally sell to customers in Russia or China, then you can block traffic from those countries automatically. Other countries can be added if they represent more in terms of fraud than legitimate business.

Another tactic is to stop requests from old browser versions. While there are many potential customers out there that don’t practice good habits around updating their software regularly, the reality is that most customers will be on the latest versions of their browsers. Stopping requests attributed to old browser versions can stop around 10 per cent of all fraudulent traffic.

Similarly, it’s worth checking which bot activity you allow on your site in the first place. While the Google and Microsoft bots can be beneficial to your business, others may not. White-listing those that you want and stopping the rest can block up to 25 per cent of bad traffic.

Preventing bots from working on your site can reduce fraud and costs due to criminal activity. However, it can also help spot the bad activity that would otherwise go under the radar. While there are benefits from automation for customers, stopping bots should help those real-world customers get a better experience too.