Googlebots are an essential part of the Google search engine. Constantly crawling the world wide web looking for new links and websites to ensure that you get the latest and most relevant search results. However a recent study has revealed that for every 25 genuine Googlebot visits you’re site gets you’ll also be visited by a fake Googlebot — more than 23 percent of which are used for DDoS attacks, hacking, scraping and spamming. This is among the findings of security company Incapsula’s 2014 Search Engine Study.
The study is based on over 400 million search engine visits to 10,000 sites over 30 days and shows Google is the most active search engine. Googlebots are responsible for more than 60 percent of all page crawls, the second in line the MSN/Bing bot notches up only 24.5 percent. What’s also interesting is that there’s no correlation between the number of human visits to a site and how often it’s crawled by Googlebots. Google pays just as much attention to the backwaters of the web as it does to popular sites.
Most interesting and concerning though is the number of fake Googlebots. The study shows that over 4 percent of bots using Google’s HTTPS user agent aren’t what they claim to be. The benefit of this to hackers is that site owners generally allow unhindered access to Google’s crawlers in order to protect their search results.
Incapsula’s logs reveal that fake Googlebots are used mainly for DDoS attacks but also in spamming and hacking activity. Security solutions that don’t use case-by-case traffic inspection are unable to spot the real bots from the fakes.
Feel free to read the entire report, in detail, here. Otherwise I will leave you with the following infographic:
