Has your site ever dropped suddenly from the index or disappeared mysteriously from search results? Have you ever received a notice that your site is using cloaking techniques? Unfortunately, sometimes a malicious party “hacks” a website: they penetrate the security of a site and insert undesirable content. Sophisticated attackers can camouflage this spammy or dangerous content so that it doesn’t appear for normal users, and appears only to Googlebot, which could negatively impact your site in Google’s results.
In such cases it used to be very difficult to detect the problem, because the site would appear normal in the eyes of the user. It may be possible that only requests with a User-agent: of Googlebot and coming from Googlebot’s IP could see the hidden content. But that’s over: with Fetch as Googlebot, the new Labs feature in Webmaster Tools, you can see exactly what Googlebot is seeing, and avoid any kind of cloaking problems. We’ll show you how:
Let’s imagine that Bob, the administrator of www.example.com, is searching for his site but he finds this instead:
That’s strange, because when he looks at the source code of www.example.com, it looks fine:
With much surprise Bob may receive a notice from Google warning him that his site is not complying with Google’s quality guidelines. Fortunately he has his site registered with Webmaster Tools, let’s see how he can check what Googlebot sees:
First Bob logs into Webmaster Tools and selects www.example.com. The Fetch as Googlebot feature will be at the bottom of the navigation menu, in the Labs section:
The page will contain a field where you can insert the URL to fetch. It can also be left blank to fetch the homepage.
Bob can simply click Fetch and wait a few seconds. After refreshing the page, he can see the status of the fetch request. If it succeeds, he can click on the “Success” link…
…and that will show the details, with the content of the fetched page:
Aha! There’s the spammy content! Now Bob can be certain that www.example.com has been hacked.
Confirming that the website has been hacked (and perhaps is still hacked) is an important step. It is, however, only the beginning. For more information, we strongly suggest getting help from your server administrator or hoster and reading our previous blog posts on the subject of hacked sites:
If you have any questions about how to use the Fetch as Googlebot feature, feel free to drop by the Webmaster Help Forum. If you feel that your website might be hacked but are having problems resolving it, you might want to ask the experts in our “Malware and Hacked sites” category.
PS Keep in mind that once you have removed hacked content from your site, it will generally still take time for us to update our search results accordingly. There are a number of factors that affect crawling and indexing of your content so it’s impossible to give a time-frame for that.
Written by Javier Tordable, Software Engineer and John Mueller, Webmaster Trends Analyst
Webmaster Level: Beginner
It has probably happened to you: you’re reading articles or watching videos on the web, and you come across some unrelated, gibberish comments. You may wonder what this is all about. Some webmasters abuse other sites by exploiting their comment fields, posting tons of links that point back to the poster’s site in an attempt to boost their site’s ranking. Others might tweak this approach a bit by posting a generic comment (like “Nice site!”) with a commercial user name linking to their site.
Why is it bad?
FACT: Abusing comment fields of innocent sites is a bad and risky way of getting links to your site. If you choose to do so, you are tarnishing other people’s hard work and lowering the quality of the web, transforming a potentially good resource of additional information into a list of nonsense keywords.
FACT: Comment spammers are often trying to improve their site’s organic search ranking by creating dubious inbound links to their site. Google has an understanding of the link graph of the web, and has algorithmic ways of discovering those alterations and tackling them. At best, a link spammer might spend hours doing spammy linkdrops which would count for little or nothing because Google is pretty good at devaluing these types of links. Think of all the more productive things one could do with that time and energy that would provide much more value for one’s site in the long run.
Promote your site without comment spam
If you want to improve your site’s visibility in the search results, spamming comments is definitely not the way to go. Instead, think about whether your site offers what people are looking for, such as useful information and tools.
FACT: Having original and useful content and making your site search engine friendly is the best strategy for better ranking. With an appealing site, you’ll be recognized by the web community as a reliable source and links to your site will build naturally.
Moreover, Google provides a list of advice in order to improve the crawlability and indexability of your site. Check out our Search Engine Optimization Starter Guide.
What can I do to avoid spam on my site?
Comments can be a really good source of information and an efficient way of engaging a site’s users in discussions. This valuable content should not be replaced by gibberish nonsense keywords and links. For this reason there are many ways of securing your application and disincentivizing spammers.
- Disallow anonymous posting.
- Use CAPTCHAs and other methods to prevent automated comment spamming.
- Turn on comment moderation.
- Use the “nofollow” attribute for links in the comment field.
- Disallow hyperlinks in comments.
- Block comment pages using robots.txt or meta tags.
For detailed information about these topics, check out our Help Center document on comment spam.
My site is full of comment spam, what should I do?
It’s never too late! Don’t let spammers ruin the experience for others. Adopt security measures discussed above to stop the spam activity, then invest some time to clean up the spammy comments and ban the spammers from your site. Depending on you site’s system, you may be able to save time by banning spammers and removing their comments all at once, rather than one by one.
If I spammed comment fields of third party sites, what should I do?
If you used this approach in the past and you want to solve this issue, you should have a look at your incoming links in Webmaster Tools. To do so, go to the Your site on the web section and click on Links to your site. If you see suspicious links coming from blogs or other platforms allowing comments, you should check these URLs. If you see a spammy link you created, try to delete it, else contact the webmaster to ask to remove the link. Once you’ve cleared the spammy inbound links you made, you can file a reconsideration request.
For more information about this topic and to discuss it with others, join us in the Webmaster Help Forum. (But don’t leave spammy comments!)
Written by Adel Saoud and Paolo Petrolini
