GoogleBot Gets Smarter: Crawling AJAX Comments
For years, Google has not crawled a significant part of websites and blogs: the comments. There were two reasons for this:
- Google always aims to provide the best webpages in its results. Historically, many comments can be filled with spam, or otherwise be irrelevant or unhelpful to the content of the post.
Regarding #1, Panda updates have done a great job in removing spam sites with bad comments. Also, most websites have now established a platform and community in which the comments are helpful and relevant. Sometimes the comments can even be more helpful than the post itself, which is a frequent occurrence on blogs like SEOmoz.
So What Does Comment Indexation Mean To You?
It’s simple. Remember point #1 above? Google wants to deliver relevant, non-spam sites to its users. A primary signal to relevancy is recency. Something that has fresh content never appears stale to Google. Indexation of comments shows that there is fresh content on the site. Fresh, engaging content that changes each time Google visits your site. At the beginning of the month Google launched a “freshness” update that solidifies the commitment to fresh content. Both of these announcements are clearly connected and provide a singular message: keep content updated often.
Bear in mind that crawling new comments doesn’t always mean your site will get boosted to the #1 position. There are different parts of your website. If the comments are the only thing staying fresh and the rest of your site remains unchanged for months/years at a time, Google is smart enough to recognize this.
So What Should You Do About It?
If you have Disqus or Facebook comments enabled on your blog, check Google to see if your comments are being indexed. Find a comment and copy it directly into the Google search box surrounded by quotes (to get an exact match). If you see the comment, dig a little deeper to be aware of the context surrounding it.
Also (and this is the hard one), be actively engaged with users of your website. This means responding to complaints, providing help to those who need it, and sharing news often. This also means that negative comments can be indexed, so it’s additionally important to be responsive.
The Bigger Picture