Home  › Marketing › Strategies

Caffeine: How Google's New Approach to Indexing the Web Changes Everything

  |  June 11, 2010   |  Comments

Google has overhauled the way it captures content on the Web. What does it mean for your business?

For the last several years, the ultimate question on every marketer's mind was simple and highly focused: how do I get to number one? And by "number one," I don't mean, "how do I get to dominate my category and sell more of my stuff than any of my competitors." Number one was very clearly, number one - the top spot on a Google search for a valuable keyword.

It was, of course, with good reason that marketers sought this spot. The difference between number one and number two could be hundreds of dollars a day. The quest to divine the secret of the algorithm - that complicated math problem that determined who got number one and who got number (and beyond) - was taken as seriously as any marketing challenge could be. We worked diligently to figure out if headings mattered more than image text, if inbound links were from the right places, and what, what, what should go into a meta tag.

Of course, all of this rested on a solid, but shifting platform. We were trying to figure out how the engineers at Google programmed its crawlers to create a picture of the Internet, a picture that it could (and did) change however and whenever it wanted. Whenever the index would shift, based on some supposition about what constituted relevance, rankings could be turned upside down. This happened, frequently, without a lot of warning, causing site owners to either scramble to get back on top or smugly post that their site remained high in the rankings.

Google's latest shift in the index was a bit different though. Its latest release, called Caffeine, was talked about for a long time, and the company had long signaled its intention to move away from its old and original way of categorizing the Web to a new method that better matched the real-time nature of the Web. Google finally pulled the trigger on this new release and it seems like a big deal.

What Is Caffeine?

The single most valuable piece of technology that Google has is its ability to capture content on the Web. Over the decade or so that the company has been doing this, the Web has shifted drastically, both in terms of user behavior and content. Google's original approach to indexing content was really focused on words and images, most of which didn't change all that frequently. Google got by on some version of the 80/20 rule: 80 percent of the content on the Web changed very slowly, and 20 percent changed very rapidly. It could put more of its resources toward the 20 percent and stay pretty current.

That's hardly the case today, where technology has been focused on collapsing the time gap between something happening, being captured, and being viewed. The Web is now more about real-time experiences than it is about massive amounts of mostly-static content. Google has long been talking about how it wanted to do a better job of capturing this dynamic content, and has done a few things (like putting Twitter results into results pages), but this is a real, fundamental shift in the way that the engine operates.

What Caffeine Means for You

Recent incursions by Facebook and Twitter aside, Google remains the number one way people get to sites on the Web, including the one where you sell your stuff. That means that, yes, you do still have to figure out how to get to that number one spot - or at least close. There's no guaranteed way to get to number one. You are always going to be competing against a set of people as smart and motivated as yourself. But here are three things that you must start thinking about and implementing:

  • Number 1: If your SEO (define) group doesn't get social media, get a new SEO group. Google's spider has always been attracted to content that changes frequently. You want to be seen as a part of the 20 (or so) percent of the Web that needs special attention. Now that dynamic content is even more important, and social media remains the best way to do this. Just make sure that the content that you're bringing in is still relevant to your core site's topic. Otherwise, you're efforts won't help your cause.

  • Number 2: Experiment and explore. Caffeine, as its name implies, is fast. Like, really fast. In the past, site owners had to wait days and often weeks for Google to index and re-index your site. Not anymore. Using a massively powerful and parallel set of computers, Google is cruising the Web all the time. That means you should be able to play around with new ideas and concepts and see the results fairly quickly. Stay away from anything that feels like it's trying to trick Google. But give some new ideas a try.

  • Number 3: Watch your competitors closely. A shift in the index means that new companies will rise in the rankings and old companies will fall. A few bloggers have suggested that local companies will suffer in the Caffeine era, as national brands rise to the topic of local searches. That may be the case, as will some other new shifts that maybe we haven't anticipated. The only way to know for sure is to pay very close attention to where you are ranking, and who is on your heels. Everyone else is going to be playing around just as much as you are, and they may discover something great.

Building an online business means that you have to pay very close attention to how you generate traffic and how much that costs. Google offers an amazing opportunity to capture lots of people, but it's a tricky business that gets trickier each time it makes a shift to its platform. This shift is a big one, but, in this case, it's really Google playing catch-up to a Web that has dramatically changed its nature. This should ultimately be a good thing, especially if you are a marketer who's used to looking for new opportunities.

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!

ABOUT THE AUTHOR

Gary Stein

Gary Stein is SVP, strategy and planning in iCrossing's San Francisco office. He has been working in marketing for more than a decade. Gary lives in San Francisco with his family. Follow him on Twitter: @garyst3in. The opinions expressed in Gary's columns are his alone.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

Resources

Jobs

    • Customer Service Consultant
      Customer Service Consultant (Bonner and Partners) - Delray BeachBonner & Partners: Full-time Customer Service Consultant Position Who we are...
    • Financial Editor
      Financial Editor (Confidential) - DurhamSIX FIGURE EDITORS WANTED: To enforce lofty NEW editing standards. Easy Conditions Unlikely. Promotion and...
    • Information Processing Specialist
      Information Processing Specialist (Agora Inc. ) - BaltimoreInformation Processing Specialist – The IP specialist position ensures the successful...