Home  › Marketing › Strategies

Caffeine: How Google's New Approach to Indexing the Web Changes Everything

  |  June 11, 2010   |  Comments

Google has overhauled the way it captures content on the Web. What does it mean for your business?

For the last several years, the ultimate question on every marketer's mind was simple and highly focused: how do I get to number one? And by "number one," I don't mean, "how do I get to dominate my category and sell more of my stuff than any of my competitors." Number one was very clearly, number one - the top spot on a Google search for a valuable keyword.

It was, of course, with good reason that marketers sought this spot. The difference between number one and number two could be hundreds of dollars a day. The quest to divine the secret of the algorithm - that complicated math problem that determined who got number one and who got number (and beyond) - was taken as seriously as any marketing challenge could be. We worked diligently to figure out if headings mattered more than image text, if inbound links were from the right places, and what, what, what should go into a meta tag.

Of course, all of this rested on a solid, but shifting platform. We were trying to figure out how the engineers at Google programmed its crawlers to create a picture of the Internet, a picture that it could (and did) change however and whenever it wanted. Whenever the index would shift, based on some supposition about what constituted relevance, rankings could be turned upside down. This happened, frequently, without a lot of warning, causing site owners to either scramble to get back on top or smugly post that their site remained high in the rankings.

Google's latest shift in the index was a bit different though. Its latest release, called Caffeine, was talked about for a long time, and the company had long signaled its intention to move away from its old and original way of categorizing the Web to a new method that better matched the real-time nature of the Web. Google finally pulled the trigger on this new release and it seems like a big deal.

What Is Caffeine?

The single most valuable piece of technology that Google has is its ability to capture content on the Web. Over the decade or so that the company has been doing this, the Web has shifted drastically, both in terms of user behavior and content. Google's original approach to indexing content was really focused on words and images, most of which didn't change all that frequently. Google got by on some version of the 80/20 rule: 80 percent of the content on the Web changed very slowly, and 20 percent changed very rapidly. It could put more of its resources toward the 20 percent and stay pretty current.

That's hardly the case today, where technology has been focused on collapsing the time gap between something happening, being captured, and being viewed. The Web is now more about real-time experiences than it is about massive amounts of mostly-static content. Google has long been talking about how it wanted to do a better job of capturing this dynamic content, and has done a few things (like putting Twitter results into results pages), but this is a real, fundamental shift in the way that the engine operates.

What Caffeine Means for You

Recent incursions by Facebook and Twitter aside, Google remains the number one way people get to sites on the Web, including the one where you sell your stuff. That means that, yes, you do still have to figure out how to get to that number one spot - or at least close. There's no guaranteed way to get to number one. You are always going to be competing against a set of people as smart and motivated as yourself. But here are three things that you must start thinking about and implementing:

  • Number 1: If your SEO (define) group doesn't get social media, get a new SEO group. Google's spider has always been attracted to content that changes frequently. You want to be seen as a part of the 20 (or so) percent of the Web that needs special attention. Now that dynamic content is even more important, and social media remains the best way to do this. Just make sure that the content that you're bringing in is still relevant to your core site's topic. Otherwise, you're efforts won't help your cause.

  • Number 2: Experiment and explore. Caffeine, as its name implies, is fast. Like, really fast. In the past, site owners had to wait days and often weeks for Google to index and re-index your site. Not anymore. Using a massively powerful and parallel set of computers, Google is cruising the Web all the time. That means you should be able to play around with new ideas and concepts and see the results fairly quickly. Stay away from anything that feels like it's trying to trick Google. But give some new ideas a try.

  • Number 3: Watch your competitors closely. A shift in the index means that new companies will rise in the rankings and old companies will fall. A few bloggers have suggested that local companies will suffer in the Caffeine era, as national brands rise to the topic of local searches. That may be the case, as will some other new shifts that maybe we haven't anticipated. The only way to know for sure is to pay very close attention to where you are ranking, and who is on your heels. Everyone else is going to be playing around just as much as you are, and they may discover something great.

Building an online business means that you have to pay very close attention to how you generate traffic and how much that costs. Google offers an amazing opportunity to capture lots of people, but it's a tricky business that gets trickier each time it makes a shift to its platform. This shift is a big one, but, in this case, it's really Google playing catch-up to a Web that has dramatically changed its nature. This should ultimately be a good thing, especially if you are a marketer who's used to looking for new opportunities.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!

ABOUT THE AUTHOR

Gary Stein

Gary Stein is SVP, strategy and planning in iCrossing's San Francisco office. He has been working in marketing for more than a decade. Gary lives in San Francisco with his family. Follow him on Twitter: @garyst3in. The opinions expressed in Gary's columns are his alone.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

ion Interactive Marketing Apps for Landing Pages White Paper

Marketing Apps for Landing Pages White Paper
Marketing apps can elevate a formulaic landing page into a highly interactive user experience. Learn how to turn your static content into exciting marketing apps.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?

Jobs

    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora, www.agora-inc.com, continues to expand! In order to meet the needs of our...
    • Design and Publishing Specialist
      Design and Publishing Specialist (Bonner and Partners) - BaltimoreIf you’re a hungry self-starter, creative, organized and have an extreme...