Measurement Without Experimentation is Vanity - How To Do Big Data Better

,   |  November 20, 2013   |  Comments   |  

With big data, organizations have a choice: they can either discover new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Big data has yet to yield major new insights or spark meaningful change within many organizations. This isn't a fault of data, but more a result of our inability to reframe its storytelling capability.

As Edward de Bono says: “Most executives, many scientists, and almost all business school graduates believe that if you analyze data, this will give you new ideas. Unfortunately, this belief is totally wrong. The mind can only see what it is prepared to see.”

The limits of perception are the limits of knowledge. The insights that big data can yield are limited by the quality of the questions an organization can ask. The culture of an organization is tied to its ability to see and perceive differently from “business as usual”.

All stories start with a language, and the language of business is data (on the face of it, at least). The problem is not that data tells stories, but that we’re complacent about hearing the same ones. We need to examine and re-design the experiments we use to interrogate the data, to start asking better questions.

Most large organizations are culturally unprepared for the opportunities that big data offers, especially in terms of their ability to re-imagine themselves and re-shape the industry they operate in. With big data, organizations have a choice: they can either discover new stories and new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Exploding some myths

1. Big data is quantitative and qualitative

Most organizations routinely privilege quantitative data over qualitative data, misunderstanding the function of each and the relationship between them.

Qualitative techniques are investigative. The data generated is typically theme and insight based. This is great for re-framing a situation, discovering the right problem to solve and generating a set of hypotheses.

Quantitative techniques are analytical. The data generated is statistical and number based. This approach is great for validation and verification - for proving and disproving hypotheses.

As Benjamin Yoskovitz (Lean Analytics) notes: “Use qualitative to discover and quantitative to prove”.

Big data is quantitative and qualitative. Once this relationship is understood, you create a hypothesis engine for your business that can scale with speed and impact - a powerful source of strategic value.

2. There’s no such thing as raw data

There’s no such thing as raw data, and no data is objective. Decisions about what to measure and why are always explicitly or implicitly made. Such decisions reflect the culture of an organization and are constrained by what is considered to be acceptable knowledge. This limits the scope of insight that big data is able to provide.

3. Data does not make decisions

Data creates a frame of reference for decisions to be made. In many organizations, that frame of reference may stifle decision making, instead of enabling it. Gathering the wrong data in the “right way” diverts attention away from real issues and opportunities. Using data as a political tool to defend the status quo is bad business strategy, it's much better to test opinions, assumptions and closely held beliefs instead.

Stop measuring, start experimenting

Most organizations will easily build the technical competence to capture as much data as possible. The challenge will be to develop the ability to understand and interpret that data in an intelligent, meaningful and actionable way.

In order to meet this challenge and discover sustainable advantages, businesses need to adopt an experimental mindset:

a. Build an “Experiment Design” capability

Experiment design is one of the most important emerging business practices surrounding big data and analytics. In the same way that financial fluency is key to running a successful business, so is a culture of experimentation. Done well, this capability will position you as a market leader.

b. Expand the scope of big data within your organization

Big data is not just the realm of the marketing department. Expand its scope to include all parts of the organization, so new frames of reference can be created, and new stories can be told.

Using big data to run a business experimentally is not only a powerful risk mitigation strategy, it’s a profoundly sensible way of navigating a chaotic and ever-changing market.

Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!


Patti  Hunt

Patti Hunt is founder and director of On-off Design & Technology, a strategic design consultancy based in Hong Kong. Patti has held senior positions as a User Experience (UX) consultant and service designer in Australia and most recently head of strategic design for a Hong Kong based agency. She’s an advocate for using design thinking and practices to spark and drive meaningful change.


Kristin Low

Kristin Low is founder of On-off Design & Technology, a strategic design and digital services company, based in Hong Kong. He has a background in user experience (UX) design, service design, and product management. Kristin founded the Design Thinking network in Hong Kong, which has over 500 members. He’s also an MC, mentor, and trainer, and hosts jams and hackathons. He’s passionate about blending human-centered insights with quantitative data to drive business growth and innovation.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!




Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.


    Information currently unavailable


    • Lead Generation Specialist
      Lead Generation Specialist (The Oxford Club) - BaltimoreThe Oxford Club is seeking a talented writer/marketer to join our growing email lead-generation...
    • Health Marketing Editor
      Health Marketing Editor (Agora Inc.) - BaltimoreCome flex your intellectual muscle as part of Agora, Inc’s ( legal team...
    • Technical Business Analyst
      Technical Business Analyst (OmniVista Health) - BaltimoreOmniVista Health is looking to add a Technical Business Analyst to our expanding team...