Measurement Without Experimentation is Vanity - How To Do Big Data Better

,   |  November 20, 2013   |  Comments   |  

With big data, organizations have a choice: they can either discover new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Big data has yet to yield major new insights or spark meaningful change within many organizations. This isn't a fault of data, but more a result of our inability to reframe its storytelling capability.

As Edward de Bono says: “Most executives, many scientists, and almost all business school graduates believe that if you analyze data, this will give you new ideas. Unfortunately, this belief is totally wrong. The mind can only see what it is prepared to see.”

The limits of perception are the limits of knowledge. The insights that big data can yield are limited by the quality of the questions an organization can ask. The culture of an organization is tied to its ability to see and perceive differently from “business as usual”.

All stories start with a language, and the language of business is data (on the face of it, at least). The problem is not that data tells stories, but that we’re complacent about hearing the same ones. We need to examine and re-design the experiments we use to interrogate the data, to start asking better questions.

Most large organizations are culturally unprepared for the opportunities that big data offers, especially in terms of their ability to re-imagine themselves and re-shape the industry they operate in. With big data, organizations have a choice: they can either discover new stories and new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Exploding some myths

1. Big data is quantitative and qualitative

Most organizations routinely privilege quantitative data over qualitative data, misunderstanding the function of each and the relationship between them.

Qualitative techniques are investigative. The data generated is typically theme and insight based. This is great for re-framing a situation, discovering the right problem to solve and generating a set of hypotheses.

Quantitative techniques are analytical. The data generated is statistical and number based. This approach is great for validation and verification - for proving and disproving hypotheses.

As Benjamin Yoskovitz (Lean Analytics) notes: “Use qualitative to discover and quantitative to prove”.

Big data is quantitative and qualitative. Once this relationship is understood, you create a hypothesis engine for your business that can scale with speed and impact - a powerful source of strategic value.

2. There’s no such thing as raw data

There’s no such thing as raw data, and no data is objective. Decisions about what to measure and why are always explicitly or implicitly made. Such decisions reflect the culture of an organization and are constrained by what is considered to be acceptable knowledge. This limits the scope of insight that big data is able to provide.

3. Data does not make decisions

Data creates a frame of reference for decisions to be made. In many organizations, that frame of reference may stifle decision making, instead of enabling it. Gathering the wrong data in the “right way” diverts attention away from real issues and opportunities. Using data as a political tool to defend the status quo is bad business strategy, it's much better to test opinions, assumptions and closely held beliefs instead.

Stop measuring, start experimenting

Most organizations will easily build the technical competence to capture as much data as possible. The challenge will be to develop the ability to understand and interpret that data in an intelligent, meaningful and actionable way.

In order to meet this challenge and discover sustainable advantages, businesses need to adopt an experimental mindset:

a. Build an “Experiment Design” capability

Experiment design is one of the most important emerging business practices surrounding big data and analytics. In the same way that financial fluency is key to running a successful business, so is a culture of experimentation. Done well, this capability will position you as a market leader.

b. Expand the scope of big data within your organization

Big data is not just the realm of the marketing department. Expand its scope to include all parts of the organization, so new frames of reference can be created, and new stories can be told.

Using big data to run a business experimentally is not only a powerful risk mitigation strategy, it’s a profoundly sensible way of navigating a chaotic and ever-changing market.

Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Patti  Hunt

Patti Hunt is founder and director of On-off Design & Technology, a strategic design consultancy based in Hong Kong. Patti has held senior positions as a User Experience (UX) consultant and service designer in Australia and most recently head of strategic design for a Hong Kong based agency. She’s an advocate for using design thinking and practices to spark and drive meaningful change.


Kristin Low

Kristin Low is founder of On-off Design & Technology, a strategic design and digital services company, based in Hong Kong. He has a background in user experience (UX) design, service design, and product management. Kristin founded the Design Thinking network in Hong Kong, which has over 500 members. He’s also an MC, mentor, and trainer, and hosts jams and hackathons. He’s passionate about blending human-centered insights with quantitative data to drive business growth and innovation.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...