Measurement Without Experimentation is Vanity - How To Do Big Data Better

,   |  November 20, 2013   |  Comments   |  

With big data, organizations have a choice: they can either discover new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Big data has yet to yield major new insights or spark meaningful change within many organizations. This isn't a fault of data, but more a result of our inability to reframe its storytelling capability.

As Edward de Bono says: “Most executives, many scientists, and almost all business school graduates believe that if you analyze data, this will give you new ideas. Unfortunately, this belief is totally wrong. The mind can only see what it is prepared to see.”

The limits of perception are the limits of knowledge. The insights that big data can yield are limited by the quality of the questions an organization can ask. The culture of an organization is tied to its ability to see and perceive differently from “business as usual”.

All stories start with a language, and the language of business is data (on the face of it, at least). The problem is not that data tells stories, but that we’re complacent about hearing the same ones. We need to examine and re-design the experiments we use to interrogate the data, to start asking better questions.

Most large organizations are culturally unprepared for the opportunities that big data offers, especially in terms of their ability to re-imagine themselves and re-shape the industry they operate in. With big data, organizations have a choice: they can either discover new stories and new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Exploding some myths

1. Big data is quantitative and qualitative

Most organizations routinely privilege quantitative data over qualitative data, misunderstanding the function of each and the relationship between them.

Qualitative techniques are investigative. The data generated is typically theme and insight based. This is great for re-framing a situation, discovering the right problem to solve and generating a set of hypotheses.

Quantitative techniques are analytical. The data generated is statistical and number based. This approach is great for validation and verification - for proving and disproving hypotheses.

As Benjamin Yoskovitz (Lean Analytics) notes: “Use qualitative to discover and quantitative to prove”.

Big data is quantitative and qualitative. Once this relationship is understood, you create a hypothesis engine for your business that can scale with speed and impact - a powerful source of strategic value.

2. There’s no such thing as raw data

There’s no such thing as raw data, and no data is objective. Decisions about what to measure and why are always explicitly or implicitly made. Such decisions reflect the culture of an organization and are constrained by what is considered to be acceptable knowledge. This limits the scope of insight that big data is able to provide.

3. Data does not make decisions

Data creates a frame of reference for decisions to be made. In many organizations, that frame of reference may stifle decision making, instead of enabling it. Gathering the wrong data in the “right way” diverts attention away from real issues and opportunities. Using data as a political tool to defend the status quo is bad business strategy, it's much better to test opinions, assumptions and closely held beliefs instead.

Stop measuring, start experimenting

Most organizations will easily build the technical competence to capture as much data as possible. The challenge will be to develop the ability to understand and interpret that data in an intelligent, meaningful and actionable way.

In order to meet this challenge and discover sustainable advantages, businesses need to adopt an experimental mindset:

a. Build an “Experiment Design” capability

Experiment design is one of the most important emerging business practices surrounding big data and analytics. In the same way that financial fluency is key to running a successful business, so is a culture of experimentation. Done well, this capability will position you as a market leader.

b. Expand the scope of big data within your organization

Big data is not just the realm of the marketing department. Expand its scope to include all parts of the organization, so new frames of reference can be created, and new stories can be told.

Using big data to run a business experimentally is not only a powerful risk mitigation strategy, it’s a profoundly sensible way of navigating a chaotic and ever-changing market.

Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Sept 5 to take advantage of Super Saver Rates!


Patti  Hunt

Patti Hunt is founder and director of On-off Design & Technology, a strategic design consultancy based in Hong Kong. Patti has held senior positions as a User Experience (UX) consultant and service designer in Australia and most recently head of strategic design for a Hong Kong based agency. She’s an advocate for using design thinking and practices to spark and drive meaningful change.


Kristin Low

Kristin Low is founder of On-off Design & Technology, a strategic design and digital services company, based in Hong Kong. He has a background in user experience (UX) design, service design, and product management. Kristin founded the Design Thinking network in Hong Kong, which has over 500 members. He’s also an MC, mentor, and trainer, and hosts jams and hackathons. He’s passionate about blending human-centered insights with quantitative data to drive business growth and innovation.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

Marin Software: The Multiplier Effect of Integrating Search & Social Advertising

The Multiplier Effect of Integrating Search & Social Advertising
Latest research reveals 68% higher revenue per conversion for marketers who integrate their search & social advertising. In addition to the research results, this whitepaper also outlines 5 strategies and 15 tactics you can use to better integrate your search and social campaigns.



    • Digital Marketing Analyst
      Digital Marketing Analyst (GovLoop) - Washington D.C.Are you passionate about audience acquisition? Love effective copy and amazingly effective...
    • Product Specialist
      Product Specialist (Agora Inc. ) - BaltimoreDescription: The Product Specialist is hyper-focused on the customer experience and ensures that our...
    • Partnerships Senior Coordinator
      Partnerships Senior Coordinator (, Inc.) - Las VegasZappos IP, Inc. is looking for a Partnerships Senior Coordinator! Why join us? Our...