Measurement Without Experimentation is Vanity - How To Do Big Data Better

,   |  November 20, 2013   |  Comments   |  

With big data, organizations have a choice: they can either discover new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Big data has yet to yield major new insights or spark meaningful change within many organizations. This isn't a fault of data, but more a result of our inability to reframe its storytelling capability.

As Edward de Bono says: “Most executives, many scientists, and almost all business school graduates believe that if you analyze data, this will give you new ideas. Unfortunately, this belief is totally wrong. The mind can only see what it is prepared to see.”

The limits of perception are the limits of knowledge. The insights that big data can yield are limited by the quality of the questions an organization can ask. The culture of an organization is tied to its ability to see and perceive differently from “business as usual”.

All stories start with a language, and the language of business is data (on the face of it, at least). The problem is not that data tells stories, but that we’re complacent about hearing the same ones. We need to examine and re-design the experiments we use to interrogate the data, to start asking better questions.

Most large organizations are culturally unprepared for the opportunities that big data offers, especially in terms of their ability to re-imagine themselves and re-shape the industry they operate in. With big data, organizations have a choice: they can either discover new stories and new ways to thrive in times of change, or keep telling the same stories and re-quantifying what they already know.

Exploding some myths

1. Big data is quantitative and qualitative

Most organizations routinely privilege quantitative data over qualitative data, misunderstanding the function of each and the relationship between them.

Qualitative techniques are investigative. The data generated is typically theme and insight based. This is great for re-framing a situation, discovering the right problem to solve and generating a set of hypotheses.

Quantitative techniques are analytical. The data generated is statistical and number based. This approach is great for validation and verification - for proving and disproving hypotheses.

As Benjamin Yoskovitz (Lean Analytics) notes: “Use qualitative to discover and quantitative to prove”.

Big data is quantitative and qualitative. Once this relationship is understood, you create a hypothesis engine for your business that can scale with speed and impact - a powerful source of strategic value.

2. There’s no such thing as raw data

There’s no such thing as raw data, and no data is objective. Decisions about what to measure and why are always explicitly or implicitly made. Such decisions reflect the culture of an organization and are constrained by what is considered to be acceptable knowledge. This limits the scope of insight that big data is able to provide.

3. Data does not make decisions

Data creates a frame of reference for decisions to be made. In many organizations, that frame of reference may stifle decision making, instead of enabling it. Gathering the wrong data in the “right way” diverts attention away from real issues and opportunities. Using data as a political tool to defend the status quo is bad business strategy, it's much better to test opinions, assumptions and closely held beliefs instead.

Stop measuring, start experimenting

Most organizations will easily build the technical competence to capture as much data as possible. The challenge will be to develop the ability to understand and interpret that data in an intelligent, meaningful and actionable way.

In order to meet this challenge and discover sustainable advantages, businesses need to adopt an experimental mindset:

a. Build an “Experiment Design” capability

Experiment design is one of the most important emerging business practices surrounding big data and analytics. In the same way that financial fluency is key to running a successful business, so is a culture of experimentation. Done well, this capability will position you as a market leader.

b. Expand the scope of big data within your organization

Big data is not just the realm of the marketing department. Expand its scope to include all parts of the organization, so new frames of reference can be created, and new stories can be told.

Using big data to run a business experimentally is not only a powerful risk mitigation strategy, it’s a profoundly sensible way of navigating a chaotic and ever-changing market.

Subscribe to ClickZ Asia to receive your weekly dose of the region's latest trends, tips, and insights straight to your inbox.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!


Patti  Hunt

Patti Hunt is founder and director of On-off Design & Technology, a strategic design consultancy based in Hong Kong. Patti has held senior positions as a User Experience (UX) consultant and service designer in Australia and most recently head of strategic design for a Hong Kong based agency. She’s an advocate for using design thinking and practices to spark and drive meaningful change.


Kristin Low

Kristin Low is founder of On-off Design & Technology, a strategic design and digital services company, based in Hong Kong. He has a background in user experience (UX) design, service design, and product management. Kristin founded the Design Thinking network in Hong Kong, which has over 500 members. He’s also an MC, mentor, and trainer, and hosts jams and hackathons. He’s passionate about blending human-centered insights with quantitative data to drive business growth and innovation.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.


    • Internet Marketing Campaign Manager
      Internet Marketing Campaign Manager (Straight North, LLC) - Downers GroveWe are looking for a talented Internet Marketing Campaign Manager...
    • Internet Marketing Specialist
      Internet Marketing Specialist (InteractRV) - DallasInternet Marketing Specialist InteractRV - Anywhere Telecommute, USA SEM | SEO | Content Creator...
    • Tier 1 Support Specialist
      Tier 1 Support Specialist (Agora Inc.) - BaltimoreThis position requires a highly motivated and multifaceted individual to contribute to and be...