Increasingly, the biggest issue that businesses seem to encounter while deploying analytics involves business processes, and not technology. That view was reinforced last week when I was running a workshop with a group of students studying in an Internet retailing master’s program.
As part of the session, I asked the class what they thought some of the key ingredients were in executing successful Internet optimization. They started off by throwing out items like “good data” and “technology,” and then moved on to things like “flexibility,” “willingness to fail,” “good hypotheses,” “risk culture,” and “strong business model.” I then asked the class how many of them worked in organizations that had those characteristics. No hands went up.
Often organizations are happy to spend money on new campaigns or large-scale product development on their sites without thinking or being explicitly clear about how they will measure the campaign’s effectiveness or the new piece of functionality. So how do they know that they have done a good job? Measurement also needs to be built into the campaign or product development process as well. Here is a simple framework to use:
- State the objectives.
- Define the success indicators or KPIs (define).
- Perform a gap analysis.
- Create a measurement road map.
State the Objectives
Before you start, be very clear about what you are trying to achieve. State your objectives and make sure that they aren’t “marshmallow objectives.” Marshmallow objectives are soft and squidgy objectives that don’t hold up to scrutiny and are difficult to measure. People often talk about making objectives “SMART“: specific, measurable, achievable, realistic, and time-bound. Try to make your objectives as SMART as possible. If you come up with objectives like “improve the user experience,” force yourself to be smarter and ask questions like why, which users, in what way, and by when?
Define the Success Indicators or KPIs
Next, define your success indicators or KPIs. These should relate directly back to your objectives. If the objectives are SMART, the definition of the success metrics should be relatively straightforward. Constantly ask yourself: What does good look like? If I’m achieving my objectives, what will be happening in my business or on my site? Sometimes it can be difficult to measure objectives based on direct measures, so you may need to come up with other success metrics that represent indirect measures of success.
Perform a Gap Analysis
Once you know what you want to measure, analyze what you can measure now compared to what you ideally want to measure. Where are the gaps? Do you have the right measurement systems in place, and are they configured the right way? For example, if you want to improve the likelihood of someone booking a holiday once she’s done the research on the site, you might set up a success metric based on visitors’ propensity to book at a later date. You may need a survey in place to be able to measure that. So do you need to set up a survey from scratch, or do you just need to ask a specific question on an existing voice of the customer program?
Create a Measurement Road Map
The final stage is to create the measurement road map. Count the things that count. This is effectively the plan of how you will measure the things that you need to measure. Establish priorities for the work that needs to be done. Be clear about which gaps you will plug in your measurement systems and how that work will get done. This could range from just ensuring that there is a specific custom report developed through to the implementation of a new piece of software or the adoption of a new service.
This four-part framework can take days to complete or just minutes. It might be a very strategic piece of work looking at the business overall or it might be quite tactical, such as running an A/B test on a landing page. Whichever it is, it’s a way of putting measurement and analytics at the heart of your business processes. So anytime you’re planning to do something, ask yourself (or your colleagues): What does good look like? How will we know we’ve done a good job?
Turn your data into results with true ROI-driven marketing. Join us on Wednesday, September 23, 2009, at 1 p.m., for a free Webinar to learn how to make decisions to improve campaign performance.