The Importance of QA

Thou shalt not ignore the importance of Quality Assurance (QA).

QA testing is perhaps the most important step in advanced technology development, including software, applications, Web sites, and online advertising creative. It’s the final step before your creation goes public. It’s your last chance to catch functional or other errors. Perfection is the goal, but beating spec requirements is acceptable (and more realistic). If you miss something, I guarantee the public will find it. And if it’s significant, it could have a negative impact on your brand. QA is not to be taken lightly.

Which is why I was absolutely astounded a vendor we recently worked with had no QA process in place to check its product.

We developed an email program that included HTML email with a form embedded in the message. Used to be a tricky proposition, but nowadays most email clients and Web-based email services support forms. So we forged ahead.

After delivering the code to the vendor, I got on the phone and inquired about its QA process. “They do this all the time, probably hundreds of times a day,” I thought to myself as I dialed the number. “Surely they have a robust QA process to verify what we’re doing will work flawlessly across the board.”

I ask directly, “You guys will run that through your QA process and let us know if anything’s wrong, right?”

Awkward silence. “What do you mean by ‘QA process’?” he asks.

Stunned by the response, I calmly described what I meant. I expected maybe they just called it something else. No such luck. Their QA process basically consists of sending out test email to a few different folks in our agency, then to some guys who work for the vendor and have Yahoo or Hotmail addresses. No organization, no checklist, no process. No confidence.

When we deliver a Web development project for a client, our QA process is relentless. It’s our product. It has to be as close to perfect as possible. Clients rely on us to deliver projects this way.

Our QA process looks something like this:

1. After the designers finish their work, they’ll often look at it in different browsers on both Mac and Windows platforms. They check their own work, learning what works and what doesn’t for each combination of browser and platform.
2. The programmers go at the project. They code everything, testing as they go, then check their work. They look at even more combinations of browsers and platforms.
3. The QA guy really tears into it. He’s ruthless. He’s got more computers surrounding his desk than the Lone Gunmen. It’s like mission control over there. He’s got a three-page checklist to go through every time we do an HTML banner. The checklist for a Web site is, of course, more detailed and much, much longer. The matrix of browser/platform combinations we test is like an eye chart. (Excuse the minor embellishments here, I’m trying to make a point. Consider it creative license and just work with me.) He’s meticulous and doesn’t miss a thing.
4. Our account and project managers have a go.

When we’re through, everything works the way it’s supposed to work. Or, we’ve figured out a workaround that makes for a pleasurable user experience. That code and user experience are our product. We take pride in their accuracy.

For an email vendor, helping a client achieve its marketing objectives is the product. More specifically, it’s delivering the email message without technical problems, looking the way it was designed. You may sell opt-in names and addresses, but that’s not your product. We’ve checked the piece in-house, but you require me to use your delivery system. That adds a wild card into a mix of variables I’d much prefer to maintain tight control over. Then you tell me you have no process to check your product with? You’ve got no way to ensure once delivered through your system, my email will look and function the same way in Outlook, Lotus Notes, AOL, Yahoo, and Hotmail?

I can’t fathom how this is possible. Is it a cost-cutting measure? An oversight? Is it because you don’t consider successful email delivery to be your product? Could it be names and addresses are what you believe to be your product? How can you not have a QA process?

This is the kind of impersonal, nonchalant attitude that gives email marketing a bad name. This short-sighted approach will, doubtless, limit the long-term success of any business. It is a massive gap in a service offering and should be fixed. Immediately.