DAC Essentials Series: A/B Testing Basics

. August 19, 2017

When brands create new products or services – and the marketing plans to sell them – they often base their decisions on what customers tell them. But which is more important:  What customers say? Or what they actually do?

The answer, of course, is actual behavior. And therein lies the power of a track-tested marketing measurement technique that’s helped many a marketer determine which tactics work best.

You’ve probably heard of A/B testing, and at its core the concept is rather simple: Test something several different ways in the real world and see what works best. Then go with that. What could be more basic?

Yet businesses both big and small trip on this every day. They survey customer preferences – often via social media – and base decisions on feedback. But stated preferences can’t hold a candle to actual behavior when it comes to identifying what works – and what doesn’t.

Digital mavens have long used A/B testing to try out such things as different website designs, button sizes, colors, email subject lines and an almost infinite array of other items. The concept is so straightforward and effective that it can apply to almost any circumstance where a choice must be made. For almost any marketing organization, this “test-and-learn” approach makes perfect sense.

It’s Just What it Sounds Like

A/B testing is exactly what it sounds like. You test approach “A” and approach “B” (and perhaps many others as well) and compare results. The key is that you do it in the real word with real customers or prospects and real-time results. And you keep doing it with different segments until you have the answers you need.

The results of the battle between A and B are the data-driven insights you seek. You need to do it for different customer segments because while Option A might win generally, some segments might favor Option B, C or D.

One reason A/B testing works is that customers often don’t know themselves what they really want. They might say one thing, and do another. Sound familiar? A/B testing helps cut through the fog.

Clear, Objective Answers

The beauty of A/B testing is that it has little problem delivering clear answers about highly subjective preferences for such things as taste, color, shapes, images, layouts and the like. If you simply ask customers which color they prefer, for example, they’ll express a preference. But when you actually observe and measure their response to different colored products, packaging, marketing materials or web pages with A/B testing, the results might be radically different.

In the real world, what this means is that while many things influence consumer purchase behavior, consumers themselves are often unaware of them. For example, the package something comes in has a big influence on purchasing, even when it shouldn’t matter.

Music influences what people buy and how long they spend in a shop, restaurant or pub. Lighting influences how people notice products and buy them. Other factors include smells, excitement, the number of choices (fewer can be better) and how prices are displayed.

While these things, among others, influence what we buy, consumers usually have not idea their thought process has been subtly altered.

Offer Different Choices and Measure the Results

By no means should you ignore what your customers say. But you must also pay attention to what they do by offering different choices and measuring and observing (first hand if possible) the results.

Don’t simply ask customers or prospects to “imagine” something. You have to create it and let them experience it. This live testing gets the best results by far. It works even better if people don’t know they’re part of a test. You want people to behave as normally as possible.

No matter what industry you’re in, it’s vital to continuously test, learn and adapt. You don’t have to fully understand why Option B performs better. If it does, go with it.

This handy online A/B calculator can help you compare data and determine which test is working better.

Tags: , ,

Category: Articles, Basics, How-To

About the Author ()

Daniel Kehrer is Executive Editor of the ANA Data Analytics Center (DAC), a leading voice of thought leadership and education in marketing measurement, data and analytics. He is also the Founder of BizBest Media Corp. and previously headed marketing at MarketShare LLC, an advanced marketing analytics technology company.

Leave a Reply

Your email address will not be published. Required fields are marked *