Marketing, once the wild world of slogan writing and seat-of-your-pants decision-making (or, as in this Dilbert cartoon, decisions based on politics within the organization) is now much more quantitative with strategies based on the rigorous application of metrics. A/B testing now joins the stable of tools marketers must master to inform decisions and optimize returns.
Another tool used by marketers is marketing simulation. Today, let’s discuss these two quantitative marketing analytics tools and the types of solutions provided.
A/B testing
A/B testing, at its most basic, is a way to compare two versions of something to figure out which performs better. While it’s most often associated with websites and apps, Fung says the method is almost 100 years old [source]
A/B testing involves creating different versions of your marketing materials, and the A/B designation doesn’t mean you can only compare 2 options. Consumers view only 1 version using a process of random selection to determine which version is shown to an individual consumer, so you manage the potential for bias. Marketers then monitor performance to determine which version performs best, usually based on consumer actions, such as buying the product or subscribing to a newsletter. Sometimes, marketers assess consumer emotions, such as liking, or consumer thoughts, such as knowledge about the brand after running the test.
The way you run your A/B test must match your goals for any marketing campaign. See below for an example of A/B testing.
Because it’s a powerful method for increasing conversion, A/B testing is kind of the gold standard for metrics-driven marketers (and should be for everyone doing digital marketing). Yet, problems exist.
For instance, you can only make a single change between versions of your marketing materials — such as a headline, color, or text used in your CTA (Call to Action), etc. — which may result in a lot of different versions, which then requires a large number of consumers to evaluate results effectively. Ethical concerns also exist regarding using human beings as test subjects. For instance, a classic A/B test involves offering a product at different prices to consumers to judge the impact of price on the quantity sold. Obviously, consumers purchasing a lower-priced product got benefits unavailable to consumers who were offered the higher price for the same product. Since testing usually involves live experiments, not thought experiments, the ethics of value between the two consumers raises some concerns.
Here’s another take on the ethics of A/B testing:
The problem is that, when it’s not done in a transparent, responsible way, A/B testing can leverage the worst impulses of human psychology to convince you to click on something. When it comes to political content, for instance, that kind of sensationalizing contributes to political polarization [source].
Online, examples of A/B tests include:
- Does the orange or green CTA box generate more clicks?
- Would consumers respond better if your directed materials at them, i.e., let us help you versus let us help?
- Does a white paper generate more responses than a video?
Marketing simulation
In contrast, to an A/B test, a marketing simulation sets up a “real world” (or as close to a real-world experience as possible) test to see how a product performs. And, while an A/B test determines between two (or more options) for a single factor, a marketing simulation tests a whole variety of factors that might impact the outcome.
Now, don’t confuse marketing simulation games with what I’m talking about, which is using marketing simulation as a research tool. Marketing simulation games are primarily used with students to see how their decisions impact behaviors and performance. The biggest complaint about these games is that the results don’t seem to reflect what happens in the real world.
An example of a marketing simulation is the storefront we set up at my first market research job. There was a small room with shelving like a traditional store and a cash register. Subjects received a budget, then asked to buy products fitting into the budget. Our first client in the simulated storefront made holiday gift wrap and wanted to know which design to distribute during the upcoming holiday season. We stocked the shelves with gift wrap from their proposed designs as well as those they already produced and those of their closest competitors. Subjects got $40 to spend, and we monitored the products purchased at the register. They kept the products they bought within the budget. Results provided precise information the company used to determine which brand to distribute and which products they might consider discontinuing.
Sure, the storefront wasn’t strictly identical to the one a customer faces in a real store, and their money was only available to purchase gift wrap, but it came pretty close to the real thing, much better than running a series of A/B tests.
Online marketers do the same thing by simulating a website or an app; for instance, A web designer creates a sandbox to test ideas. Unlike the sandbox created for web design, however, a marketing sandbox doesn’t just check whether a feature works or not, it examines HOW people use your website or app. Newer simulations involve testing the site or app without writing a single line of code. Based on the principles of Lean and Agile, we call these prototypes, but they’re still a form of marketing simulation.
Online examples of marketing simulations a company might conduct to use in decision-making include:
- Which consumer segment generates the most profits and, hence, we should focus on in our advertising?
- Where would consumers be most likely to buy our products, a website, an aggregator like Amazon or eBay, or from a social platform like Facebook?
While you might use A/B testing to discover the answers, marketing simulation is often a better option for the reasons listed above. Plus, marketing simulation determines responses more holistically rather than one element at a time.
Types of marketing simulations
Marketers sometimes use consumer panels for marketing simulations, attempting to match the subjects to their target market(s), if known.
In the case of a consumer panel, you use real people in your simulation, which is expensive and time-consuming. An alternative is constructing a simulated consumer panel using advanced databases and modeling through artificial intelligence to predict how these human surrogates respond to marketing efforts.
Companies like Ignite Technologies and IBM Watson work in this space. They create a simulated digital marketplace peopled by databases reflecting real consumers and modeled using sophisticated algorithms to deliver customer-driven insights that guide your marketing efforts.
Need marketing help to support business growth?
We welcome the opportunity to show you how we can make your marketing SIZZLE with our data-driven, results-oriented marketing strategies. Sign up for our FREE newsletter, get our FREE guide to creating an awesome website, or contact us for more information on hiring us.
Hausman and Associates, the publisher of MKT Maven, is a full-service marketing agency operating at the intersection of marketing and digital media. Check out our full range of services.
