The Email Savant Series: Simplifying A/B Split Testing

Mark Vogel

Written by Mark Vogel

June 14, 2018

Are your email marketing messages returning the best results possible? Are you sure? The only way to know is to test – and do so repeatedly on an ongoing basis. There are two kinds of email marketing tests: A/B split tests and multivariate tests.

A/B split tests compare an original version of your message (known as the “control”) against a second version (known as the “variant”). The variant will include a change to a just ONE element, such as the subject line, an offer, the time of day, a call to action (CTA), a single element of the design, and so on.

Multivariate testing means that you test several elements at once within your email message so that one message will be completely different from the other. While multivariate testing has its advantages in certain situations, there are also disadvantages, including statistically inaccurate results with smaller lists and/or shorter test cycles, and higher levels of complexity in understanding the results. With A/B split testing, results are usually quite clear and overt – which leads to obvious actions to implement in subsequent messages. So, for this blog, we’ll discuss A/B split testing.

Why Should You Test?

There are many reasons to create an A/B split testing effort, and to make it an essential component of your overall email marketing strategy:

What Should You Test?

Testing Email Subject Lines

Your recipients won’t click through on a call-to-action – or convert to a sale – unless they OPEN your email! Nearly half of your email recipients will open email based on the subject line, while two-thirds of them might report your message as spam based solely on the subject line. The subject line is the best place to start your testing efforts.

Testing Email Content

Many of the same tactics for subject lines can apply to the body copy of the message. Personalization is a proven method to boost response rates.

Testing Email Links

The “paradox of choice” theory says that the more choices we have, the more unlikely we are to make any decision at all. Many marketers want to give their audience all the click choices possible – which may be the wrong decision.

Testing Email Design

There are infinite ways to create your email marketing templates. While all messages need to be responsive on multiple devices, here’s a small sampling of layout ideas to consider for testing:

Testing Offers and Calls-to-Action

If the primary reason for our email is for recipients to take a specific action – buy a product, call you, download a white paper, and so on – then you need to understand what offers and calls-to-action will help you meet your goals.

Testing Email Timing

Find out when your audience wants to hear from you through testing.

Testing isn’t a once-and-done activity. Continue testing new designs, offers, and calls-to-action. Your target audiences are constantly evolving. Your email marketing strategy must evolve with them!

Mark Vogel is president of Vogel Marketing Solutions LLC and serves as a lead email consultant for Nxtbook Media. He has more than 35 years of experience in the marketing world and has been actively engaged in email campaigns for more than 20 years. His email marketing clients include Fortune 500 companies, e-commerce retailers, non-profits, local businesses, and more. He can be reached at

Related Posts


40+ Awards

Consistently ranked in the top 10 best places to work in PA, and ranked nationally in the Top 101 Best and Brightest Companies.



Nxtbook Media has supported more than 100K projects... and counting.


7.5 Years

Our top clients have been with us for an average of 7.5 years.

Back to Top