Comment are off

A/B Split E-mail Testing – An Overview

Define Method of Testing

A/B or Split e-mail testing is when you divide a portion (generally 20-40%) of your e-mail audience up into two groups for the purpose of testing the effectiveness of your marketing or communication message.  The ‘winning’ e-mail message is then sent to the remaining portion of the total list with the goal of incrementally improving the quality and effectiveness of the e-mail message or any number of other goals such as open rate, click through rate, or several other measurable mechanisms.

Setting Goals

When engaging in an A/B e-mail test, a well thought out plan is important to achieving quantifiable results.  Start by defining your goals or what about your e-mail you would like to improve upon.

  • Do you have below industry average open rates?
  • Are your open rates sufficient, but your click through rates poor or non-existent?
  • If you are getting people to your website through your e-mails – Are those users taking the next step to communicate or engage with you how you would like?


Plan of Action

Once you define your goals you need to put together a plan of action and execution.  Start by analyzing your e-mail’s subject line.  According to MailChimp “When it comes to e-mail marketing, the best subject lines tell what’s inside, and the worst subject lines sell what’s inside.”  You should use your current subject line format as your ‘A’ or baseline and draft a new ‘B’ subject line to try and improve the open rate relative to ‘A’.

Plan of Execution

Next, execute your new A/B e-mail testing plan using your new subject line.  Using your e-mail provider of choice, draft your next e-mail and then during setup, elect to do an A/B e-mail campaign.  Complete the dual subject line options and select which portion of your e-mail list you would like to test (Percentage of the whole) and how long you would like the A/B test to run (a few hours to a few days).  Refer to your existing e-mail data, if available, to determine the best duration.

Maximize Results

Overall, you want to get the best statistical data to make a decision on which subject line is the best to use on the remaining e-mail recipients and therefore should provide an improved open-rate.  After you get the final results from your campaign, you will need to analyze the data to see if your new subject line showed signs of improvement or not.  If improvements were shown, continue with the new format and consider trying additional e-mail subject tests to help refine your optimal subject line format and maximize your open rates.  If significant results were not achieved with the ‘B’ group, then go back to the ‘A’ group subject line and define a new ‘B’ group subject line and repeat the test with the next e-mail.

Continue to Improve

This article does not cover all aspects of A/B split testing, but merely gives an overview of the subject and a suggestion on how to implement split testing with your e-mail campaign subject titles.  Additionally, performing A/B split testing is not something you do one time and move on, it is a process of adjusting and re-adjusting your e-mail message to continually improve your statistical results.

A:  If you would like to learn more about A/B split e-mail testing, contact us at (317) 564-5010.

B:  Call us today at (317) 564-5010 to learn how A/B split e-mail testing can improve your results!

Scott Dine
About the Author
With years of experience in marketing and information technology, Scott has a passion for helping clients realize their ultimate potential. He does this by ensuring every Catalyst client receives outstanding marketing support and he works diligently to implement the latest online technology and keep clients abreast on recent marketing trends of CPA firms.