A/B Testing

A/B testing is a popular method used by online marketers to test different variations of a given campaign or message. In email marketing, A/B testing is used to optimize the email campaign you are working on. The most common scenarios for A/B testing are to test two or more subject lines (i.e. the title of the email) and to test different colors of an image (e.g. green vs red), which may represent the call to action button in your email. By sending the same email but with slight variations of it, you can figure out which email is the best performing one in terms of open rate and click through rate.

To start working on an A/B test you need to have a pre-existing email message saved in the Email tab and have decided what changes you want to make to it.

• The next step is to create as many versions of the original email. To do this, select the original email on the Email tab under Saved and click on create a copy from the actions drop down. Once you have created a copy of the message, go ahead and make the changes to that email. For example, your first version may have a different subject line. Continue to create as many copies of the original email for each change. 

• It is recommended that you change only one variable (e.g. subject line, images, text, etc) per email so that you can accurately track its result.

How to Create a New A/B Test

1. Once you have your original email and its variations (as modified copies of the original) ready and saved, you will be able to find them on the table:
2. Select the size of the test mailing.
    
   Note: This step only records the percentage number of the test mailing group which will be randomly generated after you choose the recipients to whom to send this mailing to in the next steps.

3. Select the performance metric for the A/B test: choose either open rate or click through rate.

   As a rule of thumb, choose the open rate metric if your email versions vary only on subject line. In fact, a more relevant subject line leads to a higher open rate with all else constant. On the other hand, choose the click through rate metric if your email versions vary in the layout, text, or images of the body of your email. In this case, for example, a brighter image (vs a darker one) may lead to a higher click through rate.

   You have to select a duration for the A/B test. Select the number of hours before the best performing email will be determined and sent out to the remaining recipients

    

   • To learn more about email performance metric view our knowledge base article.

4. Click on Continue. In this page, you choose the recipients and recipient groups to whom you want to send out the A/B test mailing. Once the A/B test is active, the email versions will be instantly sent to a randomly generated mailing test group from the recipients you have selected. After the given duration of the A/B test, the system will send the best performing email to the remaining recipients. 

    

   The A/B test cannot be scheduled. Please choose an approppriate time to send out the mailing. After the test emailing is finished, the best preforming email will be sent out to the remaining recipients.