A/B testing (sometimes called split testing) is sending and comparing two versions of an email campaign to see which one performs better. You compare two email sends by creating two or more variants (let’s call them A and B) and then send each one to a sample portion of your list at the same time. The one that gets a better conversion rate (either clicks or opens), wins!
How to Set Up an A/B Test
From your Dashboard, click Email > Campaigns in the left navigation. Click +New Campaign.
Choose A/B Test and click Next.
That will take you to the Build Campaign page. Click Set A/B Testing to choose the criteria for your test.
Choose whether you want to test the Open Rate or Click Rate, email distribution rate (test sample size), and how long to run the test.
Email distribution: The slider lets you choose the percentage of your total list that you want the variants to go to. So if your total list size is 100 emails, and the slider is at 20 percent, the sample send size to determine the winner would be 20 emails total. If you have two variants, 10 would get Variation A, 10 would get Varation B. Setting the distribution rate to 100 percent will send it to everyone immediately.
End test after: This lets you choose how much time you want the sample to collect information until we pick the winner.
Winning metric: The winning variant can be determined either by open rates or click rates.
Once you’re finished setting the conditions, click Save A/B Testing.
After editing your design, you can click the + next to Variation under Content and then click New Variant.
This will create Variation A and Variation B. You can edit the subject line, preview text, and who the email is from by clicking Edit next to each. You can add variations by clicking the blue + or remove them by clicking the red – circle.
You can edit the campaign’s design by clicking Set Design. When your campaign details, recipients, variations, content, and sending options are complete, click either Send Test or Send Now at the bottom.
How do I remove the variants and go back to a normal campaign?
Delete variants until you only have one left.
How do I send test emails for an A/B Campaign?
Scroll down to the bottom of the Build Campaign page, and click Send Test.
You can choose which variants to send in the test, and click Send Test.
What happens after I send the initial test variants?
Once you send your test variants, the system will send to your chosen sample size and begin collecting data over the time period you set before choosing a winner. You will see your campaign type under Campaigns shown as “A/B Variant,” and the Status will show as “A/B Testing.”
Once the time period is up, the system will pick your winner based on clicks or opens (whichever you chose) and send out the remaining emails.
How do I pick a winner before the timer is up?
If you decide you don’t want to wait for the timer to finish and want to send your campaign right away, click the three dots next to the campaign name and then click Pick Winner:
From here, simply click which variation of the campaign you wish to send out immediately, then click Pick Winner.
Convert to Classic
If you decide you want to change your A/B test campaign back to a Classic campaign, click the three dots next to the campaign and then Convert to Classic. Remember, this can only be done while it’s still a Draft.
