How do you use matrix AB testing?
On the main screen, or campaign screen, choose the option “Create: A/B Matrix”
In the next screen, define what you wish to evaluate in this campaign.
In this example, we have chosen to evaluate 2 subject lines, 2 from names and 2 different bodies (campaign text). In this example, we will click the ‘+’ sign for each box: “subject line”, “from name” and “body”.
The combinations among these options will create 8 different versions for the campaign:
2 content versions will be sent under 2 different subject lines. This combination will create 4 versions. These will be sent under two different ‘from names’, so that in total 8 versions will be available for the campaign. After you have selected the different values in the options, define the scope of the test group at the bottom of the page, under the title “Split Rate”.
The test group is taken from the group of contacts to which the campaign shall be forwarded.
In this example, we have defined the scope of the test group at 40%. Each contact person in the test group shall receive only one version of the campaign.
After test results are obtained, the winning version shall be sent to 60% of the contact people remaining in the selected contact group.
Alternative option is to send the campaign on two different dates, to test the optimal delivery time:
When selecting this option, the system will send the campaign to half of the contact group at one specific time (for example, 10:00 am), and to the other half of the contact group at another time (eg. 22:00 pm).
With this option, there is no option to select the scope of the test group, because the campaign tests optimal delivery times. When the delivery is completed, you can analyze and determine the most efficient delivery time (according to quantity of openings and clicks for each option), and use this information for your future campaigns.
How is the campaign’s winning version determined?
At the bottom of the screen, under the title “how to determine the winner”, define the measure for determining the winning version. You can select from among percentage of openings, percentage of clicks, or a manual option which you choose the winner on your own.
After the system computes the winning version, it will be sent to the remaining contacts in the group selected (in our example, to the remaining 60% of contacts).
You can also set the winning version to be delivered a few hours or days after delivering the campaign to the test group.
With the manual option, the system will not send the winning version automatically, and you have the option of selecting the winning version according to the clicking and opening data presented, and send the winning version of your choice, independently.
In the next screen, fill in the subject lines and different sender names.
Under “Campaign name (internal)” you can enter the name of the campaign, for internal use only. In other words, it will be visible to your eyes only.
After filling all the fields, click on “continue”, and move on to the screen where you will edit campaign contents:
Click on the first body to start editing, when finished, you will be taken back to the “Body To Edit” screen. Then, you can edit the second one.
After two campaigns have been edited, you will be moved on to the next screen for selecting the group of contacts to which you will send the different campaign versions.
After selecting the group, click on “continue” to move on to the next screen for timing and confirmation of campaign delivery.