Improving Email with Testing

Start building a testing program for your  email marketing efforts  today.

Without proper testing, there is no way to tell what your customers are actually seeing or how they are responding to your message. Before you start to test anything, you must first establish a  plan that outlines what you want to test and what you will consider a success or a failure.  Make sure to consider many factors.  For example, will a successful test be dependent on more opens, higher clickthroughs or increased purchases?  Without determining the key metrics for a successful test before performing it, you will be unable to determine which message performs better.

In order to properly test different subject lines or content, you need to verify that your customers are receiving your email and that it is rendering properly, because if they don't receive your email, nothing else matters. To do this, there are a number of valuable tools that you'll want to become familiar with, including a campaign preview tool, an anti-spam testing tool and a seedlisting tool to see if your messages make it to the inbox or junk folder.These tools should be used as part of your regular campaign process, both before and during your send.

Test to see how your message renders
Once your message is fully created and coded, test it in a campaign preview tool. These tools allow you to see how your message will appear in the various email clients. If any issues arise during this test (image rendering, formatting, content blocks, etc.), you then should make the necessary changes and test it again. Without this form of testing, you run the risk of confusing your customer or missing an opportunity to communicate key pieces of information that can result in increased sales and customer satisfaction.

If your solution does not have that capability, just set up various test email accounts that you to can monitor at the various ISPs and send you message to your list that has all of these addresses. 

Test to see where your messages are placed
If your message doesn't make it to the inbox, you are missing out on potential revenue. Plus, having your email land in the junk folder is  damaging to your brand. If you don't test inbox placement during every campaign, you run the risk of doing the same thing over and over, resulting in multiple ineffective campaigns. Pivotal Veracty has a great tool for testing this using email seeds.

Test your content
Content is one of the most fundamental and overlooked aspects of testing. By making easy and small changes to your content, like changing one word in the subject line or highlighting a product feature, you can increase clicks and sales dramatically. In most cases, A/B and multivariate testing capabilities will be embedded into your email platform. If not, you'll want to find a solution that natively supports these critical testing functions. When testing campaigns with different content, you'll want to start by changing one element at a time. This allows you to more easily see what factor is having a direct impact on your email response rate. An easy and effective way to do this is to pull two randomized samples from your entire list. Depending on the size of your list, there are two common ways to perform A/B split tests.

A/B testing
For smaller lists, it is common to split the entire list into two randomized groups and send one version of the email to each of the groups and measure which group had a better response rate for opens, clicks or revenue generated. Next, apply those findings to future mailings. For larger lists where statistical significance is not a concern, many marketers use a 10/10/80 split, but you could also select an exact number of recipients for each test mailing. This method allows you to determine the best performing email and then send it to the remainder of the list for maximum impact. 

Multivariate testing
There is also value in testing a carefully selected combination of factors so that you can determine if any of the test elements interact with each other positively or negatively. There are cases where two best performing elements from earlier tests demonstrate poor performance when combined. However, before you embark on a multivariate testing program, you should recognize that multivariate testing is much more complicated than A/B split testing and requires advanced planning, stringent adherence to the test plan and an extra level of sophistication to interpret the results.

Learn from your results
Be sure to track and document your testing results and share information on what works with key stakeholders involved with your email program. So remember, don't be afraid to test. By understanding your customers better, you will send them more relevant messages, which will increase your ROI.

Comments

[James Hurst]

Hmm, email multivariate testing is less complicated than AB testing. At least, the scores of experiments we have done are. I work for Memetrics and we would be glad to demo you how easy it is to do in batch mode.