by James Lawson.

17 Feb 2012: With email marketing continuing to grow rapidly, delivering the very best possible email is the only sure way to stand out from the crowd. Just as in direct mail, testing multiple versions is the way to optimise – and to avoid wasting budget on mailers that are fundamentally flawed. We took advice from the UK’s top ESPs on the methods and software features that will maximise email response and income.

Test and test again

“If there is one word we email-marketers use, and use a lot, it’s ‘test’,” says Kate Gowers, Account Director at Adestra. “Email is unique in that it is cheap, easy and effective to test many elements of a campaign, and the results are quick and measureable.”

Before any work takes place, defining the testing process and the metrics to be employed will help ensure that comparisons are valid. Keep adequate records of test findings too – what happens when the marketing manager moves on? This applies to all campaigns: feeding learning from each campaign into the next to determine which general rules work best is even more valuable than campaign-specific testing and will help focus quickly on the most rewarding approaches in future campaign development.

“State and document the hypothesis of the test in advance,” says Paul Gibson, StrongMail’s Head of Sales for UK and EMEA. “This sets expectations internally regarding the anticipated performance of the test.”When starting a new series of campaigns, Cathryn Tuffnell, Digital Campaign Manager at Communicator Corp, recommends testing contrasting treatments to establish what works best for the customer base or a banker list. For example, pitting specific against general or positive versus negative subject lines, or trying warm, “As a valued customer…”, or urgent, “Hurry, only five remaining”, treatments. Personalisation based on previous buying behaviour can also work well in subject lines: “Big discounts on similar products purchased”.

“Only test one thing at a time,” she advises. “If doing an A/B split test on subject line, only test that. Keep the time and date constant and once you have established the subject line that prompted the highest open rate, then move on to test variables.” For those selling internationally, test emails locally in each country in which a company operates. It’s generally better for a native speaker to draft new subject lines from scratch, if not the whole copy.Subject line, call to action and offer are standard test targets as are the landing page, tone of voice, image number, size and location: all these are valid targets for optimisation.

Other possibilities include text length, navigation menu location, call to action placement, and sending time of day or day of the week. Testing is really only limited by the time available.Most good software will offer some testing functionality, typically A/B splits and subsequent reporting. Packages like Omniture’s Test & Target or Silverpop’s suite go a good deal further. The latter’s multivariate evaluation tools mean that multiple emails can be tested simultaneously, allowing users to break the “one-at-a-time” rule. 

Where appropriate, Silverpop can also optimise as a campaign goes out, effectively using the first sends as the testing phase. By analysing the real-time results, the software determines what is working best and so changes the content for subsequent emails. Cleverly, it can also change the images supplied to previously sent, but as-yet unopened emails once it has pinpointed the best combination.Silverpop also has a built-in send time optimisation function. By looking back over historical data on previous opening times, it can infer the best time to send each individual email in a campaign. Client Little Tykes saw a 75% increase in email-generated revenue by picking the right send time.

“Reaching customers at the time set aside for Internet activity meant clickthrough and time-on-site metrics went through the roof,” says Richard Evans, Director of Marketing, EMEA at Silverpop, who also highlights the need to report on a wider range of metrics rather than being obsessed with open and clickthrough.

“Little Tykes saw tiny fluctuations in open and clickthrough, 0.3-0.5%,” he explains. “But when they looked at page views, order rate and revenue, they noticed a huge difference.”Statistically valid samples are essential in every test, otherwise results are inconclusive at best and wastefully misleading at worst. Depending on the size and quality of the database being used, testing for each main segment is desirable and a well-resourced marketing operation will have different treatments for each segment. This might involve variable text or image content within the same broad template or a completely different treatment altogether.

“Know your sample size,” says Gowers. “This may seem obvious, but you do need a decent-sized list to get meaningful results from your testing programme.”Free applications are available online to help work out the valid sample size for a given list and many email packages now have this built in. Another key point is to take a representative 1-in-N selection from your list to test to, not a random one.

Don’t forget the control group either. The champion-challenger approach to improving an existing campaign must also be of sufficient size to offer statistically valid comparison results.“When testing, it is important to measure the incremental lift in the test from a normalized baseline,” says Gibson. “In order to do this, you should always hold out a statistically significant control group to measure against. This allows you to know how the program would have performed when approached in the same manner as usual and then to measure the test against the audience to determine the impact it had on the recipients.”

Separate from evaluating content and list performance is pre-send testing of how email creative will look in various browsers. For years, this has been a standard – and vital – step, but the plethora of new platforms arriving over the last couple of years has complicated the issue massively.

“Testing for rendering and message display across multiple devices is the most challenging and the most essential part of testing,” says Evans. “There were 14 new tablets in 2011 alone.” With tablets and mobiles increasingly the preferred place to read email, checking that messages display as intended is essential. A design style will also vary in effectiveness depending on how it is displayed; mobile clients truncate the “From” name and the subject line is a quarter of its PC-client size. Single-column text layouts tend to be the most versatile, working well across all platforms. 

“Be aware that mobile users use the subject line to ‘triage’ their inbox,” advises Evans. “You also need to make sure the call to action works for a mouse or a fingertip. There should be no need to zoom in.”Pivotal Veracity and Litmus are two popular applications that offer render checking amongst other testing and analytics functionality, and are used by the likes of ExactTarget, Yesmail and Silverpop. Send an email to the Litmus online service and the company sends back screenshots of how it will display in over 30 different devices.Testing by Stoke Park Golf Club and Adestra is a good example of the power of a visible preview subject line.

Tweaking to ensure the call to action was readable in the inbox screen lifted their open rate from 16% to 36%.The other pre-send essential is spam scoring. Good email software will offer this as standard, while the two services above can also vet for spam-like attributes.If the score is too high, there are many options. For example, avoid overusing punctuation or words like free, win or offer as they are well known by ESPs as indications of spam email. Long subjects lines should be cut and reducing the number of images may help too.

“We are regularly sent HTMLs as one long image, which then causes an issue if a recipient has images turned off as they will not view any of the message at all,” says Emily Goodyear, Marketing Manager at WRM-Media. “In these cases, we convert the HTML to text and test both versions. Text versions are always more successful as the majority of the message, if not the design, is not lost.”

Outside of the emails themselves, experts advise advance testing of any bought-in list prior to campaign roll-out to check validity. Here and in the campaign itself, using known seed email addresses will help track and evaluate the real delivery rate of the emails sent. Checking deliverability and response at the last minute using a small sample may avoid extreme disappointment on roll-out.

“We may select a test batch, for instance 100,000 records and assess the results of this, before rolling out an entire campaign,” says Goodyear. “We would compare the nature of the campaign to other similar previous campaigns to anticipate what the data requirement would be to achieve the desired results.”All the advice here applies to b2b as well as b2c emails but the distinctly different nature of the business marketplace must be taken into account. Consumers often buy immediately on impulse.

Business purchases are generally higher value and the buying cycle is far longer, tends to involve more than one person and often requires extensive research on the part of the purchaser. “There are a whole load of factors that can skew the testing process,” says Jamie Gledhill, Managing Director and founder of Emailmovers. “You shouldn’t expect them to bite immediately, it’s a different emotional process.”

According to Gledhill, testing a series of emails is essential to see which ones – or which combinations – tease response from different business buyers. “We use plain text mailers to test before we considering sending HTML and use an autoresponder system that allows immediate feedback from the email rather than trying to force them to click through to a site or fill in a form.”

As well as the number of contacts needed, the creative and proposition are still primary concerns. For example, using a subject line of “Are you sick of late payers?” rather than “Are you owed money?” may well work better when offering factoring services. “You may need to ask in three different ways before they will respond at all,” he says. “The smallest change in the copy can still make a big difference and b2b doesn’t respond as well to HTML as it does to text these days.”

Keep learning

The diversity of the advice above only reflects the infinite variety possible with email marketing. If there is one golden rule, it’s never to stop testing and improving. “There is a tendency for clients to think that once they have tested a campaign they don’t need to test that variable again, but that isn’t the case,” says Tuffnell. “Every campaign should be treated as an on-going test and there is something to be learnt from every campaign dispatched.”