In this series of posts around conventional wisdom in email marketing we look at a common misconception around how the success of email campaigns is measured.

This has serious implications as teams are either dramatically under reporting email so it gets starved of investment, or completely over reporting the success and therefore failing to address the real challenges for email marketing.

What do people get wrong about the value of their email marketing?

So who has heard something like this at a weekly trading meeting for the marketing team?

“Last week’s email campaign generated £72,000! Well done everyone!”

This is something we try and challenge all the time, how we measure not just email marketing but all online marketing channels.

Where email revenue is under-reported

Most people think email marketing should get 100% of the credit for a sale if someone clicked on the link in your email and then completed the purchase. Again, a reasonable assumption. But the assumption is based upon the idea that the email only influenced someone to buy if they opened it and clicked on your lovely big call to action.

What if they received your email, perhaps saw something they like but went back later without clicking? What if just seeing the email in the inbox reminded them you existed and then they went online later and bought?

Well these seem far-fetched compared to conventional wisdom that Google Analytics is the source of all truth right? Well wrong – and here are 2 pieces of evidence.

Firstly, next time you send your email to your main database keep a random portion back. Then, ignoring who opened or clicked compare those who were sent the email and how much they purchased to those that don’t get sent the email. The difference between them is your incremental revenue which can only be associated with the one difference of the email being sent. This is often 20-40% more than Google Analytics will report.

Even if a recipient does not open the email just seeing your brand in the inbox can increase the chances of them buying.

Where email revenue is over-stated

Secondly, it can show the other way. A little-known setting in Google Analytics is the campaign tracking cookie timeout length. This records the last campaign visit or search the customer has made. So, if the last click was a PPC search then the cookie will be reflected with this. This is then used when reporting on revenue to say which source generated the transaction.

However, did you know this cookie is not reset if the customer comes back direct without the aid of a search engine or campaign? This causes a lot of issues on repeat purchases where the customer comes direct each time, but because this doesn’t reset their cookie all their future purchases get assigned to the original source.

By default the cookie only resets after 6 months so potentially you can have a customer click on a link in the email and not buy to set the cookie. But they could come back direct several times over the next 6 months to place orders, but visit by typing in your website name each time. This won’t reset the cookie so every one of those purchases is going to be awarded to the original email campaign they clicked on.

You might not think this is a huge issue for you but certainly in industries where there is a high frequency of purchase I have personally seen last click attribution errors into the millions of pounds per year for every channel.

It suits a lot of people for this to be left as conventional wisdom. Google being the primary benefactor as this will inflate the reported revenue from PPC, often more so than email (especially if you have brand PPC terms turned on).

Getting Google who are selling you the PPC ads to provide all online revenue measurement without realising the massive conflict of interest is a mistake. A cynic would suggest this is why Google Analytics is free.

Email is easier to generate revenue for than other channels

A third way conventional wisdom is wrong on how we measure email is to do with lifetime value. Not all purchases are equal, and the first purchase a customer makes is far more valuable than the others in the sense that they are likely (you hope) to return and spend again.

As much as PPC marketers have it hard in keeping the cost of acquisition down, this is nearly always offset and more by the fact they bundle in sales they have stolen from other channels beyond the first order, either with measurement gaps like the above GA example or because they get easy sales from things like brand search terms or poorly targeted retargeting campaigns that bring back your active customers who were shopping with you anyway.

Yet email marketers often have it easy as they are always measured against fellow members of their team whose job is to acquire customers. Clearly generating repeat business through active customers is going to be easier so should email marketers be pushed harder? Most email marketers are not measured on KPIs like lifetime value, repeat order rate or churn. This lack of focussed reporting means all of their sales are treated equally.

Email marketers should in an ideal world not be directly measured on revenue but on how many lapsed customers they have brought back, what percentage of first-time customers have turned into second purchasers and other key CRM numbers. It is an uplift in these types of numbers that will grow the bottom line of a business more than any other.

The worst culprits for terrible email measurement...

And my final bug bear with measurement – cart recovery.

There are a few approaches to how cart recovery vendors do attribution. Mainly they claim anyone who was sent a cart recovery email, and then subsequently buy within a certain number of days is 100% down to them. Their revenue reporting figures and ROI claims reflect this.

To say this is wrong is a dramatic understatement, and in some instances I see borderline fraud and lies with these claims.

On average we see 12% of those who abandon their shopping carts return and complete a transaction when we don’t send an email. For those who we send an email it might be something like a 15% return rate. Therefore, email has really only delivered the difference of 3% of these sales.

Where it really hurts is if you are paying commission on these recovered sales, especially as often it is sold in as low risk as you only pay for recovered orders (I.E. Anyone who received an email and bought). If you are paying at the 15% return rate, on say an average order value of £50 and commission of 7% for every 100 orders, your ROI is claimed as:

100 orders x £50 = £5,000. 7% commission is £350, an ROI of 1328%.

Actually, it is this:

3% of 100 orders = 3 x £50 = £150. 7% commission on the whole 100 orders is still £350 gives you a net loss of £200 – and that’s even before margin on the sales is taken into account.

Not looking so low risk now is it?

Even on a fixed price price licence deal you see crazy ROI figures banded about when there isn’t anywhere near to the incremental uplift as claimed.

We overcome this by always trying to have a control cell for at least a period of a few months to show the incremental uplift in campaigns. It isn’t always possible as the downside of control cells is you need a certain level of volume to make the test statistically valid, so even when we can’t use control cells we can at least make assumptions based upon typical return values and not fantasy numbers of make believe.

The golden rule - is it incremental?

If I was to summarise the top 4 actions we think brands should undertake we would suggest these:

1) Think Incremental: Wherever possible use control cells to prove the incrementality or otherwise of the activity.

2) Percentages vs. Total Numbers: Don’t obsess over percentages such as open and click rate. You 100% of not a lot is always going to be not a lot. Instead think about the overall reach of total number of unique openers and clickers. This will help you to realise large unresponsive segments of your base will deliver tiny percentages but add up to a significant uplift of overall numbers.

3) It’s not like acquisition channels: So don’t measure it in the same way. The CRM function should be about increasing lifetime value through retention, reactivating lapsed customers and increasing the frequency of spend of active customers. Only when you have KPI’s targeted towards these will you create strategies to improve each of them.

4) Change the Google Analytics campaign cookie length: we see no legitimate reason why a marketer would want this at 6 months unless they want to artificially boost certain channels by stealing direct sales.

‍