Billion Dollar Split Test - Case Study
Posted: April 26th, 2021, 8:05 pm
So the other week I posted a direct response mail piece from a satellite internet company.
I called out that piece for being poorly targeted with the offer - If you're interested you can see it here in this other post
Why is this direct mail piece a failure?
But this time I have something better to share with you...
This next direct mail piece comes from "Capital One" and I think it's a fine example of split testing.
Of course I have no idea if they actually spent a billion dollars on this campaign. But when a company has a 62 Billion dollar market cap it's s safe bet they at least spend hundreds of millions on advertising each year.
Anyway, these 2 pieces came to me about a week apart, and they have a couple of interesting differences.
Can you spot the differences?
First here's the envelopes...
So as we can see there's a slightly different offer with the envelope copy in an attempt to boost open rates
And now here's the main letters...
They're essentially the same. Except they changed the offer at the top of the letter.
One thing I really like is the "reservation" code and an "access" code near the bottom.
This not only gives the offer an aire of exclusivity, but it doubles as a tracking mechanism. (I love when elements have more than one purpose)
And finally, here's the enclosed postcards...
On the postcards they changed the headline and the order of the bullet points.
If I had one critique about this whole package it would be they changed too many elements at one time. Which makes it more difficult tell which factor has the biggest impact on response.
But, giving the benefit of doubt, an outfit this big is probably testing different variables across multiple demographics, and combining the best results with subsequent split tests. (of course that's just a guess because I don't work there).
Anyway, I thought it was a pretty cool example of a real world split test. Now I'll wait to see if one of these shows up again because that usually indicates which one got the better results.
And at the risk of sounding cliche I'll just finish this post with a short quote from the legendary ad man, David Ogilvy...
"Never stop testing and your advertising will never stop improving"
I called out that piece for being poorly targeted with the offer - If you're interested you can see it here in this other post
Why is this direct mail piece a failure?
But this time I have something better to share with you...
This next direct mail piece comes from "Capital One" and I think it's a fine example of split testing.
Of course I have no idea if they actually spent a billion dollars on this campaign. But when a company has a 62 Billion dollar market cap it's s safe bet they at least spend hundreds of millions on advertising each year.
Anyway, these 2 pieces came to me about a week apart, and they have a couple of interesting differences.
Can you spot the differences?
First here's the envelopes...
So as we can see there's a slightly different offer with the envelope copy in an attempt to boost open rates
And now here's the main letters...
They're essentially the same. Except they changed the offer at the top of the letter.
One thing I really like is the "reservation" code and an "access" code near the bottom.
This not only gives the offer an aire of exclusivity, but it doubles as a tracking mechanism. (I love when elements have more than one purpose)
And finally, here's the enclosed postcards...
On the postcards they changed the headline and the order of the bullet points.
If I had one critique about this whole package it would be they changed too many elements at one time. Which makes it more difficult tell which factor has the biggest impact on response.
But, giving the benefit of doubt, an outfit this big is probably testing different variables across multiple demographics, and combining the best results with subsequent split tests. (of course that's just a guess because I don't work there).
Anyway, I thought it was a pretty cool example of a real world split test. Now I'll wait to see if one of these shows up again because that usually indicates which one got the better results.
And at the risk of sounding cliche I'll just finish this post with a short quote from the legendary ad man, David Ogilvy...
"Never stop testing and your advertising will never stop improving"
