Billion Dollar Split Test - Case Study

Dissect & discuss great (and not so great) copywriting examples
Post Reply
User avatar
SARubin
Copywriter
Posts: 557
Joined: January 12th, 2019, 8:00 am
Location: USA
Tagline: Spinning Words Into Gold
Has thanked: 120 times
Been thanked: 108 times
Contact:

Billion Dollar Split Test - Case Study

Post by SARubin »

So the other week I posted a direct response mail piece from a satellite internet company.
I called out that piece for being poorly targeted with the offer - If you're interested you can see it here in this other post

Why is this direct mail piece a failure?


But this time I have something better to share with you...

This next direct mail piece comes from "Capital One" and I think it's a fine example of split testing.

Of course I have no idea if they actually spent a billion dollars on this campaign. But when a company has a 62 Billion dollar market cap it's s safe bet they at least spend hundreds of millions on advertising each year.

Anyway, these 2 pieces came to me about a week apart, and they have a couple of interesting differences.

Can you spot the differences?

First here's the envelopes...

Cap1point5offer-envelope.jpg
Cap200cashbackoffer-envelope.jpg

So as we can see there's a slightly different offer with the envelope copy in an attempt to boost open rates

And now here's the main letters...

1point5rebate-offer.jpg
200cash-back-offer.jpg

They're essentially the same. Except they changed the offer at the top of the letter.

One thing I really like is the "reservation" code and an "access" code near the bottom.

This not only gives the offer an aire of exclusivity, but it doubles as a tracking mechanism. (I love when elements have more than one purpose)

And finally, here's the enclosed postcards...

liftcard-1point5-offer.jpg
liftcard-200cashback-offer.jpg

On the postcards they changed the headline and the order of the bullet points.

If I had one critique about this whole package it would be they changed too many elements at one time. Which makes it more difficult tell which factor has the biggest impact on response.

But, giving the benefit of doubt, an outfit this big is probably testing different variables across multiple demographics, and combining the best results with subsequent split tests. (of course that's just a guess because I don't work there).


Anyway, I thought it was a pretty cool example of a real world split test. Now I'll wait to see if one of these shows up again because that usually indicates which one got the better results.

And at the risk of sounding cliche I'll just finish this post with a short quote from the legendary ad man, David Ogilvy...

"Never stop testing and your advertising will never stop improving"
A good marketer knows how to think like a marketer - A great marketer learns how to think like the customer...
SARubin - Direct Response Copywriter / Conversion Flow Expert
User avatar
Wordsmith
Member
Posts: 27
Joined: March 24th, 2021, 1:59 pm
Location: Tallahassee, FL
Tagline: It's In The Research!
Been thanked: 5 times
Contact:

Re: Billion Dollar Split Test - Case Study

Post by Wordsmith »

That looks like Shawn Stone's work.

He is a Senior Copywriter for Capital One.
User avatar
SARubin
Copywriter
Posts: 557
Joined: January 12th, 2019, 8:00 am
Location: USA
Tagline: Spinning Words Into Gold
Has thanked: 120 times
Been thanked: 108 times
Contact:

Re: Billion Dollar Split Test - Case Study

Post by SARubin »

And The Winner Is...

$200 One-Time Cash Bonus Offer

Here's the latest offer they sent. The sales letter was the same as the last one (so I won't post it again) But these guys are real pros and they're not finished testing yet.

This time the envelope is jet black with raised lettering. You can't tell from this picture, but the lettering is silver colored which give it a certain sophisticated flair...


cap1directmailfollowup.jpg

Almost can't wait to see what they send me next time...
A good marketer knows how to think like a marketer - A great marketer learns how to think like the customer...
SARubin - Direct Response Copywriter / Conversion Flow Expert
User avatar
Franklin
Copywriter
Posts: 221
Joined: January 27th, 2019, 12:00 am
Location: Right in front of you
Has thanked: 45 times
Been thanked: 76 times

Re: Billion Dollar Split Test - Case Study

Post by Franklin »

I would sign up just to get the $200. I pay my bill in full every month so FREE money :D
“There are 3 kinds of people in this world. Those who can count, and those who can't”
Albert7
Newbie
Posts: 2
Joined: June 27th, 2021, 8:35 am
Location: Biella - Italy

Re: Billion Dollar Split Test - Case Study

Post by Albert7 »

Grazie!
really interesting
Share This With Friends
Post Reply
  • Similar Topics of Interest
    Replies
    Views
    Last post