I have a love/hate relationship with testing.
I love being able to do an A/B split test on a headline, block of copy or call to action.
It's great to be able to judge the writing according to actual conversions and sales.
In other words, I'm a big fan of what testing actually delivers.
What I don't like is when test results are poor interpreted.
Let's say I wrote two versions of a headline for a client. And we then tested them.
#1: Sign up NOW and save 37% on the regular subscription rate.
#2: Save 37% on the regular subscription rate when you sign up NOW.
Let's say that #2 outperforms #1 by 15%.
That's a great discovery. And by conducting the test the client make a lot more money from that page.
But here comes the bad part...
A few months later I write some more copy, but for a different product, with a totally different sales page.
But when I submit the headline and copy, the client says, "No, no! Testing PROVES that if you put the savings at the beginning the headline you'll make more sales."
Not true!!!!!
That test didn't prove any such thing. The result was valid only for THAT sales page, when viewed at THAT TIME, with THAT page layout, THAT supporting copy and THAT call to action.
At best, it provides a clue at to what MIGHT work best for similar products under similar circumstances.
It drives me crazy when people take the results from a single test and then extrapolate them across all their future marketing efforts.
That's bad science and bad marketing.
