Want to make your marketing more effective? Of course you do.
A/B testing helps you find the most effective route – it’s a way to experiment and compare two different approaches before you release the winning approach to your entire audience.
A/B Testing is great for landing pages, ads, email and more: run an experiment, see which approach brings the results, run another, choose the best combinations and use those to drive growth!
It’s always a good idea to try different approaches and experiment with your marketing communications.
Well, this guide will show you how, so you can choose all the best elements that work together to really build growth and results.
You can do any number of tests, but with A/B testing, just make sure to vary one factor at a time (the thing you’re testing). Keep everything else the same, and then you’ll know what difference that one factor has made, before you move on to the next test.
Finding which approach works best is worth the effort. True A/B testing always has a knock-on effect – if one call-to-action performs better than another for example, and you roll out the better call-to-action, you’re likely to get a lot more clicks. What if these clicks were adding product to a basket? Or completing a purchase? You could have more sales. That’s the bottom line. A/B testing helps bring results and it can be done on a small budget.
Basic A/B testing can be relatively straightforward to implement. Depending on what system you use for each channel, there are usually ways to A/B test with minimal technical knowledge.
Imagine you run an email test. You create two emails that are exactly the same, except for the subject line. Each email is sent to 5000 people.
These are the results:
In this test, two different subject lines were used, resulting in a 6% difference in the open rate. The content in the email was the same apart from the subject line and we see that the click thru rates were the same. However, because more people opened email B, the result was 12 more people visiting your site.
But is 12 people enough of the sample size to be a significant result?
Always use a statistical significance calculator to see whether the results are significant.
We love this one: AB Test Calculator
For the test above, visits generated is your conversion in the calculator. There was a 0.8% conversion rate for email B versus 0.56% for email A. That doesn’t sound like a lot, but it’s actually an uplift of 43%.
Running the conversions through the test calculator shows you it’s a statistically significant result and therefore it’s worth using email B’s subject line in the future – email B will drive 43% more traffic to your site than email A.
A/B testing here has delivered a meaningful result which can influence your decision on which email to send out to the wider audience.
Just think of the knock-on effect on your conversions – it’s worth it!
Kate Fairhust, GrowthMinds CEO and co-founder
Standard A/B testing involves testing the effect of varying one thing (the headline, or the location of a CTA for example). Usually taking a sample of the whole audience for a particular channel, and using the results to inform future activity.
If the audience is too small, you might not be able to test and get a statistically significant result, or it might not be worth running from a revenue perspective.
For example, if your email list is less than 1000 people, it might not be worth your while running a test to 10% or 100 people, because the difference in results might not be significant. Instead, you might have to go for a 50/50 test to your whole audience. Then as your email list grows, you can reduce the percentage of your audience that receives a test.
Hypothesise what the results might be using current performance where you can.
A/B tests and experiments don’t have to blow your budget, but when you’re planning, if your test involves paid advertising, make sure to check you have the budget to get the reach you need to prove the results are significant.
And don’t forget to work that into your plan. Maybe you don’t make a return on investment from the experiment. That’s OK. You’re proving a hypothesis. What’s crucial is that it delivers ROI when you scale it. If it doesn’t, it’s probably not the best test to run.
If you’re unsure: give us a call. We’re always happy to help.
And we can definitely help with more sophisticated A/B testing requirements.
There’s a bunch of tools out there to help your A/B testing. We’ve listed a few here to get you going here for websites, emails and ad campaigns.
These tools give you on-site testing with inbuilt reporting.
Most email systems have inbuilt tools to help you set up A/B tests. Some (like MailChimp) offer simple 50/50 tests, whereas more expensive systems (like Hubspot or Marketo) give you the opportunity to run more complex tests with holdout groups and automated winning email roll outs.
You could spend hours messing about with different colours for your Call-to-Action buttons, but you’re probably more likely to see an uplift by changing the wording or focusing your audience on the action you want them to take by simplifying a page.
What works for one company might not work for another.
Take a look at the pages on your site that aren’t converting as well as you’d like and run a test.
Tim Hanson, GrowthMinds Growth Hacker
GeoSpock approached GrowthMinds to develop and test some custom landing pages. We developed a new page to test, which featured a video alongside the lead form. Using heatmap technology from Hotjar, we could instantly see the difference.
The images below show the original page (left) and the test page (right) and the heat marks represent where a visitor spends time looking at the page.
The video focused audience attention on the page and drove engagement.
And the results were impressive:
The original page (without the video) converted at 3.63% (below average) The new page (with the video) converted at 8.27% – a 128% improvement!
GeoSpock are now driving traffic to the new page.