Stephen Pratley – a CRO consultant at The Conversion Co – kindly agreed to answer our questions about A/B testing in marketing and why it is important to do it before launching new features.

Stephen Pratley delivers a talk at Outreach Digital

Let me ask you some general things thirst. What is A/B Test & What can you A/B test?

An A/B test (also known as a split test) is a way of trying out changes on a website, ad campaign, email, or any other marketing asset.

I’ll focus on testing web pages here for simplicity.

Importantly, it “splits” the traffic to each version of  your page. Version A, then Version B, A, B, A, B, hence the name.

What this does is make sure that any other influences, like the weather, the day of the week, big news events, are the same for both versions.

Why is it essential to do A/B test before launching a product/ new feature?

Every new feature has a cost. 

It can mean a slower site, a more complex decision for the prospect, or another feature being bumped down the page.

Testing before launching means you know if the benefits of the new feature outweigh it’s costs.

Where do you start when doing A/B test? What are the first steps?

There are two big things that will give you ideas for what to test.

For an existing site, try to uncover the little hurdles and friction in your site that you can remove. We use tools like Hotjar a lot to watch what people are doing and we do lots of user tests where we can get feedback as users go from page to page.

The other end of the equation is to find out what your customers REALLY want. Uncovering their real motivations can help you highlight features, and change little things like headlines, which can make a massive difference.

Unstructured research like that should be shared with your copywriters and as much of your marketing team as possible. Most companies don’t do nearly enough of it.

When you have some ideas for what to test, you need to prioritize.

Tests need traffic, so for most smaller businesses they’ll focus on where they have the most traffic – at the top of their funnel.

That means emails, advert creatives and landing pages.

If you have more traffic, then tests at the end of the funnel, can have a bigger effect. Tests on pricing will have the biggest impact on your profits, but they’re the hardest to do.

What was the most unexpected result after doing A/B test? Any unpredictable results? Outstanding cases?

Pricing always surprises us a lot. I’ve seen loads of cases where increasing pricing increases the volume of sales made, not just profits.

Something about the perception of value changes when you play with price.

Photos can also have a huge effect when you hit on something that works. I put one in of a client giving a TedX talk and we’ve not beaten that in 2 years of testing. That’s very rare in online.

Something about the authority it gave us was incredibly powerful.

Stephen Pratley delivers a talk at Outreach Digital

Would you recommend any articles, webinars, books to learn more about A/B test?

The place I started from was when I read “Don’t make me think” by Steve Krug.

It really taught me how much you can improve a site by making it simpler, not by adding more to it.

Another one which will give you some great ideas is “108 Split Tests” by Russell Brunson. A lot of people write him off as a bit spammy, but Russell is one of the best Direct Response marketers working today.

When it comes to implementing the tests, the websites of the tools that you use are the best place.

Stephen Pratley delivers a talk at Outreach Digital

Google Optimize has a great resource centre:


Lastly, follow @conversioncohq on Twitter where we share insights and a few of the test results that aren’t kept secret by our clients.

We’ll be launching some short courses in the new year that will get beginners from an idea to their first test results.

Thanks Stephen for the interview.

Interviewed by Ekaterina Korchikova.

Read related posts below

It’s About What You Know AND Who You Know (How to Get Your First Client)

“It’s not about what you know, but who you know”, as the old adage goes.  When it comes to getting your first client, whether as a freelancer or small business, this is what you’ll hear a lot...

10 Video Lessons on Personal and Professional Development

In current times full of uncertainties, we could either sit and worry or grow! Quit TikTok for now. OD carefully curated 10 development lessons to up the quality of your screen time. Although this...

Key Principles of A/B Testing [interview]

Stephen Pratley - a CRO consultant at The Conversion Co – kindly agreed to answer our questions about A/B testing in marketing and why it is important to do it before launching new features. Let me...

Copywriting is your sales executive: Steven Kent, CXL Certified Optimiser

Steven Kent has been a man of words for more than a decade, helping clients from dynamic start-ups to global brands like Microsoft, Intel and Sitecore. Taking us behind the scenes of content...

10 tech startups at the forefront of innovation

The ultimate listing of ground-breaking tech startups, as recommended by the founding editor of WIRED UK David Rowan David Rowan at outREACH Conference 2019 ©️...

SEMrush on Social Media Marketing

On Monday September 5th, we welcomed Olga, the Head of Social Media of SEMrush, to share some hands-on experience and actionable tips on social media integration with our members. As...

Data scaling and Digital Profiles

Recently, Florian Douetteau (CEO, @dataiku) explored how to overcome various technological and HR-related challenges faced by the industry with our members. He touched upon the topics like how...

Moneyball Marketing Attribution

What do marketing attribution and baseball have in common? Clue: It's not the pinstripe uniforms. Scoring a direct conversion on your website within just one visit is the marketing equivalent of a...

Machine Learning for Digital Advertising

When using TV, radio, or street banners for our company marketing, it is difficult to assess what in our campaign is working, and what is not. But via digital marketing, we can access a large amount...