Everything You Need to Know About A/B Testing

June 1, 2021
| by David Miles

What is A/B Testing?

A/B Testing or ‘Split Testing’ is a method used by online marketers and UX specialists to compare two options for one outcome. The idea is to see which of the two options achieves better results.

Most of us wish we could use A/B testing in real life. Wouldn’t it be nice to be able to watch two different versions of one scenario before choosing which to go with? We’d always be one step ahead and be able to decide the option that will turn out best for us in the long run.

Well, unfortunately, we can’t do that in real life. But in the marketing world, it is highly encouraged. A/B testing helps marketers find the best ROI (return on investment) before backing a single option.

A simple example of A/B testing is when you try out two different web design variations. Each customer will be shown one of two different landing pages, and the results will showcase which design is more effective.

A/B testing can provide actionable data for any business — big or small. The method is there to help you improve your customers’ shopper journey and increase revenue and conversion rates.

Examples of things that can be tested by split testing include:

  • Which layout is better for the product page?
  • Which taxonomy is easier for the customer?
  • Which call-to-action button provides a higher clickthrough?

By testing small details like this, you’re bound to get easy to interpret results that can lead to higher conversion rates in the long run. Using actionable data, you can feel how your customers react to certain scenarios and what needs to be changed to make them happier. Happy customers = more engagement = more click through = more revenue. Simple.

Who Uses A/B Testing?

Marketers, web designers, business owners, everyone! The largest companies in the world are constantly A/B testing right under your nose. Amazon continuously tests different homepages to see, which has the highest clickthrough to the product page. They also test product pricing and ads for different brands.

Netflix tries to encourage you to watch different shows by changing their thumbnails, modifying genre names, and switching up the featured shows to see what persuades you to stay on their app for longer.

Google, the master of marketing and indecisiveness, test the minute things you won’t even notice. In 2000, the search engine tested ‘how many results are the right amount per page?’. They’ve also been known to do over 70 tests on ‘which is the right color blue?’ And you thought you were indecisive…

Split testing is also a huge tool for election campaigns in the modern world: Changing ads, buttons, and much more to see, which converts voters or increases donations.

This quote from Jeff Bezos, Founder of Amazon, explains why A/B testing is so vital in today’s online society: “Experiments are key to innovation because they rarely turn out as you expect and you learn so much…If you can increase the number of experiments you try from a hundred to a thousand, you dramatically increase the number of innovations you produce.”

How A/B Testing Works

First, you’ll need some clear objectives and goals. What is it that you want to achieve? Goals are set with large picture outcomes in mind, and testing the smaller details can help you reach that.

Next, what is it you want to test, and why? Do you have a gut feeling that if you changed your Facebook page button from ‘click here’ to ‘shop now’ you might get more clicks? Why not test it. Can’t decide whether to have 4 products per row or 5? Test it.

Before attempting an A/B test, you’ll need to get an idea of your baselines, too. For this, you can use a tool like Google Analytics to understand the current behaviors of your customers. If your goal is to increase clickthrough rates on your landing page, what is your current rate?

Once you’ve got into the details of the goals and objectives and know your baseline, you’ll need to set ‘key performance indicators’ (KPIs), which will help you define success or failure after the test. If the test’s objective is to sell more of a specific product, the obvious initial KPI will be the number of sales.

Unless you want to test your hypothesis on specific demographics, you’ll need to split audiences into two sets at random.

After you’ve run a test, it should be easy enough to see which option was the most successful. From this, you can either go forth this the successful option or tweak the test and try again to get a more definitive answer (recommended).

The Challenge & Benefits of A/B Testing

Of course, as with everything in life, there are both challenges and benefits.

The Benefits

It saves you a lot of money

With this testing method, marketers and web developers can identify processes or designs that bring around the best return. That way, there’s less risk of putting large sums of money behind a certain project or idea. With the help of the data, businesses can put their money where it’s going to make the most sense, rather than taking a blind leap of faith and hoping for the best.

It increases profits

A/B testing increases your profits in a similar vein by helping you make definitive decisions that you know will deliver. 60% of businesses believe A/B testing improves conversion and leads to higher revenue.

It’s easier to identify current issues

Using an A/B test can help you find small errors within your current processes. By identifying these errors, your business can run smoother. AB testing tools online can help you recognize these errors within the marketing campaigns so that you can create more effective tests and marketing schemes.

Testing is great for UX specialists as it can quickly help ascertain errors in a web design that deter customers.

It provides a style guide to improve the content

By testing out colors, pictures, fonts, etc. you can gain a better idea as to which aesthetics are most effective for your customers. With this data, you can create your team a style guide for future campaigns or designs which have a proven track record of working for you.

A/B Testing Boosts Business Image

70% of companies are said to run two or more A/B tests a month. In doing this, businesses can constantly improve their websites and remove the roadblocks that are stopping visitors from converting into customers. By ridding themselves of these unnecessary processes, customers will have a better impression of the company and build loyalty.

Higher Engagement

A lot of companies incorporate testing into their email marketing so they can understand which subject headers are working the hardest to increase sales. Using this data, the business can adapt their content to draw in more engagement.

It improves KPIs

A/B testing can have a huge impact on your KPIs in the long run. From a return on ad spend, customer acquisition, or something as simple as email signups.

The Challenges

It’s quite difficult for beginners

You can start off with really small and specific tests, but to use A/B testing effectively, you’ll need practice and help to start with. There are a wide selection of online tools and platforms that can help you with this but it’ll be a steep learning curve.

It’s a trial-and-error process

The first test you try isn’t likely to make a massive difference to your revenue. You will no doubt have to test and test and test to impact your conversion rates. Committing to A/B testing is a long-term thing, and giving up after the first couple of tests isn’t going to provide you with the results you’re looking for. It’s best to make a long-term plan for testing and sticking to it.

You need to have traffic already

If you’ve just started your online business, A/B testing is not the tool for you. Sure, it might provide some interesting insights but if the audience base is only ten people, you can’t take this data and make huge decisions on it.

If you’d like to get started with A/B testing early on in your development, try to use some paid advertising to bring traffic to your site, to begin with.

How to Use A/B Testing for Your Business

A/B testing for businesses is all about making the shopping experience better for customers. Anywhere you might be able to remove friction for the customer should be looked at and studied.

  • There is a vast range of ways you can use A/B testing for your business:
  • Determining the best calls to action
  • Understanding if your web layout works
  • If your navigation is easy to use
  • Ascertaining the right prices
  • Which products to cross-sell
  • Which filters work best for product pages
  • What media is most effective?

The beauty of testing is that you can literally do as many tests and you like, and work through the long list of things you want to test systematically to get results.

No matter what you are looking to test, you can refer to the ‘DUMB’ objective process — anything you test needs to be Doable, Understandable, Manageable, and Beneficial.

Using an external tool will help you create clear and specific objectives that deliver results. If you’re not sure about the test’s specifics, such as how long to run the test for, and what a good audience size is, you can use tools like Optimizely’s Sample Size Calculator to give you a starting point.

In terms of how long you should run your test for, unfortunately, there’s no right or wrong answer. Email marketing tests will quickly provide results in the first 1–2 days — as you’ll be looking for the average open rate. But if you have low traffic, and want to test a layout on your website, you might wish the test to last a week.

There’s so much to discuss and talk about at length when it comes to A/B testing that’s there’s just not enough time in the world. Perhaps we should A/B test how much information someone needs to know about the test before they can successfully do their own?

Automate Returns & Exchanges

Related Posts