What Is A/B Testing?
In the world of marketing and advertising, sometimes it can be very difficult to tell what images or word choice will resonate with your audience. Even with the best market research, it can be difficult to pinpoint the precise message or call to action that will make your customers take notice.
One method marketers use to make more effective marketing choices is A/B Testing. Let’s take a look at the best way to execute, measure and optimize your testing.
Step One: Determine your goals and methods.
With the advent of online advertising and metrics, it’s become much easier to measure your advertising success. This makes it even more important to determine what you want to test and how you will measure the success of your test.
Here are some of the different metrics you can test:
- Audience: Demographics that you are targeting
- Effectiveness: Click-through rates, open rates (for emails) conversions, sales, requests for more information or phone calls
- Geography: Location of customers, clicks or sales
After determining what you need to test, you’ll want to form some hypotheses or assumptions around these elements. For example, you may want to test the best call to action. One ad may list a phone number and tell readers to call in; the other may list a website and instruct them to sign up online. Before you begin testing, you should predict which one will generate more response and why. Then, once the test is finished, you’ll be able to see if your assumption was correct.
A word of caution: be sure to tightly control your results by testing just one thing (e.g. only the call-to-action, only the image, only the headline). If you change more than one element in a test, you won’t be able to determine which change actually affected the results.
Step Two: Set a budget.
While you want to be sure to get good results from your test, you also don’t want to spend too much money or spread your test results too thin. It’s important that you find a happy medium between testing enough variations and directing enough traffic to each version, all while staying in budget. If you don’t direct enough traffic to each variation of your test, it will not generate enough data to provide significant or reliable results.
Analyzing the action of 100 viewers will provide less accurate results than analyzing the actions of 1000 viewers, and tracking customer activity for an entire week will provide more insight than tracking activity for a single day. However, directing more traffic for a longer period of time will also be more expensive. Set a measurable budget that allows you to test enough people for a long enough period of time to get significant results.
Step Three: Determine what you want to test.
You can apply A/B testing to many different marketing initiatives, including:
- Paid advertisements
- Email campaigns
- Landing pages
- Print ads
- Retail strategy (read more here)
Step Four: Decide what elements to test.
Depending on what marketing initiative you are testing, there are many different elements you can adjust for testing purposes. Just remember that, with A/B testing, it’s important to test a single thing at a time to get accurate results. Here are some elements you can test:
- Subject Lines (email-specific)
- Buttons (shape, color, size)
- The offer itself (for example, you can offer free shipping or a 10% discount and see which performs better)
- The call to action
- Overall layout, template or design elements
Step Five: Determine when you want to test.
It might be best to test radical changes (like an entirely new email template) during non-peak business months when you have more time to analyze the results and won’t be risking some of your more important marketing initiatives. For other elements of an email, like the subject line or the call to action, you might choose to run A/B tests for almost every campaign. The more you can learn about what your subscribers respond to, the better your emails will become over time.
When it comes to testing advertisements, it will be up to you to determine when (and how long) you want to run a test. In paid advertising, customers may behave differently depending on the time of day, the day of the week or the time of the month (for example, more people may be willing to purchase after payday). While it may not be feasible to run a test for an entire month, come up with some hypotheses on how your target customer will be spending money at different times, and test those hypotheses.
Step Six: Analyze your results.
The best way to measure the results of your online ads or landing pages is to leverage analytics tools. As for email testing, most email marketing services provide internal analytics for tracking and analyzing different subject lines, layouts and more.
Don’t forget to look at every aspect of the results. Here are some questions you can ask to get started:
- Who saw these tests?
- Who responded to them?
- What regions, ages or market segments performed better?
- When did customers see the tests, and were there different results based on the time or day?
- Which elements resulted in more clicks or more engagement?
It’s important that you set goals and make predictions before running A/B tests and then use the results to determine how your hypotheses played out. Although your predictions may be wrong, sometimes you will discover an insight where you least expect it.
Step Seven: Implement your findings.
Once you’ve determined which advertisement, email, design or approach worked best, you can implement those changes in your future initiatives. However, don’t get too comfortable; audiences change, so it’s important to keep testing and re-testing different elements to keep refining your approach.
With a little forethought and planning, A/B testing can be a very effective and efficient way to figure out the best method to advertise and reach your customers. And with the ability to track and record just about every online interaction, it’s never been easier to understand and interpret results.