-
The release of Google Website Optimizer
-
In October 2006, Google announced the release of its website testing tool, Google
Website Optimizer. Although the tool is less powerful than some of its commercial
counterparts, it allows most websites to introduce testing into their online marketing
initiatives.
-
The recession of 2009
-
As the economy tanked in 2009, many online marketers had to find new ways to convert
their existing traffic into revenue. So, although visitors still mattered, conversions
mattered even more.
Chapters Chapter 3 through Chapter 8 introduced the principles of the Conversion Framework that guide how
a visitor interacts with your website. Each of these elements impacts whether a visitor is
persuaded to stay on your website and ultimately convert, or whether he leaves. The seventh
element of the Conversion Framework asks you to test any change you make against the original
design. By doing so, you can determine the impact these changes have on your bottom
line.
You can deploy two types of tests on a website. Each will vary in terms of the investment required
for deployment, the complexity, and the type of data required before concluding. When
testing, we are always looking for a design that “works best.” How a design works best is
different from one situation to the next. In some instances, you may be looking for a design
that generates more orders. In other instances, you might be looking for a design that keeps
visitors more engaged. We will refer to the better design as the one that creates more
conversions.
A/B tests allow you to test a baseline design against one or more
variations to determine which one converts more. If you are testing more than two designs, you are conducting an A/B/N
test.
Figure 9-1 shows how A/B testing software works. In this example, the original page receives 100,000
visitors. When testing a second version of the page, the testing software directs 50,000
visitors to the original design and 50,000 visitors to the new design. The software tracks
which design persuades visitors to convert. If one design results in more visitors
converting, that is the winning design.
Figure 9-2 shows the original category page design for one of our clients in an A/B test. Figure 9-3 shows the second version of the category page in
the same test. The goal of this test was to increase the click-through rate (CTR) from the category page to the product page. Version B (Figure 9-3) increased the CTR to the product page by 40%
compared to Version A.
Multivariate tests are designed to test multiple elements of a
single page at the same time. Using testing software, you can test different headlines, images, buttons, or any
other elements on a page to measure their impact on your conversion rates.
Figure 9-4 shows how multivariate
testing software works. We used the software to test different variations of the page
headline, the main image, and the call-to-action button on a page. The software
tested:
-
The original headline against three other possible headlines, for a total of four
possible headlines
-
The original image against two other possible images, for a total of three
possible images
-
The original call-to-action button against three other possible buttons, for a
total of four possible call-to-action buttons
This resulted in 48 variations for that one page. The total number of testing
scenarios depends on the number of elements you will test on the page (headline, image,
buttons, etc.) and the number of variations you will test for each element. You calculate
the total number of scenarios by multiplying the number of variations for each
element.
The number of scenarios or variations can grow very quickly. Some software allows you
to test millions of variations of a single page. We do not like such large-scale
experiments for reasons we will address throughout this chapter.
As a visitor arrives at a page, the software displays one of the four headlines, one
of the three images, and one of the four call-to-action buttons. Figure 9-5 shows four of the possible 48
designs the software created. Your team does not have to create all 48 designs; the
software will swap the different variations and create the designs automatically.
Tip
DON’T FORGET!
-
Although it may seem like testing different elements on one page (multivariate
test) is less complex than testing one or more pages against one other (A/B test),
in most cases the opposite is true. This is especially the case when testing
software is used with dynamic websites to insert different variations of
elements.
-
You can always think of a multivariate test as a specialized form of A/B
test.
