Figure 9-6. Original shopping cart design for site selling nurses’ uniforms, without assurance
center
Should you start with an A/B test or a multivariate test for a particular page? There is no universal answer to this question. A/B tests are good for testing alternative designs of an entire page or process. They are very helpful in determining which high-level changes have the most impact on visitors. Multivariate tests allow for granular testing of a page. They are helpful in determining which elements have the most impact on visitors.
If you have a brand-new page or a limited number of visitors or conversions, we recommend starting with an A/B test. This will let you test major design changes. If you already have an existing page, we recommend starting with a small multivariate test (fewer than 12 different scenarios). The goal of this initial test is to determine which elements (headline, image, benefit list, etc.) resonate most with visitors. Analysis of the first test results will help guide the need for further multivariate or A/B tests.
A successful test starts with a good plan. Consider each of the following elements when creating your first test:
Most analysis focuses on pages with high bounce and exit rates. This is a simplistic approach to a complex problem and rarely yields significant increases in overall conversion rates.
Although your site may have 300,000 visitors per month, a particular test page may receive far fewer visitors during that time. Your goal is to run the tests for fewer than six weeks.
With most testing software, you will need a minimum of 200 conversions per month for the test to conclude within the same period.
Not every test should have the goal of increasing the macro conversion rate. Many successful tests help to increase micro conversion rates. This is particularly important if your website or landing page does not receive enough conversions. Starting out with micro conversion tests allows you to still conduct tests on a smaller scale.
Not all elements on a page will have the same impact on your conversion rate. Determine which elements will have the most impact on your bottom line based on marketing data, personas, and analytics.
Tools are useless without people who can properly run them. Most testing software allows marketers to create and start simple tests in a few hours. But that is the easy part. Designing successful test scenarios, assessing results, and creating meaningful follow-up tests are ultimately where many companies fail.
Poorly designed experiments can take years to complete. Even worse, they might not provide concrete insights to which elements will convert more visitors into customers. Imagine a case where you plan to test different headlines on a page. You start by coming up with 10 possible headline variations. What criteria are you going to use to determine which headlines you should test? Why not test all 10? You will most likely find yourself relying on guesswork to determine which versions to include in the test. The same logic, of course, applies to all the elements you want to test on the page. Without being judicious with test scenarios, we have seen clients attempt to test millions of combinations.
Testing is only one important component of any optimization project. It should take place after you have completed other, equally important stages of optimization work, such as persona development, site analysis, and design and copy creation. Each of these elements provides a building block toward a highly optimized website that converts visitors into clients.
Successful testing starts by creating different hypotheses to explain why visitors react to certain elements on a page. You then use your tests to validate the hypotheses. So, how do you come up with the different hypotheses to test? In our practice, we use the elements of the Conversion Framework (trust, FUDs, incentives, etc.) to create the different hypotheses. This approach guides our testing work with every client and removes the guesswork from the process.
Figure 9-6 shows the original design of a shopping cart for one of our clients that sold nursing uniforms. When our team examined the analytics data for the client, we noticed the high checkout abandonment rates. Abandonment rates for unoptimized checkout are usually anywhere from 45% to 80%. This client reported checkout abandonment rates close to 82%. Nothing in the checkout explained this above-average abandonment rate. The team then conducted a usability test. Nurses were invited to place an order with the site while the optimization team observed and conducted exit interviews to gather information from participants. The nurses revealed that the biggest problem that caused them to abandon the site was their fear of paying too much for an item. Nurses are price-conscious, and they can buy the same item from other competing websites or brick-and-mortar stores. So, price was a big factor in deciding where to buy a uniform. Our client was aware of nurses’ price sensitivity. The client offered a money-back guarantee and a 100% price-match guarantee. The problem was that most of the site visitors landed on category and product pages first, and the company’s price assurances were only displayed on the home page. Therefore, most visitors knew nothing about these assurances.
The hypothesis for this test was as follows: online visitors are sensitive to price, so adding assurances can counter the FUDs the visitors have due to price concerns. Figure 9-7 shows the new design for the shopping cart. The team added an “assurance center” on the cart page’s righthand navigation panel reminding visitors of the 100% price match and the money-back guarantee. The new version of the page resulted in a 30% reduction in shopping cart abandonment.
It’s important to note that a hypothesis that works for one website may not succeed, or even worse, may deliver negative results for another site. After the results of the aforementioned client were published in the Internet Retailer online magazine, another client approached us to test an assurance center on their site. This client was also looking for a way to reduce their cart abandonment rate. Figure 9-8 shows the original design of their shopping cart. Figure 9-9 shows the new design with the assurance center added to the righthand navigation panel. Again, this test had the same hypothesis as the previous one, that most online visitors did not convert on the site due to the price FUD and that adding assurances to the cart page will ease shoppers’ concerns. When we tested the new version with the assurance center against the old version, however, we received completely different results. The new assurance center caused the website conversion rate to drop by 4%. So, although the assurance center helped the first client, it produced a negative impact for the second client.
