What to Test When You’re New to Testing

You should split test the headline first! Bollocks. The images! What a load of crap.

That’s the kind of advice most articles out there would give you. The truth is that answering “what to test first” is like answering “how much does a car cost?” – it depends.

Eventually you should test everything that you have traffic for, but since you can’t start everywhere at once, you need to prioritize.

Where should you run your first tests?

You should start testing where it hurts the most, and where you can learn the fastest.

1. Where the biggest, the costliest leak is

Maybe your home page is your most popular landing page. Gets enough traffic, like 75k uniques per month. The bounce rate is 65% , sub-optimal. At the same time, your checkout page gets 500 unique visits per month. Your monthly transaction count is ~100. Where do you start?

I hope this is obvious.

Large chunk of home page visitors are likely not gonna buy anything during this visit, but people who’ve made it to the checkout page have added a product or two to their cart and shown all kinds of engagement. So if only 20% of the people that make it to the very last step of the checkout funnel end up buying, that’s where you’re bleeding cash the most. That’s where you should start optimizing and testing.

Consider all the steps in the user flow, measure drop-offs

Let’s say you run an ecommerce site. Your category pages gets a ton of unique pageviews. Product pages get roughly the same amount of pageviews. What does that mean? It’s likely that your category page sucks ass! Investigate the crap out of that.

Data doesn’t tell you anything, but if you’re smart, you might be able to pull an insight out of it, and use it to go deeper down the rabbit hole. You need to measure the drop-off rates between the logical steps in the buying process to learn where the biggest problems are.

If you had 300k product page views, 5k cart adds and 1k checkouts – where would your problem be?

But if you had 300k product page views, 100k cart adds and 1k checkouts – it’d be in a different place!

Check my article on 10 Google Analytics reports that will help you identify where your leaks are.

2. Where you have the most traffic

You can’t run tests everywhere. Testing is a tool for learning, so if it takes you more than 30 days to run a test until validity, it’s too slow.

Hence you need to prioritize pages where you get plenty of traffic. Run your Behavior -> All Pages report to see your top 10 pages with the most traffic, and use a sample size calculator or test duration calculator to figure out if that’s enough traffic.

In addition to traffic you need to pay attention to total transactions per month. If it’s less than 500, it’s tricky (probably not enough). You then either need to swing for big wins (if your site is decent, that’s *very* hard), or skip testing for now (pick another strategy).

If you don’t have enough traffic for testing (within a reasonable amount of time), then you can still optimize. Optimization does not equal testing.

What to test on those pages

Don’t follow stupid articles that say test X, Y or Z. You don’t have general problems. You have specific problems. 

A blog post might say “test your product photos”, but in reality your product photos might be freaking amazing – but maybe nobody can read the product copy since the font size you use is 11px and light gray on a white background.

Ignore generic advice. Your job is to figure out what specific problems the pages in question have, and test a hypothesis that addresses that specific problem.

1. Low-hanging fruits

Most websites have obvious problems, so starting with those is a good idea.

How to spot them? Use heuristic analysis – assess the pages in question using specific set of criteria. Random criticism (“this sucks!”) will only take you so far. You’ll be much more effective when you have a structured approach. Do it in a team of 2-4 people for maximum effectiveness (more people see more things).

• Clarity: Is it crystal clear what the page is about,  who’s it for and what the user should do here? Reasons clearly stated? Is there any jargon used? Any language that you wouldn’t use in a conversation with your friend? How can we improve clarity?
• Motivation and value: Are the benefits of taking action clear? What else could be done to increase user motivation? Can we use any persuasion triggers?
• Friction and anxiety: How can we make it simpler? Are we doing a good enough job explaining why the user needs to fill out all these form fields? Is it clear what happens next? Can we improve the perception of security?
• Distraction: Is there anything on the page that is not helping the user take most wanted action? Can we get rid of it? Is there any animation, automatic movement? Can we kill it? Are the most important things dominant and above the fold (for most screen resolutions)?

I find it helpful to add comments on screenshots, something like this:

2. What are users having difficulties with?

Do you know what actually causes problems for your users? User testing and usability testing can shed a lot of light here.

User testing gives you direct input on how real people perceive your site (of course they’re not actually risking with their money, but it’s still useful). You may have designed what you believe is the best user experience in the world, but watching real people interact with your site is often a humbling experience. Because you are not your user.

With user testing (either using usertesting.com or equivalent, or doing it in-person) you want to have people conduct 3 types of tasks on your site.

Image credit

In most cases you want to include 3 types of tasks in your test protocol.

• A specific task
• A broad task
• Funnel completion

So let’s say you run an ecommerce site that sells clothes. Your tasks might as follow:

• Find dark jeans in size 34 under $50 (specific task)
• Find a shirt that you like (broad task)
• Buy the shirt (funnel completion)

You have users that know what they want, and users who’re browsing around. This test protocol accounts for both. And funnel completion is the most important thing – you want to make <ipurchasing as easy and obvious as possible.

After watching 5 to 10 users go through your site, performing various tasks, you’ll have a good idea about what they struggle with.

3. What the users are telling you about their intent and perceived friction

Surveying people as they browse specific pages on your site (e.g. by using Qualaroo) is another great source of insight. Essentially you ask a single question at a designated time. It is a particularly good tool for finding out why your customers are not taking desired action.

The question to ask depends on your overall goal. Do you want to understand if your site or products/services on your site are meeting their needs? Or do you want to understand what potential sources of friction are keeping customers from buying?

Experiment with your question to see what gets the most responses and responses with the most insights. If “What’s keeping you from X?” wasn’t as successful as you had anticipated, ask “Do you have any questions about Y?”

Conclusion

You don’t have general problems, you have specific problems. Take even 5 minutes to check your digital analytics data to see where it makes sense to start with A/B testing. There is no “one size fits all” recipe.

Once you know WHERE the issues are, start learning WHAT they are. Get some insight into the doubts, hesitations and fears your target audience has, you can come up with a test hypothesis that would address those issues.

This post originally appeared on the ConversionXL Blog, republished with permission from the author.

About the Author

Peep Laja runs conversion optimization agency Markitekt, and world’s most popular CRO blog ConversionXL. He is an entrepreneur and conversion optimization expert who has been doing digital marketing for more than a decade in Europe, Middle East, Central America and the US. He has extensive experience across verticals: in the past he’s run a software company in Europe, an SEO agency in Panama, real estate portal in Dubai and worked for an international non-profit.