It’s still surprisingly common to find ecommerce teams that don’t employ usability testing as part of their UX strategy. There’s always an excuse not to of course – too expensive, takes too long, too hard to recruit and the big question, “What will it tell us and what will the ROI be?”.

Having been involved in planning and running usability tests for a good few years, I’ve gained a fascinating insight into the benefits it delivers and the pros and cons of different methods.

The overall benefits are probably obvious – usability testing gives you the opportunity to observe your customers using your website first hand, see what they like about it and more importantly what they don’t or what they struggle with. Parts of your user journey that you may think work perfectly well can suddenly show themselves to be tricky, annoying or both. It’s all about finding areas that could work better and then setting about reducing that friction.

We all like to think we know what works online. The problem is that without rigorous usability testing, there’s always going to be an element of assumption and those assumptions could be costing your business money. It may be that your web designer, web agency or Head of Ecommerce have very strong opinions about what they think (or “know”?) will work best, perhaps citing past experiences of other ecommerce sites. Those opinions can be compelling, but what about your customers, buying your products, from your website? The only way to really know is to test it.

I’m not saying that usability testing is the only research you should do of course. Monitoring services such as Hotjar are great for tools such as heatmapping, session recording and pop-up questionnaires. It’s also essential to keep on top of your analytics to give you quantitative data. Combining this insight with the qualitative data from usability testing builds up a very strong picture of what’s going on with your website. I often say to clients that analytics tells you what is happening, usability testing tells you why.

Let’s take a quick look in more detail at the common reasons why ecommerce teams don’t always employ usability testing.


“It’s too expensive”

Not all usability testing has to involve several days in a lab. We’ll look into the various methods you can employ below, but the main question to ask yourself is “What might we find to improve and what if it gave us even a tiny uplift in conversion?” With most ecommerce sites of a certain size, even a fraction of a percent uplift on mobile alone for example would pay many times over for the usability test and resulting UX/development required to implement the findings. Think about the big picture, not just the initial investment required and you can always start small, prove the process and then grow it.


“It takes too long”

Again, usability testing doesn’t have to be full on lab testing, although there are some big benefits to it that we’ll cover shortly. We’ve seen some really interesting results from even just a couple of hours spent testing stuff on non-UX designer colleagues, friends or family. What have you got to lose by trying it for yourself? Remote usability testing can take as little as 24 hours to turn around.

“It’s too hard to recruit”

We’ve done varying levels of recruitment, from testing on colleagues, to testing on client employees, to full recruitment from the general public. It does take time to recruit customers, but you can shortcut the process by using your own database and offering vouchers as an incentive to users who are willing to participate.

Usability testing methods

We employ three main ways of usability testing. Each has their place, depending on the project, timescales, budgets etc.

1. Ad hoc

Any usability testing is better than none, as long as you get the basics right. Where budget or timescales are restricted, you can do meaningful tests with those closest at hand – colleagues are great, but only those not involved in day-to-day web UX, development or operations – and try to pick people who are in your target demographic. 63-year old Brian from Accounts might be willing, but if you’re selling teenage fashion, not much point testing him. Make sure you explain that you’re interested in their impartial opinions and ask them to think as customers, not employees; and reassure them that anything they tell you is in confidence and not about criticising anyone’s design work.

You can also do guerilla testing on the street or in a local store, but this can be difficult to get people to spend enough time with you to get a good quality test.

Finally, testing on friends and family can be OK, but they can find it hard to be impartial and often say what they think you want to hear, so be careful with that one.

Pros: Quick, cheap, easy

Cons: Quality and impartiality of participants

2. Remote

Remote usability testing can be a good intermediate solution. Online services such as are easy to use and relatively inexpensive. They allow you to recruit any number of users to your specification based on gender, age, social grade and a qualifying question. You write a test plan for the user to follow and receive back a screen recording of them going through the test and a recording of their voice so you can hear their thoughts and opinions. It often takes no longer than 24 hours, depending how niche your demographic is and you can run tests on desktop, tablet and mobile.  

It’s important not to be too prescriptive with your test; you want the test to be as realistic as possible. We often ask users to navigate around the site freely for the first 5 minutes to get initial impressions, before asking them to do something more specific such as, “Find three dresses that you would consider buying for a night out and go through the process of buying them”. It’s also a good idea to ask more general questions in the test such as what they think of your brand and where else they shop for similar items.

Pros: Quick and easy to set up, simple recruitment, relatively inexpensive, good quality output

Cons: The conversation is one-way, you can’t ask questions during the test

3. Lab

Although I don’t like the word “lab” as it sounds too formal, face-to-face testing where you’re sat alongside users is the best method available. Good UX labs are very far from what that word implies and often feel more like someone’s front room, designed to make users feel relaxed. This is very important if you’re going to get as realistic a test as possible. They also often have two-way mirrors where observers can come along and watch, which is usually of great benefit to our clients.

We have recruited from clients’ customer databases in the past (can be good as you know they’ve shopped with you before) or employed a recruiter to find users to a very specific criteria if relevant, which is great as they handle the whole process including reminding users to turn up on the day!

In terms of the quality of the tests, it’s unrivalled as you can have a two-way conversation with the users and this allows the test to develop organically. You may find an unexpected nugget that you want to dive into in more detail, that you wouldn’t have the chance to with remote testing.

Pros: High quality recruitment, two-way conversation with users, chance to dig into issues in great detail, observers can attend

Cons: More expensive and time consuming to organise (but remember the possible ROI!)