On 27-29th April, the highly-respected Future of Web Design Conference  @FOWD is taking place in London. And we’re thrilled that our MD Rob Smith and User Experience Director Chris Jones are involved.  The duo will be delivering an interactive, full-day workshop titled Humanising the E-Commerce Experience on 27th April and a speech the next day titled Is E-Commerce an Art or Science?

Ahead of the conference, Chris Jones shares his thoughts on one of the questions often running through his head:

“Is user testing just design by committee?”

If ever you find yourself frustrated by the feedback you receive from a round of user testing, just think how far design has come from the days when the only feedback you got was from clients.

Design was easy back in the day before the web, right? When I was at university and even in the early days of my career, briefs would contain information about the target audience: income, location, gender, age and maybe likes and dislikes or some lifestyle preferences.

These profiles may have come from market research, or more often, they were anecdotal. The brief represented who the client thought (or hoped) they were selling to. As a designer, my job was to meet the client, understand the brief and then create a solution that I believed would most effectively encourage the target audience to buy what we were selling.  

The operative word in that sentence being ‘believed’; very often we didn't know. We were designing for print or advertising, so finding out what effect your work was having was much harder than it is online. We may run a (notoriously unreliable) focus group to test different solutions, or attribute an increase in sales to a new ad campaign but not only could this take months, it was never completely clear what percentage of the increase, if any, was down to our work. It could have been an improved product, cheaper price or higher media spend; the list goes on.

Design, Not Art

It's an old maxim that design is where art meets science. If you study any kind of visual design, you’re taught to understand the target audience and design for them, but there's also a lot of intellectual ‘arty’ stuff; discussions about the interpretation of this and the juxtaposition of that.

OK, so I'm not saying it isn't important to explore while studying at university, quite the contrary, it’s a time to enjoy the freedom. But it can be a harsh lesson to learn once you leave, that business people don't care about these things, they simply want results. Business owners and marketing or ecommerce managers want to know that the design they're paying for is going to bring great ROI and the more you can assure them of that, the better. 

The design decisions being made in the first half of my career were a mixture of my experience as a professional designer and what my client thought would deliver the best return. What's wrong with that? I'd spent years studying and working on design projects, so I knew how to create something that would appeal to a specific target audience. My clients had spent years selling to their customers so they knew their businesses. I knew the right questions to ask and had the skills and experience to deliver great design that fulfilled the brief. Simple!

Unfortunately, it wasn't so simple. What if I was recommending a design solution and my client disagreed? Perhaps they didn’t feel it was right for their business or customers? Maybe they just didn't like it? How could I prove that it would work and deliver cold, hard cash for the business? I could justify my solution with my experience of what had worked for other clients, maybe even clients in the same industry. But sometimes we'd reach an impasse. After all, design is highly subjective and how can you argue against someone's heartfelt opinion... especially if they’re paying for it?

The Dreaded Design By Committee 

My enemy in those days was the dreaded ‘design by committee’. A project would go like this: I'd take the brief and present a design solution that I believed was perfect. The client would then ask for a few, perfectly reasonable, amends. I'd then send over a second version. They'd share the design around a few other people, for their input. And then all hell would break loose.

Stakeholders who had never seen the brief would start to give their opinions and sometimes demands. Ever heard of HIPPOs – Highest Paid Person's Opinion? My client would be fielding requests from all areas and lose sight of the brief (and hence the target audience). We'd end up with a design that was trying to please everybody, but that in reality appealed to no-one, including the target customer. Ask everyone what their favourite colour is and then take an average of the results and you'll end up with beige. And really, who likes beige?

It was sometimes difficult to explain why a design would fail if everyone's opinion was taken into consideration. I lost count of how many times I started a sentence with, "In my experience ..." What I needed was an impartial point of view to take it away from being just my professional opinion; an input that the client would value more than mine and dare I say, even more than their own.

A New Design Process For The Web - User Testing 

Fast forward to today and design is very different. The web has given us the ability to see what users are doing, quickly find out what does and doesn't work, then test, iterate, and improve. This applies equally to all other aspects of what has become known as User Experience – users' journeys through the sites we create, the language we use to guide or inform them, the interactions we employ to give feedback and make a website tactile can all be tested.

These days I rarely present a design solution that isn't based on user testing in some form. So, where is the line between what I'd recommend as a professional web designer and what we might observe users doing? Should website owners still listen to the expertise of web designers or should they react to every insight provided from user testing? If they do, aren't they guilty of bowing to design by committee and creating beige work that gets lost in a sea of mediocrity?

My experience of running user tests is a happy one. You might think that I'd far rather not have to take real users opinions on board in my work - occupy the moral high ground and take an " I know best because I'm a professional designer" approach, but it couldn’t be further from the truth. Without wanting to sound too voyeuristic, it's always a thrill watching users try something that I've designed, even if the results aren’t positive to start with. Whether the design works brilliantly for users or the test highlights problems, we learn something valuable every time. It gives me pride in my work to know (rather than believe) that the work we're doing is providing a good ROI for our clients.

So user testing is a good thing, but how do you stop it from turning into design by committee? If you test a website on ten users and they all say slightly different things, do you try and accommodate every point? No is the short answer. User testing is as much about interpreting the results of the test as it is the test itself.

When we review results, we look for patterns in behaviour. If a few of those ten users make the same comment or face the same issue, that's a pretty clear indicator that we have something we need to work on. Even if overall they’re in the minority, it’s something to look into. After all, even if only 20% or 30% of testers raise the same issue, that's a lot of people if your site gets millions of visitors. 

Now for the one-off issues and comments: should we ignore those? I won’t lie, it's tempting to think "there's always one." The person who sounds like they're just having a bad day and don’t want to be taking the test. They probably hate websites anyway. But listen to what they have to say, their opinions are just as valid as anyone else's.

Balancing User Reactions and Design Experience 

The skill comes in interpreting users' reactions and I believe this is where the balance is between what users say and what we've learnt as web design or UX professionals. If something that a user says sounds ridiculous, try to put it into context. Did they read the instructions properly? Would they react that way if it wasn't a test? Were they distracted or bored? Perhaps most importantly, if you react to every single comment from every single user, will the overall journey through the site still make sense or will it be become inconsistent, fragmented and ultimately not much fun for anyone? 

When your head is immersed in the detail of user testing, make sure you keep one eye on the bigger picture and have the confidence to temper it with your own knowledge and expertise. User testing is an essential part of web design/UX and will improve the work you do, but think of it as a useful addition to what you already know, not a replacement. Have the courage to make your own informed decisions and never ever design by committee.

Tickets for Chris’ workshop and for the wider FOWD conference are still available but selling fast. So, if you like what you’ve just read, click here for more details on the conference and to purchase your ticket.

To see more about Chris' workshop and session, visit Future of Web Design

Comment