What Is User Testing? Methods, Benefits and When to Use It

Analytics data is great at telling you what users are doing and showing you trends in how they engage with and navigate on your site but it fails to tell you the why. That’s where user testing comes in. A form of qualitative research, user testing involves watching real people complete tasks in real time or answering questions, helping you to uncover pain points and struggles by directly observing a user’s experience of the site. 

GA4 and Adobe Analytics are examples of quantitative data sources. They draw from much bigger sample sizes and show you what is happening at scale. This data sits as the foundation of your insight. Qualitative data that you get from user testing allows you dig deeper and understand the ‘why’ behind the trends you are seeing, using a much smaller sample size. Layering qualitative data from user testing on top of trends identified in quantitative data gives you much richer levels of data to underpin your hypotheses and your understanding of your users.

Depending on the capabilities of your testing platform, you can opt to recruit your own customers or use a research pool to target people who match your audience. In a user testing project we did with Greater Anglia, for example, we used an external pool of participants and targeted people who travel by train. You can find out more about how we helped Greater Anglia here. 

There is a range of user testing methods you can use depending on the aims of the study and the capacity of the researcher:

Unmoderated testing: Mostly task-based where the researcher sets a range of tasks that the user completes independently. The audio and onscreen activity of the user can also be recorded depending on the capabilities of the user research tool. 

Examples of unmoderated testing:

• Task-based testing: giving users a realistic scenario and watching how they complete it (this is great for spotting friction in key journeys).
• Tree testing: testing your navigation without any visual design but, to see if users can find the listed pages within your current structure.
• First click testing: seeing where users click first from a task. This great for testing how clear your copy and labels are.
• Card sorting: asking users to group content into categories, which shows how they think your site should be organised.

Then there are moderated interviews where a researcher guides the session in real time. These enable researchers to dig deeper into the “why” through follow up questions.

Want to find out more? We have a full blog that outlines the key differences that you can read here.

User testing works best building off of analytics data not instead of it. It’s not a statistically significant way to collect data but rather should be seen as a source of further understanding of the trends you have already identified. It works great  and a source of hypotheses for you to test. The insight gained differs from data that you can get for other qualitative sources like surveys as what users say and what they do can differ and user testing, especially the task based allows you to observe behaviour and not just collect user opinion. 

So when should you use user testing? 

Here are some key questions you might find yourself asking, the situations where user testing can help and the method to consider: 

If you find yourself asking “do I need to prove impact?”, that’s where experimentation comes in. User testing can tell you why users behave in the way that they do, but it can’t tell you whether a change will move the needle. That’s the job of an AB test. The two work best together: user testing uncovers the problems and shapes the hypotheses, and experimentation proves which solutions actually make a difference.

We worked with Greater Anglia using user testing to understand how customers plan journeys, buy tickets and check disruption on their website. This insights from user testing shaped nine test hypotheses, showing how user testing can underpin an experimentation roadmap. Find out more here.