A NSW Government website

NSW Government
Digital NSW

Usability testing

Usability testing helps you understand how real users interact with your product or service. It can show where people get stuck, what they expect to happen, and what could be improved.

Use it throughout the design process to test ideas, check assumptions and improve the experience before and after launch.

What is usability testing?

Usability testing involves observing real users as they try to complete tasks on a product or service.

This could include a website, app, form, online account, or other digital service.

The goal is to understand whether people can complete key tasks easily, and where the design may cause confusion, errors or delays.

Why usability test?

The goals of usability testing vary by study, but they usually include:

  • identifying problems in the design of the product or service
  • uncovering opportunities to improve
  • learning about the target user's behaviour and preferences.

It's important to do usability testing as part of your iterative design process so that any design decisions about your product or service are based on observations of real users, and of their interactions with the design.

Types of usability testing

There are different types of usability testing, but the core elements in most usability tests are the facilitator, the tasks, and the participant.

  • The facilitator guides the participant through the test process, working to ensure that the test results in high-quality, valid data without accidentally influencing the participant's behaviour. They give instructions, answers the participant's questions, and ask follow-up questions.
  • The tasks in a usability test are realistic activities that the participant might perform in real life. They can be very specific or very open-ended, depending on the research questions and the type of usability testing.
  • The participant should be a realistic user of the product or service being studied. They might already be using the product or service in real life or might just have a similar background or needs to the target user group.

Qualitative vs. Quantitative

There are 2 main types of usability testing:

Qualitative usability testing – helps you understand why users struggle and what needs improving.

Quantitative usability testing – helps you measure how well users complete tasks using data such as success rates and completion times.

Moderated vs unmoderated

Usability testing can be moderated or unmoderated.

Moderated testing – a facilitator guides the session while the participant completes tasks and talks through what they are doing. The team can observe, take notes and ask follow-up questions. Moderated testing can happen in person or remotely using tools like Microsoft Teams.

Unmoderated testing – participants complete tasks on their own using a testing tool. This can be easier to organise, but you cannot ask follow-up questions or help clarify what happened during the session.

Number of users to test with

The number of participants needed for a usability test varies depending on the type of study. The objective is to test up to a point where you can identify trends and patterns and further tests will not give you much new insight moving forward.

For a typical qualitative usability study of a single user group, we recommend using 5 participants to uncover the majority of the most common problems in the product.

It's important to ensure you test with at least one user with physical or cognitive limitations or who needs assistance with digital applications. Testing with real users can uncover accessibility barriers better than a checklist.

Planning your tests

Before planning any sessions, work with your team to consider:

  • Research goals and tasks – you need to design test tasks to make sure they answer your research questions.
  • A prototype - the purpose of usability testing is to put something in front of the user and watch them work. This could be a prototype, working website, physical products.
  • Test participants – these need to be actual or likely users of your service.
  • Roles and responsibilities - each member participating in user research will have specific roles and responsibilities.
  • Time – you need time to plan the usability test. Budget in time for test preparation as well as running tests, analysing the data and presenting the findings.
  • Costs - consider how or where you will recruit your participant's and if you will be compensating them for their time or travel.
  • Test location – you don't need a formal lab. You can use meeting rooms, run pop up sessions or test remotely.

From testing to actionable insights

Usability testing is not just a milestone to be checked off on the project schedule. The team should have a goal for why they are testing and then implement the results. 

After conducting your research, analyse the data so you can produce useful insights that will help you design your service.

Resources

This content is based on work by the Ontario Digital Service (licensed under the Creative Commons Attribution 4.0 Licence).