Remote usability testing allows UXers to conduct user research with ‘hard to get users’ in their own environment. Remote testing is preferable when in person testing isn’t possible due to budget or time constraints or testers availability at multiple geographical locations.
This blog is aimed at pointing out few critically important things to be considered when you’re planning for remote usability testing.
- Objective of the test:
First and the foremost thing is to be sure about the objective of the test. Try to figure out answers to these two basic questions before moving ahead:
- What do you want to achieve from this test?
- How will the outcome benefit you in designing better user experience?
To get the answers to these two questions, stakeholders are the best person to reach out.
Based on the objective of the test, you can choose the number of tasks you’ll be assigning to the users (Ideally between 3-5) and time to be given for completing the tasks (10-30 mins).
- Moderated v/s Unmoderated (remote usability testing):
Remote usability testing can be also moderated or unmoderated which typically depends on tasks, devices/ resources available and expected results.
Moderated user testing requires additional screen sharing/recording software’s and an audio connection to be able to analyze user interaction with the design and be able to ask questions. On the other hand, in unmoderated remote usability testing, users available at different geographical locations are assigned to set of tasks. They are free to perform these tasks independently, without an interaction with the test moderator. Their results are gathered at one place and can be analyzed by the reviewer once the task is complete.
Moderated testing is best suited for complex tasks that do not have an organized sequence of steps to follow and where questioning the user’s move can help in better understanding of the test results.
Un-moderated testing is effective when you have very specific and direct tasks to put forward to users and get UX insights like time spent by users, no. of interactions made, pain points with the UI and UX in completing the tasks and others.
Be clear in choosing the one which suits your objective.
- Tools required:
For Remote moderated usability testing, real-time communication tools like Skype, hangout, WebEx, GoToMeeting, JoinMe and others are used. They help connecting moderator with the tester in real time and have both audio and video communication.
Further, video editing tools can be used to edit the testing sessions into highlights that point out common usability problems. Few well-known video editing tools are Movie Maker, Adobe Premiere and so on.
For unmoderated user testing, a lot of tools are available for live projects. Now it’s important to analyze whether you can afford to iterate on your live app that would eventually cost you a lot of time, effort and money or you want to be more accurate on the first go by performing usability testing on the early designed prototypes (CanvasFlip is pretty amazing for usability testing of prototypes).
Testing on both live app and prototype has its own advantage and disadvantage. Hence, based on your preferences and process you follow, you have to choose the tool that works best for you.
- Number of users to test with, and where to find them:
You have to be sure of the number of users you’re going to test your project with. If your product is for multiple target groups then for each specific group you should have 3-5 users as qualitative insights like interaction heat-map and drop off on each screen, then, can be very effective in identifying the problem areas in your screens.
It’s ideal to test your product with real users, assigning them tasks and analyzing their behavior in doing so. If you still don’t have real user’s to test, start building a beta tester community by giving them privileged with the product or some kind of motivation to do so.
- Measuring the result:
Once you’ve all above discussed points in place, it’s time to identify the key metrics you’ll consider in measuring the result of the remote usability testing.
For moderated, results basically are in the form of keynotes derived from the interviews/answers given by the testers and is used to understand user expectation and behavior. On the other hand, unmoderated testing result depends on few key metrics like-
- User videos (Identify friction points and areas of struggle for your user)
- Drop-off on each screen data (Spot the screens with higher drop-offs and fix the design.)
- Interaction heat-map (Find out why user’s dropped there)
- Time spent by user to complete the task (Lesser the time spent to complete the task, better it is)
- Success rate (No. of users able to complete the assigned task vs total users)
(These Insights can easily be generated using CanvasFlip)
There are a lot of new tools, techniques, and metrics to consider when you’re planning to conduct remote usability testing. Thus, before jumping into it, use checklist and make sure you are well prepared.