Moderated usability studies, which are live sessions with a facilitator, are more helpful than unmoderated sessions (where the user does a session on their own) because you can ask follow-up questions during the session. They involve one facilitator and one user at a time, as well as any observers working on the project. These are mostly conducted remotely now using video and screen sharing software and a prototype or live site/app.

Here are the detailed steps and recommendations involved in planning an effective moderated usability study, conducting usability sessions, analyzing findings, and presenting findings and recommendations.

Planning a Moderated Usability Study

When it comes to planning a usability study, there are several steps involved and you’ll also need to collaborate with your business/product stakeholders. Planning can take 1 - 2 weeks depending on stakeholders and their feedback.

During your first meeting with stakeholders:

  1. Determine if they have any experience with previous usability studies, to determine how much education you need to provide them with on best practices
  2. Determine goals of the usability study
  3. Determine tasks:
  • Any areas that are new
  • Any interactions that might cause issues
  • The main tasks a user would need to complete

4. Determine tools that will be used:

Prototyping software or live site?

Scheduling: Calendly free or pro plan?

Calendly free plan:

  • It helps to build out your availability based on your timezone
  • The organizer can send out one link to multiple people, and once someone books a timeslot it removes that time so it can’t be double booked
  • Participants can easily reschedule

Calendly pro plan is currently $12-15/mo, and allows you to do everything that the free plan offers, plus:

  • It creates a calendar invite with instructions and links when someone books a timeslot, which removes a process step for the organizer
  • Customizable email notifications and reminders, instructions and any links can be added

Usability sessions: Zoom or Lookback?

Zoom:

  • No extra cost whereas Lookback is currently $99/mo
  • Most people know how to use Zoom
  • Can call in if someone doesn’t have audio on their computer
  • Works better on older computers

Lookback is currently $99/mo, and has some advantages over Zoom:

  • Opens prototype link for users automatically
  • Timestamped notes, and can see observers notes in real-time
  • Observers join muted and without video automatically
  • Sessions are automatically recorded
  • Export clips & recordings to share, and houses them all in one place
  • Transcribes audio
  • Opens a link for GoTeam video conferencing for debrief after

5. Determine observers

  • Include stakeholders, product managers, developers, and anyone interested
  • The more observers the better, when people hear it first hand they are more likely to remember a user struggling

6. Determine user recruitment based off of personas and if they will be paid

  • Determine how much participants will be paid, if at all
  • Ask for an email list of 30+ possible participants
  • Recommend to schedule 7-10 sessions making sure to allow for 1 cancelation

7. Determine proposed schedule, to be build out in Calendly

  • Recommend 4 sessions a day max throughout one week
  • 1 hour sessions, with 30-min break in between sessions to allow for 15-min discussion after each and a 15-min personal break

8. Determine when the team will meet up after to work out next steps

  • If using a prototype, you will be building out prototype for tasks
  • Building out usability script
  • Creating an email template with Calendly link

After your meeting, use this to do list as a guide:

  1. Build out Calendly:
  • Recommend 4 sessions a day max within one week
  • 1 hour sessions, with 30-min break in between to allow for 15-min discussion after each and 15-min personal break
  • Update instruction emails, add phone number if there’s issues, and reminder emails

2. Create and review email template with business, and then send out email invitations to participants:

  • Email should be short. Use the template below to write your email, customizing everything in between the “< >” spaces:

    Title: Give us feedback on <product name>

    Body:

    Hi,

    I am < your name > and I am partnering with < company name >, as a consultant from Levvel. We are currently conducting research on an improved experience < >. We would like to spend < number > minutes with you showing you a concept and testing our designs.

    We are conducting this research between < date range >. Are you interested in partnering with us in this research effort or is there a better person that we can talk with? If so, please book an appointment < (make it a link) & insert calendly link >.

    Thanks,
    < your name & email signature >
  • Review in 24 hours to see if you need to send out more invites
  • Once a calendar invite has been sent to you, create a duplicate for each session for observers, include Zoom or Lookback link

3. Make sure prototype is built for tasks

4. Build out usability script and get feedback from business

5. Build out a note taking document. If using Zoom it can be in the usability script and a copy can be made each session.

6. Build out a file for all notes. You can use Levvel’s FigJam UX Research Notes + Analysis template as a starting point.

7. Schedule and run a trial of the usability test, also know as a Pilot test: Schedule with:

  • 1 facilitator
  • 1 observer
  • 1 “participant”, can be coworker not involved in the project
  • Take notes of anything that needs to be updated, make updates afterwards

8. Review expectations with observers ahead of time:

  • They will be sent individual calendar invites to observe + 15-min discussion after each
  • If using Zoom (stay on mute + no video), stay of after a session to discuss for 15-mins
  • If using Lookback, take notes in Lookback during session, at the end it will prompt you to join a different video session to discuss

Conducting Usability Study sessions

Now that all that planning is complete, you are ready to start the usability sessions.

Day of:

  • Make sure you have water and snacks close by

5-10 minutes before:

  • Have Zoom or Lookback up, taking up half of screen with usability script on other half
  • Disable all notifications
  • Have Slack and email accessible incase there are issues, or if a participant is late:
  • Sometimes you have to send an additional reminder email to see if they are able to join now (wait 3-5 minutes after start time)

Once a participant joins, start with the script.

  • If using Zoom, hit RECORD after consent to record
  • Take quick notes, and note time of any important things to watch recording ex. (12:15)
  • Be mindful of the time, allow for 5-10 minutes for questions from participant (skip questions if needed, or ask if the participant can go over an additional 5-10 minutes)

Once the participant leaves, hold discussion session with observers

Analyzing Findings

If you have time in between other sessions, don’t have to wait until you’ve been through all the scheduled usability sessions, rewatch recordings to grab verbatim quotes and more accurate notes. This process should take a few days to complete.

Use this FigJam template, start inputting notes for each session, color code green = positive finding, red = improvement opportunity


Once you have completed all your sessions, you’ve collected tons of insightful qualitative data during the process. Now it’s time to go through and analyze it:

  • Copy post-it notes from the same task from each participant into the Affinity Grouping area.
  • Label any similar themes from participants.
  • Do this for each task, and you may find themes from other tasks also fit within others.

Once you have all your groupings, figure out how to organize them all. For example you can organize them by page or functionality. Then, put all positive areas together and all improvement areas together.

Then use the analysis area to put task/experiment, insights, and opportunities for each theme/page.


Presenting Findings & Recommendations

In general, it’s best to present findings during an hour-long meeting with a presentation deck. If you just send out findings through email, it’s likely not to be read in detail or at all.

  1. Schedule a meeting ahead of time with stakeholders and observers to review findings and recommendations
  2. Put together a slide deck using client’s or Levvel’s powerpoint branded slides to present findings. This will take a few days to complete depending on how detailed the research was.
  • Use and gather screenshots, video clips, quotes
  • Keep presentation to 30-45 minutes to allow for discussion and questions
  • Create a title slide
  • Create an Agenda slide
  • Why we tested slide, Business and usability case
  • Who we tested with slide, # of participants and info about them
  • How we tested slide. What was asked, ie. Asked for their top challenge using software for winemaking. Given 6 tasks using a desktop clickable prototype, to evaluate using a certain area. Also asked for feedback on the list page and detail page, how the information was organized and if there was anything not useful for them.
  • Learnings & Opportunities slides. Start off with general learnings and positive learnings, then go into improvement opportunities. Each slide should only contain 1 learning or general learning + visual and/or quote, add more info in the speaker notes that you’ll speak to.
  • Next Steps & Recommendations slides. Summarize recommendations in a list. Provide a timeline of next steps.
  • Appendix slides. If there’s a long list of detailed page improvements, prototype link, and additional screenshots.

3. (Optional) Review presentation with 1 stakeholder or product manager before larger group review

4. Present slideshow and keep an eye on the clock to allow for questions at the end

5. After presentation meeting, send presentation deck and recording to attendees

Well done! You have shared usability findings and actionable recommendations. Now you can start working on next steps, which may include making updates to a prototype.