usertesting.png

Industry-sponsored Project

Background

As part of the Georgia Tech Project Studio course, I was paired up with UserTesting as part of an industry-sponsored project. UserTesting is a remote usability testing platform that allows companies and researchers to test products with a panel of participants.

Contributions

I was the research lead for this project. I led the research initiatives for our initial surveys and interviews as well as the moderated and unmoderated test protocols on the UserTesting platform.

My Role

Lead UX Researcher

Project Duration

January 2021 - May 2021

The Team

Jason Dong

Stephanie Lee

Problem Statement

Audit and provide an analysis of the current panelist experience with any opportunities for improvement in areas of retention, accessibility, and/or reaching unique audiences.

 

In other words, our research objective was "What are common pain points and opportunities for improvement in the current panelist experience?"

Research Framework
mixedmethodsphoto.jpeg

Our research object/question was our first and perhaps the biggest challenge because it was broad, so we brainstormed and decided to take a user centered, mixed methods approach.

 

There are three parts to this approach: 

 

Background Research - competitive analysis and becoming UT panelists ourselves to be able to empathize with users

 

Reddit Research - surveys and interviews with UT panelists we recruited in order to gain unbiased feedback

 

UserTesting Research - unmoderated survey style tests on UT platform moderated interview style tests on UT platform.

Background Research

Competitive Analysis

Objective: To gain insight into current usability testing platforms and competitors of UserTesting.

 

Rationale: Give us a starting point to go off of by gathering information on pros and cons of each platform and what users like and dislike.

Description: Analyzed onboarding process, payment process, customer support, and overall features.

competitiveanalysis.png
Reddit Research

Qualtrics Survey

Rationale: To gain an unbiased perspective from UserTesting panelists and to narrow down our pain points

Methods: Recruited through subreddit r/usertesting

Demographics: 18+, located in the U.S., current or past panelist on UT

n = 27

Findings: 85% of participants indicated customer service was their worst experience.

Rank the following in order of best (1) to worst (5) experience.

Widget_fe173193-60b7-4638-90ec-a8782cbf6047 (1).png

Reddit Interviews

Rationale: To dig deeper into data collected through our survey. What are specific changes panelists want to see on UserTesting?

Methods: Remote, semi-structured, conducted through Zoom

Demographics: n = 2, recruited through Reddit Survey

Findings: For screening, users had trouble with the availability of tests, repetitive questions, and question formatting

"Prescreening screeners are my biggest concern. I wish there was a way for my answers to be relayed directly to the testers.“- User 1, 6 years experience

UserTesting Research

After conducting the initial Reddit research, we moved on to UserTesting research where we'd be able to recruit current panelists on the platform and have more flexibility in the questions we'd ask them

Unmoderated Testing

Rationale: Understand the most pressing needs of the panelists on UT. What are the biggest pain points in their current experience? Compare to findings from Reddit.

Methods: Unmoderated, survey-style test including a mixture of verbal, rating, multiple choice, and written response questions.

Demographics: n = 10, 5 participants who have been panelists for over a year and 5 panelists who have been panelists for less than a year.

Findings: 

Screening - Eligibility of tests, repetitive questions, screener interface

Communication - Customer support, feedback from testers

System - Home page interface, loading problems

"I notice screeners would show up again even after I have been rejected from them." - User 1, 4 years experience

GTxUT (3).jpg

Moderated Testing

Rationale: Focus on specific pain points which are: screening, communication, and system issues. What are specific design changes panelists want to see on the UserTesting dashboard? Build off results from unmoderated testing.

Methods: Remote, moderated interview-style testing. Acted like a co-design activity where users would discuss specific design changes they'd like to see.

Demographics: n = 5, 3-5 years of experience, currently unsatisfied with the UT system.

Findings: 

Design Changes: Filtering screening questions on home page, greater visibility of "Help" button on home page, more specific feedback from companies after completing a test

"I hope that the customer would provide detail [on their ratings]. The customer has a level of responsibility as to how content they are with my response.” - User 2

"Add a direct ‘contact us’ icon on the main dashboard.” - User 5

Research to Design

We created a journey map to help us visualize where the pain points were in the users' journey through UserTesting

UTJourney map.jpg

After consolidating all of our research findings, we created a table of user needs and the resulting design implications. We rated each user need by how important it was for each the panelist and how important it was for UserTesting, after meeting with our UT advisors/stakeholders.

UTdesignimplications.png

In addition, we created a feasibility/need matrix to help us visualize which changes were most feasible and most needed. You can see in our matrix that there is very little in the ‘optimal’ area of high need and high feasibility and a lot more in the high need/ low feasibility area. This was one of the main challenges we encountered - balancing business and technical constraints of UserTesting with the needs of the panelists.

GTxUT (4).jpg
Design Mockups

We created a mockup of the panel dashboard and made some changes to the upper menu bar, the drop-down menu that you see when you click on your user profile, and to the feedback options that customers see from their end. 

New DB_a.png
New DB_b.png
New Rating.png
Learnings and Challenges
  • One of the biggest challenges throughout this project was dealing with a broad and ambiguous research prompt? Where do we start? How do we know what pain points to focus on?

  • Meeting customer needs + business needs with our users’ needs and our needs as researchers was another difficult area to balance. How do we deal with a problem that users have that isn't in line with the company's goals?

  • The key to user-centered design is making informed decisions based on user feedback, iterating with experts, and including users in the design process.

Future Direction
  • Internal review of our research, findings, and design opportunities

  • More user testing + design iterations

  • Hopefully, integration of our design opportunities!