armoire

Armoire

We were tasked with creating a usability study for Armoire, a clothing rental company, surrounding search & discovery of clothing items and features. We created a usability study plan, metrics, and presented findings to stakeholders.

My Role

My primary role was UX Researcher. I performed a heuristic evaluation, co-developed the usability testing screener and script, created the note-taking form, and was responsible for compiling qualitative findings, including quotes, video clips, and recommendations.

The Problem

Armoire is a women’s clothing rental startup based in Seattle that gives its members access to a rotating closet of curated contemporary styles.  Our team investigated the usability of Armoire’s item discovery navigation, focusing on the site’s search functionality and browsing options. 

We conducted usability and survey testing with 8 of Armoire’s existing users. The core research questions informing our study were:

• How efficiently do participants find a desired clothing item from Armoire’s home page? 
• Are there identifiable usability issues pertaining to the item discoverability process?
• Are participants satisfied with their search results?
  

For our usability study, we connected with eight existing Armoire users via Zoom to conduct usability testing of the clothing item search and discovery process on Armoire’s desktop web interface.

Methodology

To better scope the usability study, we needed to clearly understand what types of users we wanted to recruit and specifically what key tasks they would need to perform.

Target Users

Armoire’s primary target user group is established professional women between 30 and 50 years old, who are looking for great clothes to wear every day.  

Armoire’s Primary Persona, Vanessa 

This product has a core group of subscribers that stick around not just because they love the product, but also because they have an emotional connection to the overarching company culture and mission. Because of this, a representative for Armoire has suggested that we focus on new/slightly experienced subscribers to gain insight that would give them insight into the user experience of someone new to the product. More specifically, we aimed to recruit current subscribers with accounts less than a year old. This is also reflective of the company’s current interest in new user acquisition.

Primary Tasks

In order to evaluate the usability of the item discovery functionality, we tested the following tasks on Armoire’s desktop web interface: 

Task 1: Using the Search Function

• Find and type in the search bar for an item by color or brand name (e.g. black cardigan)
• Use filters to hone in on items
• Add an item that meets the search criteria to add to the case
  

Task 2: Browse Categories & Using Your Favorites. 

• Using the “curated for me” section, add an item to your favorites.
• Navigate to the “curated for me” area
• Add an item from the “curated for me” carousel to user’s favorites 
• Add an item from your favorites to your case
• Access favorites carousel or page 

Task 3: Rate Upcoming Styles

• Navigate to the “Upcoming Styles” section.
• Rate upcoming styles.

‍

Study Strategy

Now with this background information, we developed a study strategy. We conducted individual heuristic evaluations of Armoire’s website to determine project scope. Potential project scope was directly discussed and determined with stakeholders from Armoire. Then, participant recruitment was focused on current Armoire subscribers with accounts less than a year old. We successfully completed this study with 8 participants. Interviews included open-ended questions as well as follow up questions using Google Forms. This study is categorized as a remote moderated usability study.

Each session consisted of three primary participant tasks:

• Searching for items (both a zoom appropriate top and a black Boden top)
• Favoriting and unfavoriting items
• Rating items on Upcoming Styles
  

We conducted participant interviews using Zoom. Users were asked to search for items on the Armoire website, save items for later using the “my favorites” section, as well as to rate items using Armoire’s upcoming styles function. Following each task, we asked our participants open-ended questions, and had each participant complete a google survey questionnaire.

‍

Heuristic Evaluation

The product was evaluated using Jakob Nielsen’s 10 Heuristics and 4-Step Severity Rating Scale. 

Jakob Nielsen’s 10 Heuristics:

• Visibility of system status
• Match between system and real world
• User control and freedom
• Consistency and standards
• Error prevention
• Recognition rather than recall
• Flexibility and efficiency of use
• Aesthetic and minimalist design
• Help users recognize, diagnose, and recover from errors
• Help and documentation

‍

Jakob Nielsen proposed the following four-step severity scale:

0 = I don’t agree that this is a usability problem at all

1 = Cosmetic problem only: need not be fixed unless extra time is available on project

2 = Minor usability problem: fixing this should be given low priority

3 = Major usability problem: important to fix, so should be given high priority

4 = Usability catastrophe: imperative to fix this before product can be released

‍

We first conducted individual heuristic evaluations.

Then, we compiled our results together as a group. 

Study Design

Before beginning this usability study, our team generated a list of research questions that guided us throughout the project. 

Research Questions

They were:

• How efficiently do participants find a desired clothing item from Armoire’s home page?
• Can users find the search bar?
• Can users find “curated for me”, “my favorites”, and “upcoming styles”?
• Can users favorite an item? Do they understand the favoriting process?
• Are participants satisfied with their website navigation experience?
• Are participants satisfied with their search results?
• Are there identifiable usability issues pertaining to the item discoverability process?
• Are there identifiable usability issues pertaining to the “my closet” browsing feature?
• How do the users of the Armoire website rate its ease of use?
• Do participants feel confident when interacting with the Armoire web interface?
• Would users recommend Armoire?

Goals & Metrics

We reorganized this into objectives & metrics to be collected:

These objectives are summarized in our goals slide below:

Participant Recruitment

Our team performed usability testing with 8 participants. Before starting recruitment, we identified the ideal participant sample size range to be 5-8 participants based on factors such as the project scope, single-group participant sample (all relatively novice users), resource needs, and number of overall Armoire subscribers. Participants were offered a bonus item as compensation for their time. 

We focused on new & slightly experienced subscribers to gain insight into the user experience of someone new to the product. We recruited current women-identifying subscribers with accounts less than a year old. It is also important to note that the participant sample subscription plan breakdown reflected the overall actual subscription plan breakdown.

Study Overview

This study was performed as a remote moderated usability study.

Each session consisted of three primary participant tasks:

• Searching for items (both a zoom appropriate top and a black Boden top)
• Favoriting and unfavoriting items
• Rating items on Upcoming Styles
  

During each session, participants would be instructed to think out loud. The moderator would also ask pre-scripted open-ended questions as well as impromptu exploratory questions inspired by the 5 Whys method of interviewing. After the completion of each task, participants would complete a short section of Likert scale questions on a Google Form. At the completion of all 3 tasks, participants would complete the SUS and NPS sections and submit their Google Form.

Study Execution & Analysis

We utilized a script to conduct our study. This usability study script details the tasks the participant completed as well as explicitly defines how the tasks were presented. Participant instructions, notes to the moderator, and miscellaneous note-taking information are included.

At certain points, participants were instructed to fill out a Google Form. This form collected quantitative post-task data, as well as post-study global data (SUS and NPS).

During each session, one person was assigned to be a notetaker, with each member of the team note-taking at least once. They collected both qualitative and quantitative data on this Google Sheet.

Affinity Diagramming - Qualitative Analysis: 

After collecting the data, our team worked together using affinity diagramming to explore common themes in qualitative data. It was also used as an assistive tool to help us generate severity ratings. Our collaboration occurred on Miro both synchronously and asynchronously.

Google Sheets - Quantitative Analysis:

Data was analyzed on a Google Sheets document with multiple sheets. Each sheet provides calculations and details on how data was analyzed.

Example: Screenshot of the SUS data analysis that shows calculations step-by-step.

Study Results & Presentation

We compiled our findings into a summary slide deck and research report that was presented to Armoire. To view the full report, click here.

Here is the summary of our findings:

These tags were used to categorize areas of success and areas for improvement. 

• Efficiently Completing Tasks 
• Confidence in Search 
• Learnability 
• Satisfaction 
• Usability 
  

Here is an example of one of our findings slide: