I planned and executed the generative and evaluative research program for BEAM, an application and browser extension that helps people safely share their logins with their friends and loved ones. The application turns password sharing into an opportunity for improving password security.
I was approached by a team of designers and an engineer who were passionate about building a solution for underserved populations improve their cybersecurity practices.
Good password practices (generating unique and complex passwords for every online account) are impractical to maintain. Only 12% of Americans use password managers to keep track of their passwords.
Although password managers help reduce the memory demand for maintaining large sets of unique and complex passwords, these tools are not a silver bullet for adopting good password practices (Pearman et al 2017).
In my role as the lead researcher, I designed a research process that primarily helped my team develop an obsession with the problem. Over four months, our development team comprising of two UX designers and one engineer, went through numerous cycles of observation, problem framing and re-framing, solution ideation, and empirical solution testing.
I conducted semi-structured interviews to allow flexibility for participants to go into each talking point with as much or little detail as they wish. This type of interviewing style suited our research objectives because our team’s main objective was to understand participants’ password habits without any real expectation of the answers we were going to receive.
I conducted expert interviews with 2 cybersecurity professionals to increase the team’s understanding of the subject matter, and uncover some best practices for online digital security
The second round of interviews were 45 minute conversation with 18 people who ranged from elderly in retirement, adult couples, and people with a high school diploma and no college education. In these interviews my research goals were;
Uncover motivations behind common password generation behaviors.
Investigate why password managers have notoriously low user adoption.
Explore ways to reduce privacy harms that arise from digital risks caused by poor password management.
In addition, I conducted competitive usability evaluations in which participants, who had never used password managers, completed critical tasks in existing password management tools. In these usability evaluations my goals were;
To understand how other password management solutions set up to solve the same design problem
To see what’s working and what’s not from the perspective of users.
Formative Evaluative Research
I implemented research methods that helped the development team to discover insights and shape the design direction in the early stages of our product development.
I lead our team through a Cognitive Walkthrough where we put ourselves into the shoes of our intended user group and walked through scenarios in our low-fidelity prototype. This method allowed our team to cover scenarios and identify a range of issues in our low-fidelity prototypes. When examining individual issues, we considered if the issue could be applied more generally across the product. It also allowed us to inspect the usability of Critical User Journeys(CUJs) within the prototype. The insights gained using this method were used for further iteration on the designs.
In addition, I conducted Rapid Iterative Testing & Evaluation (RITE) to quickly identify large usability issues that prevented users from completing tasks or did not allow the product to meet its stated goals. In employing this method, all members of the team observed sessions, and following each session where a blocking usability issue was identified, the team agreed on a solution.
In contrast to traditional usability testing where five or more people see the same design, in my implementation of RITE, only two participants saw the same design before changes were made for the next session. Our team felt confident that we built a hi-fidelity prototype free of major usability issues after implementing this method with a total of 6 participants.
Insights & Findings
We found that the low adoption of password managers was primarily because;
People believe that they have no “problem” managing
Poor usability of existing password managers
Password sharing while often frame as a bad practice, was very common. This is corroborated by Pew Research data which shows that 41% of Americans share at the password details for at least one of their online accounts. This sharing behavior was very common amongst family and loved ones.
Password sharing facilitates trust and productivity, and there are certain situations in which password sharing is appropriate and necessary. When sharing is initiated people become for conscious of their password generation habits.
Our team reframed password sharing not as a deviant practice to be stamped out, but rather a nuanced practice to be engaged in with thought and care.
To further evaluate the usability, and desirability of the product. I aim to conduct a longitudinal study that explored both attitudes and behavior in the product over a period of three weeks. In the semi-structured interviews, we found that password sharing does not happen frequently. As a result, in the beginning of the study we would ask dyads about what passwords they already share and request that they enter those passwords into Beam, together with any new passwords they would like to share.
This evaluation will not only allow our team to evaluate how the product changes password generation and sharing behaviors, but also can help explore concepts related to trust, reliability, uncertainty, and effectiveness.
Some sample follow-up questions from the longitudinal study include;
How confident are in you in Beam’s ability to keep your passwords safe?
Beam allows you control how you share your passwords (ie: set durations, share without revealing). How reliable were these sharing controls for you?
If you were to explain how Beam works to someone who has never used it, how would you explain it?
Were there any things you were unsure or uncertain about when using the application?