"User Experiences of Spanish-Speaking Latinos with the Frontier Behavioral Health Website"

Download PDF

About the Author

Raquel L. Dean holds a B.A. in Psychology and a M.A. in English with an emphasis in Rhetoric and Technical Communication from Eastern Washington University. She currently works for Kalispell Regional Healthcare as a School-based Mental Health Worker and plans to obtain her Ph.D. in Clinical Psychology in the near future. Her research interests lie in both psychology and technical communication, with topics including: minority mental health, child psychopathology (childhood trauma and anxiety disorders) and evaluating barriers among minorities when accessing and receiving mental health care, by facilitating usability tests to examine information design and observe the users' experience.

Contents

Introduction

Literature Review

Methodology

Test Goals and Objectives

Participants

Scenario and Test Tasks

Metrics

Results

Findings and Discussion

Conclusion

Limitations of Study

Recommendations for Future Research

References

Appendix A: Pre-Test Survey

Appendix B: Recruitment Email

Appendix C: Post-Test Interview

Metrics

This study required a mixed-methods approach to collect data. While each participant navigated through the website and completed each task, I asked users to "think-aloud." Think-aloud protocol encourages the test participants to use the website while continuously thinking out loud—verbalizing their thoughts as they move through the user interface. For example, as a user completed each task, they talked aloud, expressing their thought process and verbalizing why they were making certain decisions (clicking on a link, searching for a keyword). Once the user felt they had completed their task, or found the information they were asked to find, they said “done” or “found it” out loud. As Barnum (2011) states: “hearing from the participant while he or she is working, and learning what pleases, frustrates, confuses, confounds him or her is illuminating,” (205). This information is essential to collecting qualitative data that will illuminate user-related issues that may occur, that are directly tied to the design of the website.

I used the "see-say-do triangle" to collect data, as displayed in Table 2. Still and Crane (2017) developed the see-say-do triangle, as a method for close observation during usability testing. I chose this framework because it allowed me to have a close observation of the user's physical, verbal, and emotional responses to the website—which would deepen my understanding of their overall experience. The elements of the see-say-do triangle include: observing what users do (see), listening to what users say (say), and measuring what users do (do). According to Still and Crane (2017), through the use of the see-say-do triangle, you balance observation, self-reporting, and performance data (191). For the “see” element, the designer would observe what the users are doing while interacting with the design (product). This kind of observation includes user navigation with the design and user behavior (emotional responses, body language). The “say” data is the user feedback you obtain about the design (product). For this study, I utilized think-aloud protocol (TAP), pre-test surveys, and post-test interviews, which are all under the “say” category. According to Still and Crane, “TAP asks users to talk about their thoughts and decision-making processes while completing tasks” (203).

Then, the survey and interview collect data about the user's thoughts on mental health services in general, but also gauge the user’s satisfaction after the usability test (what they find most appealing; what they struggled with) and allow them to write as little or as much as they would like to offer. Finally, the “do” refers to the performance data. This includes the time on task, mouse clicks, and error rates. The time on task refers to the amount of time (seconds) the user spends completing each task. The mouse clicks refer to the number of clicks a user makes, per task. The mouse clicks include clicking on a link, hitting a submit button, etc. Then, the error rates include the number of errors made and the severity of those errors, which may occur during a task. Through the use and collection of data through the see-say-do triangle, I will be able to note the “patterns, errors, gaps, and even incidental actions from different sources, each one representing a different approach: user performance, user verbalization, and designer observation” (Still & Crane, 69). All this data will not only point to the navigating and thought process of a sample of Spanish-speaking Latinos, but also bring out any user-related issues they may encounter with the Frontier Behavioral health website.

I used two metric scales when analyzing the tasks completed by the users. The first was a task completion scale that I used to assign how easy or difficult it was for each participant to complete each given task. This metric scale is the one that is already prepared for use in the MORAE audio-video screen recording software. This metric scale goes along with the efficiency and learnability guidelines of usability testing (see Table 2).

Table 2: Ease of Completing Task Scale

The second metric scale I used was an error severity scale. If there was an error(s) made during a given task, I would assign it a rating, based on the severity of the error (see Table 3). This error scale is the one that was already in place by the MORAE Software. This is important to consider because, if there are a high volume of catastrophic errors occurring during certain sections of the website, this can point to a real design issue.

Table 3: Error Severity Scale

Once I provided the participant with a consent form and it was signed, I pressed record through MORAE and it prompted the pre-test survey. The pre-test survey asked a series of questions that were split into four sections: demographic information, language proficiency, mental health services (if they had ever accessed MHS, how likely they would be to pursue MHS), and technological skills (for full pre-test survey, see Appendix A). After each participant concluded the usability test, I pressed the “stop recording” button through MORAE. This prompted the post-test interview, which was composed of six questions varying from "what did you find most appealing about the website" and "what do you feel is the website’s purpose?" (for full post-test interview, refer to Appendix B).