O47 Chat with Maya: Assessing the Usability of the Texas WIC Chatbot

Background To address declining participation rates in the Women, Infants, and Children (WIC) program, Texas WIC developed and deployed Maya, the Texas WIC chatbot, in June 2020. Maya is an automated software application that allows visitors of TexasWIC.org to ask questions about eligibility and begin the process of making an appointment. Maya has been developed within a user-centered design framework, with WIC clients, potential WIC clients, and WIC staff consulted throughout the ideation and design process. In continuation of this process, usability studies are ongoing. Objective The objectives of this study were to assess the usability of Maya and solicit user feedback on overall satisfaction. Study Design, Setting, Participants A Think Aloud interview methodology was employed to assess participants’ real-time cognitive processes when using Maya. Participants (n = 9) were recruited after accessing Maya on TexasWIC.org . Interviews were conducted via Zoom with a moderator and assistant moderator. First, participants explored Maya's features with minimal input from the interviewer. Next, utilizing a semi-structured interview guide, the interviewer probed for participants’ perceptions regarding how Maya may facilitate convenient information retrieval for WIC clients. Measurable Outcomes/Analysis Employing a 2-coder structure, interviews were recorded, transcribed, and themed. Results Maya's ability to immediately provide critical information about benefits, enrollment, recertification, and WIC-related frequently asked questions was described as efficient and highly useful. Additionally, participants approved of Maya's overall layout and expressed confidence in their ability to find what they needed. The few issues that were uncovered include minor confusion on how to return to Maya's main menu, some chat responses seeming long, and occasional glitches. Conclusion Maya was well-received with specific appreciation for her ability to efficiently provide information. However, some issues related to functionality indicate room for improvement. Enhancing Maya's usability to ensure easy usage has the potential further facilitate optimal information retrieval and program participation.