A mixed methods evaluation of a pilot open trial of a mentor-guided digital intervention for youth anxiety
Department of Psychology, University of Virginia
How they used Delve
Clinical psychology researchers at the University of Virginia used Delve to code post-intervention interviews with seven adolescents and seven mentors who piloted a mentor-guided anxiety app. The team built separate mentee and mentor codebooks, trained coders to a Krippendorff's alpha of .80, had two coders independently code each remaining interview, resolved discrepancies by nominal group consensus in weekly meetings, and uploaded the final version of each transcript in Delve.
“Interviews were coded using Delve qualitative data analysis software [63] and coders were trained via intercoder reliability tests (ICR). Intercoder reliability was measured using Krippendorff’s alpha, a measure of reliability in content analysis [64]. Once an ICR of .80 was reached, two people from the team were assigned to independently code each of the remaining interviews, and coding discrepancies were discussed among the research team in weekly meetings. [...] After each transcript was discussed, a “final” version was uploaded in Delve.”
- Field
- Clinical psychology / digital mental health for adolescents
- Data
- Post-intervention interviews with 7 adolescent mentees and 7 adult mentors from 15 mentor-mentee dyads in a pilot trial of a mentor-supported CBM-I app, alongside questionnaire and program-use data
- Approach
- Mixed-methods pilot evaluation; post-intervention interviews analysed by inductive content analysis with separate mentee and mentor codebooks, coders trained to Krippendorff's alpha of .80, then two coders per transcript with weekly nominal-group consensus meetings
- Data types
- Interviews, Open-ended survey responses
Abstract
Digital mental health interventions (DMHIs), such as cognitive bias modification for interpretations (CBM-I), offer promise for increasing access to anxiety treatment among underserved adolescents, but data regarding their efficacy are mixed. Paraprofessionals and other caring adults in youth’s lives, such as non-parental adult mentors, may be able to support the use of DMHIs and increase teen engagement. The present mixed methods evaluation of a pilot open trial tested the feasibility, acceptability, and preliminary efficacy of implementing MindTrails Teen (an app-based, youth-adapted version of the web-based MindTrails CBM-I intervention) within mentor/mentee dyads. Thirty participants (composed of 15 dyads) participated in remote data collection for 5 weeks. A subset of participants (n = 7 mentors; n = 7 mentees) also provided qualitative feedback. Intervention outcomes (change in anxiety symptoms, and positive and negative interpretation bias), feasibility, and acceptability were assessed via a mix of qualitative interviews, quantitative change in questionnaire scores, and program completion and fidelity metrics. Outcomes were compared to pre-registered benchmarks. Large effect sizes were observed for changes in anxiety among youth. Small to medium effects were observed for change in positive interpretation bias, and no change was found for negative interpretation bias. Intervention outcomes should be considered with caution given very low internal consistency of the interpretation bias measure and the lack of a control comparison group. Acceptability of the intervention was rated positively by mentors and youth. Feasibility benchmarks were met for mentors but not for youth. Qualitative feedback indicated mentors perceived the app as helpful to their mentees, found that it either improved or did not affect their relationship, but also identified implementation challenges. Youth overall perceived the app as helpful but identified barriers to engagement.
Citation
Emma C. Wolfe, Alexandra Werntz, Audrey M. Michel, Yiyang Zhang, Mark Rucker, Mehdi Boukhechba, Laura E. Barnes, Jean E. Rhodes, Bethany A. Teachman (2026). A mixed methods evaluation of a pilot open trial of a mentor-guided digital intervention for youth anxiety. PLOS Digital Health. https://doi.org/10.1371/journal.pdig.0001187