A mobile intervention to reduce anxiety among university students, faculty, and staff: Mixed methods study on users’ experiences

How they used Delve

Psychologists at the University of Virginia used Delve to code 22 interviews about a mobile anxiety intervention, training four coders through six rounds of intercoder reliability testing to a Krippendorff's alpha of 0.83 before double-coding 27% of transcripts and moving to single coding.

“Interviews were coded using Delve qualitative data analysis software [59]. We followed an inductive approach of open coding to allow for unexpected, data-driven themes to emerge. [...] Four coders were trained by conducting inter-coder reliability (ICR) tests. Inter-coder reliability was measured using Krippendorff's alpha, a measure of reliability in content analysis. We underwent six rounds of ICR testing and reached an ICR rating of 0.83. Each round of testing involved independent coding of the same transcript by each coder, followed by comparison and discussion of coding differences.”

Field

Digital mental health; user experience and acceptability of a Cognitive Bias Modification for Interpretation mobile app for anxiety across a university community

Data

Semi-structured interviews with 22 users of the 'Hoos Think Calmly' mobile app - undergraduates, graduate students, faculty and staff at a large public university - transcribed with Descript and checked against the audio; the mixed-methods design also drew on biweekly user experience questionnaires from 134 participants in the parent trial's active treatment condition

Approach

Pre-registered inductive thematic analysis following Braun and Clarke, coded at the level of meaning. Team members reviewed transcripts against the audio, noted initial themes, and agreed a starting codebook; codes were then added and removed iteratively and grouped into themes by consensus. Four coders were trained through six rounds of intercoder reliability testing, each round independently coding the same transcript and then comparing and discussing differences, until Krippendorff's alpha reached 0.83. 27% (6 of 22) of transcripts were double-coded before moving to single coding by two remaining coders, with weekly meetings to resolve coding challenges and the lead author deliberately not coding so as to moderate disputes. Saturation was reached at 15 transcripts. The final codebook held 47 main codes (excluding sub-codes) grouped into 10 themes, reduced via thematic mapping to five reported themes. Every codebook iteration is posted on the Open Science Framework.

Data types

Interviews