Research process and sleep app design lessons learned from the reflective examination of a sleep study

Introduction Mobile sleep apps are promising accessible treatment for insomnia. Using them as data collection tools akin to sleep diaries has also been proposed. Most of these apps, however, have not been developed using evidence-based principles; limited research exists on their design as research tools (Bhat et al. 2015) (Yu et al., 2019).In the present study, we explored the opportunities and challenges experienced when using a mobile app for research with our own team’s research study as the unit of analysis. This is an intrinsic case study (Stake, 1995), which can inform other researchers as they use sleep apps in research as an intervention (treatment) or research tool (data collection). Materials and methods:Data were collected during a larger study, designed to test the effects of serial diverse imagining (Beaudoin et al., 2016), using SomnoTest, on insomnia. Data of 19 controls and 15 insomniacs, aged between 18 and 30 years, were analysed. Participants were assigned to one of two app conditions. Group 1 participants heard a countdown from 99 to 1 and Group 2 were prompted to visualise randomly selected brief scenes read by the app at eight-second intervals. Participants completed a one-week sleep diary while using SomnoTest, during the second week. This was the first study to analyze SomnoTest data using a qualitative approach involving direct interpretation of participants’ patterns of mobile app usage based on actions recorded (i.e., press start, end, pause, resume, or cancel; time stamp; count of played items), reorganization of usage patterns into tables (visualisation; tabulation), reflection of researchers on their respective experiences in analyzing the data, and the derivation of themes and selection of exemplars based on participants’ usage and researchers’ experiences.ResultsOur exploration revealed four themes: 1) unreliability of sleep diaries when triangulated against SomnoTest data, given that 9 participants had not used the app as claimed; 2) complex, intensive qualitative analysis is needed to identify valid data in an unstructured data set; 3) importance of visualisation when examining data to uncover patterns; 4) identification of “fans” who continue to use the app after their participation in the study.Our findings reveal that data cleaning involves intensive case-by-case analysis of participant data, which proved challenging with 34 participants and would prove prohibitive for larger scale studies. However, these insights can inform how future sleep studies involving mobile app. ConclusionThe development of an algorithm that can efficiently filter valid data usage patterns would facilitate data analysis and researchers’ experience. This would increase sleep app usability as a treatment and research tool. Developing a process for increasing efficiency in data analysis is necessary to exploit the advantages of large-scale data collection that a sleep app makes possible. Further, informing participants that app data would be triangulated against sleep diary during data collection and analysis might increase the accuracy of the data that participants provide in sleep diaries.AcknowledgementsConflict of Interest Dr. Beaudoin is president of CogSci Apps (develops SomnoTest, mySleepButton and Hook propductivity apps) and owner of CogZest.References:Beaudoin, L. P., Digdon, N., O’Neill, K. & Racour, G. (2016). Serial diverse imagining task: A new remedy for bedtime complaints of worrying and other sleep-disruptive mental activity.Poster presented at SLEEP 2016 (A joint meeting of the American Academy of Sleep Medicine and the Sleep Research Society). Denver, CO.Morin, C. M., Drake, C. L., Harvey, A. G., Krystal, A. D., Manber, R., Riemann, D., & Spiegelhalder, K. (2015). Insomnia disorder. Nature Reviews Disease Primers, 1, 15026.Allocca, G., Ma, S., Martelli, D., Cerri, M., Del Vecchio, F., Bastianini, S., ... & Blackburn, S. (2019). Validation of' Somnivore', a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data. Frontiers in Neuroscience, 13, 207.