Moodfit App Study: Motivational and Behavioral Factors For Long-Term User Engagement

Exploring key motivational and behavioral factors involved in long-term user engagement for the mental health app Moodfit.

Project Details

Background

• The rise in the popularity of mental health apps highlights the potential of these apps to be effective and accessible therapeutic tools for users in addressing common mental health issues, providing a wide range of services, such as information, therapy-based exercises, meditation, symptom tracking, and professional resources. 

• However, long-term engagement on mental health apps is low; according to a study examining usage patterns on mental health apps, there is a sharp decline of more than 80% of app open rates between day 1 and day 10. Other studies have highlighted high drop-out rates as a major issue hindering the adoption, use, and overall effectiveness of these apps. 

• This presents an opportunity to help uncover what factors may be involved in keeping users engaged and active in the long-term, i.e. past the 10-day benchmark, and ideally past the 30-day mark? As well as to provide suggestions for how apps might more effectively incorporate or improve on these features to increase user engagement and long-term retention.

• The Moodfit app is studied as an example of the number of mental health apps currently on the market. Specifically, the Moodfit app is conducive to this study and its research parameters as the majority of the app’s features (features it has in common with its competitors) are free and available without a paid subscription.

• This study was conducted as a personal project.

Research Methods

• Screening survey

• Generative, semi-structured interviews

• Competitive analysis

• Thematic analysis

• Persona-building

Collaborators

• N/A

Time Frame

• January 2020 - April 2020

Tools

• Google Docs / Forms

• Zoom Video Call

Objectives

• The objective of this study was to identify key factors involved in long-term user engagement and retention for a mental health app, and to provide suggestions as to how apps might more effectively incorporate or improve on these features to address the key factors hindering app engagement. 

• The study also looked into features of similar app competitors, and how such differences might be incorporated into improving or refining current app features.

Research Questions

• Business-oriented research question: How can we keep more users active in the long-term?

• General research question: What are users’ motivations in coming back to the app in the long-term?

• Overarching research question: What are users’ motivations in using a mental health app – what are they actually looking to get out of the app?

Process

Recruitment

• I contacted local community and social media groups in mental health and wellness, and asked them to share a description of the study, along with a link to a screening survey that I created, within their social circles.

• The brief survey recorded past use of mental health phone apps, whether the potential participant had used the Moodfit app before, and gauged their interest in partaking in a multi-week app study. For the objectives and scope of this study, no identifying or demographic information aside from a name and contact method were requested.

• With study budget and logistical constraints in consideration, I randomly selected 4 respondents who indicated they (1) had not used the Moodfit app before and (2) had familiarity with mental health apps. Since this study would be held remotely, extra information including participant consent forms, were communicated and sent to the participants in preparation for the first interview.

Interviews + Mini Diary Study

• The following week, I held the first round of semi-structured interviews with each participant remotely over video-call. Each interview was recorded with the participant’s permission, and lasted from 30-45 minutes. The purpose of this interview was to generate insights into the participant’s past experiences and motivations in using mental health apps, and to get to know their beliefs and thoughts at the beginning of the 6-week study.

• I prepared an interview protocol with a number of broad, open-ended questions designed to facilitate exploration into their past experiences with using similar mental health apps, how they liked to use those apps, and what their general thoughts or motivations were when they first downloaded them. (For more details into my interview process, please see my ‘How I Do Research’ page.)

• The interviews were held over two days, to allow for enough time before and after each interview to ensure consistent pacing and delivery quality. Recordings were later transcribed onto a document and qualitatively analyzed. After each interview was completed, participants were asked to download the Moodfit app, and to use the app as they normally would over the duration of the 6-week study. 

• Follow-up interviews with each participant were held every 2 weeks, again utilizing semi-structured questions to explore each participant’s experiences and thoughts of using the app as time progressed. Recordings were accordingly transcribed and coded for analysis.

• On every alternating week where there was no interview, participants were asked to write a brief reflection entry through Google Docs describing their use of the app and impressions thus far. Reflection entries were qualitatively analyzed on a rolling basis, and insights gained from them further informed individual interviews.

• After completion of the final interview and qualitative analysis, I prepared a preliminary findings document compiling emerging themes and salient notes from the data.

Interview Insights

• The first round of interviews established a baseline understanding of each participant’s history and ‘style’ of using mental health apps, and with what motivations and expectations they might have approached previous apps with before. Early coding and thematic analysis suggested that the participants’ general motivations and behaviors could be split into two general groups, under the ‘serious’ and ‘casual’ users. 

• Analysis of follow-up interviews and reflection entries over the 6-week period generated further insight into participants’ thoughts and impressions as they progressively used the app, and expanded these preliminary groupings into themes: 

• 1) Serious users who see the app as a potential therapeutic aid, not simply a tool, who are motivated to invest time and effort into long-term use of the app. The main frustrations leading to disengagement include:

  • Lack of new insight gained from long-term use of activities, e.g. use of mood tracking or therapeutic exercises within the app, with no new information or personalized insights into user’s emotional or behavioral trends over long-term collection of personal data, feels “generic and not relevant to my situation or [data] I’ve been tracking”.

    • Users feel as if their growth and long-term changes are not reflected with the app over the long-term, resulting in decreased motivation to continue long-term engagement. 

    • Recommendation: The Insights tab could be re-oriented to prioritize providing longitudinal feedback or actionable insights based on behavioral patterns; activities in the Tools tab could be organized by intention/motivation (i.e. Stress, Mindfulness, etc.) rather than by individual activities.

      These may encourage users to utilize the data-logging feature and enable them to receive personalized insights that develop over time, fostering engagement with the app as a long-term therapeutic aid.

  • Low frequency of new content, or the feeling of the app being ‘static’ and unengaging, due to lack of new activities, improvements in usability, knowledge, etc. Lack of updated material decreases long-term interest in keeping up.

    • Recommendation: Alerting users to new features that have been added or sharing updates with users; a feedback tab or feature might be included on the app that allows users to submit comments or suggestions on improvements or activities and features they would like to see, which may facilitate feelings of app growth, reciprocity and engagement with its users.

  • Usability problems that interfere with consistent use, e.g. unclear instructions on how to navigate a feature, as well as lack of data portability for external use 
    

• 2) Casual users who primarily use the app’s reminder and goal features (i.e. to plan and track exercise, medications, other habits) and don’t seek to spend much time on the app itself. Their main frustrations include:

  • The app’s features and activities feel tedious or like “extra work”, e.g. undertaking the same therapeutic exercises with no new information or content updates to incentivize repeat visits to the app, or activities like “feel like filling out worksheets” that don’t motivate engagement.

    • Recommendation: Regularly updating content or providing new activities or journal prompts could reduce feeling of repetitiveness; rewarding users for regular use may incentivize or encourage continued use (e.g. “streaks”, badges).

      Alerting users to new features that have been added or sharing updates with users may facilitate feelings of app growth and engagement with its users, and adding a gamification feature may further incentivize users to return to app.

  • The high density of information supplied during activities or exercises, leading to potential cognitive overload. Being overwhelmed by too much background or scientific information while completing an in-app task.

    • Recommendation: Separate and display only relevant content when it is needed (in-line vs. upfront vs. toggle display). This may help to reduce the potential overwhelming amount of text and information included with the activity, streamline task flow and allow user to choose when to read and access supplemental information.

  • There was positive engagement and increased motivation with tasks and content in the app that could be modified or customized to their lifestyle, i.e. personalizing goals and reminders, customizing tracking variables.

• For both groups, consistent and motivated app use is strongly associated with the usability and ease of incorporating the app’s features into their daily lives – including features like visual appeal, content credibility (trustworthiness), personalization (personalized insights and customization e.g., tracked habits), and regular content updates (e.g. new or improved exercises, information).

• Summary: These two general groups of behaviors and attitudes informed the favorability or frustrations voiced by the participants with the certain features available – or lacking– on the Moodfit app. Reframing current features to address these frustrations and motivations that better complement these two user styles may improve overall app experience and effectively increase long-term engagement and use.

Competitive Analysis

• The findings from the participant interviews showed that a range of key app features were perceived favorably and appeared to hold or increase in interest and engagement over time. To gain an understanding of how other similar leading mental health apps employ – or don’t include – these key features, I carried out a competitive analysis between Moodfit and the three apps Sanvello, What’s Up, and MoodSpace. Feature comparisons between each app were recorded in a matrix.

• Apps were chosen for similarity of therapeutic focus and themes (use of CBT-based techniques, sharing fundamental features such as a journal and meditation activities), and popularity (based on Google Play rating and number of downloads) . 

Competitive Analysis Insights

• Relevant themes: Of the four apps analyzed, only Moodfit and Sanvello utilize a long-term mood tracking feature (with the ability to track other variables available in the subscription versions), and feature content backed by professional sources or is explicitly sourced. 

  • Previous qualitative data suggests that content credibility (defined here as trustworthiness in information and features supplied by app, i.e. scientific information or therapeutic exercises are evidence-based, or professional sources are cited) is important among both user styles in building app reliability and therapeutic persuasiveness for the user. App tracking features that allow users to follow their data trends over time may incentivize users to continue app engagement in the long-term.

  • Sanvello’s mood tracking function allows adaptive feedback on user’s inputted data, such as recommending mental health practices for specific moods. Competitive analysis shows that none of the other apps utilize an adaptive feedback function.

  • Three apps have the ability to export or share data and activity logs, which is an important factor in data portability and external use emphasized by users in the participant interviews. Being able to transfer and share their data with others, such as health providers or therapists, may be a helpful tool for users looking to expand their use of their data and the app.

• Recommendations:

  • Adding or updating content to be scientifically and clinically credible may increase trust in the app and the content’s persuasiveness, and may build therapeutic engagement with users, a key thematic factor in user motivations to return to mental health apps.

  • Implementing a tracking feature (i.e. logging mood, habits, and/or other variables over time) may incentivize users to access the app and input their data more frequently, and if adaptive feedback is available for the tracking feature, might be an additional motivation for users interested in gaining new, more specific insight to their data over the long-term.

  • Exporting data or activity logs in a shareable format (e.g. a spreadsheet or image file) may provide users with other ways to more usefully incorporate the app into their mental health practices outside of the app, such as sharing with health providers or therapists.

Thematic Analysis

• After each round of participant interviews and weekly reflections, qualitative data was coded and thematically analyzed for users’ behaviors, motivations, and frustrations while using the app during the study. These were then further grouped into a thematic diagram with relevance to answering the research questions and providing actionable insight in mind. 

Thematic Analysis Diagram

Persona Building

• From qualitative analysis and identified motivational and behavioral patterns between the two user styles, I constructed two personas reflecting their key factors and primary frustrations blocking optimal long-term app use, emphasizing insights that can be targeted to improve those experiences.

Key Questions - High-Level Takeaways

Aside from the specific design recommendations and insights above, here are the main takeaways from the research answering the original key questions.

Business-oriented research question:
How can we keep more active users in the long-term?

Takeaway: Building app trust and ensuring content credibility allows a foundation for recurring use of the app. This can be supported by personalization and personal feedback (personal investment).

General research question + Overarching research question:
What are users’ motivations in coming back to the app in the long-term?
What are users’ motivations in using a mental health app – what are they actually looking to get out of the app?

Takeaway:
Key motivations:
1) Motivationally-based; ‘serious’ users are able to invest their time and efforts into the app reliably and insightfully over time, building therapeutic engagement
2) Behaviorally-based; the app is easy to incorporate into ‘casual’ users’ routines, with focus on ease of use, visual appeal, and clear directions

Challenges

• The study budget and logistic constraints limiting the number of participants in the study introduced potential sampling bias in representing the range of users who utilize these mental health apps. Ideally, with a larger research team and study budget, more participants can be recruited into the study.

• Due to participants being recruited from social circles focused on mental health and well-being, there is potential for selection bias as those who volunteered for this study are not representative of all users who utilize mental health apps. However, for the purposes and scope of this study, following the participants who due to this background may be generally more inclined to invest in sustained app use, might be more helpful in garnering insight into the factors involved in keeping – or losing – attention and long-term engagement with these apps.

• Coordinating biweekly interviews and reflection entries with each participant, communicating participant information and answering questions, while conducting rolling qualitative analysis on data could amount to a lot of work at times, and required being flexible and effective with organizing study materials and managing time.

Other Considerations

• As this was a solo research project, had there been stakeholders or other collaborators involved, communicating feedback at various points during the process – for example engaging stakeholders with the study process or involving team members with qualitative analysis – would be important to ensure everyone is informed and on-board with the direction of the study. 

• With a larger study budget and/or research team, the research strategy might be modified to include a larger number of participants, use of focus group discussions, or in place of the biweekly brief reflections, a diary study to collect valuable ethnographic, longitudinal data.