Puja Desai
Symbiosis School for Liberal Arts

Abstract

The advancement of technology and the increased reliance on mobile applications have impacted various aspects of life and are capable of influencing and changing behaviour, with an increasing number of the population becoming interested in self-management of mental health. Some of these applications have been designed to provide access to therapeutic care beyond traditional means. Several studies have been conducted in order to explore the efficacy and reliability of these applications; however, not much research has looked into the possible use of these apps simply as an alternative approach to therapy, or as a first aid measure to be used alongside traditional therapy. A majority of these studies have focused on the quantitative aspect, to develop evidence-based recommendations or to create better applications. The current pilot study aims at exploring the views of counsellors on the viability/efficacy of mobile based mental health applications. This would provide deeper insight into the realistic use of this platform, since they may have encountered clients who have used these applications, or recommended the use/ disuse of the same, as well as establish a base for future research. This is a qualitative study, which was conducted through semi- structured interviews. Participants included seven mental health practitioners who were active in their field. The methodological approach of this study was chosen in order to obtain rich data regarding counsellors’ perspectives on the matter. Results reflect the opinions of the interviewed therapists, indicating that mHealth apps are effective as supplements to traditional therapy, but not as replacements to therapy.   This research provides valuable insight to practitionersand clients, as well as app designers, regarding the efficacy of app-based interventions. However, the findings cannot be generalised owing to the limited sample size. Future research should delve deeper and distinguish between apps based on their function.

Introduction

The advent of the internet has brought about major changes in the ease of accessibility and acquisition of information, as well as means of communication worldwide. The advancement of technology and the increased reliance on mobile applications has impacted various aspects of life. The abundance of these mobile health apps and their ease of access have revolutionized the way we look at health care. Some of these applications have been designed to provide access to therapeutic care beyond traditional means. It has also been indicated that apps are capable of influencing and changing behaviour, with an increasing number of people becoming interested in self-management and monitoring of various aspects of their lives, including mental health (Luxton, McCann, Bush, Mishkind & Reger, 2011; Barak & Grohol, 2011). With the rapid increase of the development and use of such a platform, organisations such as the WHO (2015) have begun to get involved; according to their Mental Health Action Plan they have also stressed the promotion of self-care via mobile health technology.

There are several types of mental health apps, based on their function. For example, there are applications that diagnose conditions such as depression and eating disorders, apps that track mood and behaviour, apps that aim at positive affirmations, journaling apps, CBT based apps, apps that offer mindfulness and relaxation techniques and many more (Anthes, 2016). They can also be segregated into broader categories as preventive or curative apps.

Several studies have been conducted in order to explore the efficacy and reliability of these applications; however, not much research has looked into the possible use of these apps simply as an alternative approach to therapy, or as a first aid measure used alongside traditional therapy. A majority of these studies have focused on the quantitative aspect, to develop evidence-based recommendations or to create better applications.

The purpose of this pilot study, therefore, is to gain insight into the perspectives of mental health practitioners on the efficacy and viability of mobile mental health apps. The term ‘mHealth apps’ refers exclusively to mobile mental health apps in this paper, and must not be conflated with all mobile based health applications. The definition of such apps is based on the classification by U.S. National Institute of Mental Health (NIMH) into six categories as per functionality: “self-management, cognition improvement, skills-training, social support, symptom tracking, and passive data collection. Mental health apps cover all stages of clinical care provision, including immediate crisis intervention, prevention, diagnosis, primary treatment, supplement to in-person therapy, and post-treatment condition management.” (Chandrashekhar, 2018; NIMH,2017).  The current study includes all mental health apps under a single umbrella term.

Review of Literature

The literature review begins with the basic facets of mental health apps, and later examines papers which study the efficacy and reliability of this platform. A study conducted by the University of Sydney used qualitative content analysis to explore how mental health applications frame the concept of mental health and illness, and the suggestions resulting from this framing regarding the causation and management of mental health issues (Parker et.al., 2018). The study found that mental health apps (mHealth) could lead to overdiagnosis since these generally market mental health issues as being highly prevalent and common, and tend to hold individuals responsible for their well-being. Parker et.al. (2018) suggested that marketing techniques should also provide alternative views and highlight what constitutes mental health problems vs. the normal ups and downs of life, in order to prevent the problem of overdiagnosis. However, a study conducted in Germany (Becker, 2016) found that awareness of the existence of these apps was considerably low among young adults. The participants in this study also questioned the effectiveness of mobile based applications, with concerns about personal information being released, in spite of their easy usage. This suggests that there is a difference in popularity and marketing of these apps in various countries, highlighting the need for research to be conducted in India. Becker’s study also suggested that the apps should be marketed more, as supporting tools for mental health treatment.

The most common mHealth applications target anxiety disorders, depression, and bipolar disorder. Increasing evidence is revealing the effectiveness of these apps in targeting specific issues. Unfortunately, only a meagre number of these have evidence proving their efficacy. A review of 52 apps targeting anxiety disorders (Sucala et al., 2017) found that only 2 had sufficient supporting evidence. It has also been found that most of these apps do not even use or apply the established strategies to target GAD (generalized anxiety disorder), which indicates a lack of psychological expertise while developing the apps. In Canada, mental health application stakeholders structured nine principles to guide the framework of mHealth apps and identified 15 important benchmarks to assess these applications. This was done to ensure inclusion and effectiveness with regards to their societal norms and to aid and guide future programs or policies (Zelmer J., et.al., 2018).

Several studies have also created prototype applications to test the efficacy and viability of mHealth apps. A trial examined mindfulness-based intervention mobile apps and the viability of the delivery of positive intervention in such a format (Howells, Ivtzan, & Eiroa-Orosa 2014). This study emphasized the importance of using empirically supported content in creating their app, which has been previously identified as a limitation (Sucala et al., 2017). The study revealed that mHealth apps are in fact viable at enhancing well-being through happiness seeking strategies and good “person-activity fit”. Macias C. et. al (2015) created another such prototype application which aimed at promoting the physical well-being of adults with mental disorders, primarily through walking, and also provided non-physical interventions via communication with peer staff, etc. They found that older adults benefited from using such an application, with reportedly increased motivation for physical exercise.

Data has suggested a prevalence rate of about 20% for mental health issues in children/adolescents below the age of 18. These problems may extend well into adulthood, and affect the child in various ways (Merikangas KR, et.al., 2010). However, a negligible amount of apps have been created that target this age group specifically, and understanding their needs would result in more informed development of this modality in the future. Kenny, Dooley & Fitzgerald (2014) received adolescent feedback which emphasized the significance of discretion associated with mHealth apps and consequent avoidance of stigma surrounding mental issues.  Since many young adults do not often seek physical treatment, and rely on mobile based apps to help alleviate their symptoms, Radovic and colleagues (2016) categorized all the apps available through content analysis, and found that the most common purpose was symptom relief. They concluded by stating that clinicians should provide guidance to their patients regarding mental health apps and the utility of these. This may be suggestive of clinicians’ support towards the use of these applications, depending on their efficacy and effectiveness. A review concluded that there is a large gap with regards to evaluations of the efficacy, safety and effectiveness for such mobile based applications targeting children and adolescents, with insufficient evidence to determine whether mHealth applications can be effectively used by this age cohort (Grist, Porter & Stallard, 2017).

A comparison of three types of programs- mindfulness based, CBT based and self- compassion, revealed that these types of mobile based interventions are in fact effective in reducing distress and increasing psychological well-being of individuals. Cognitive-behavioural approaches have received substantial empirical support, and mainly reduce stress, promoting wellness and emotional awareness (Mak, et. al 2018). A recent review has shown that CBT based apps have consistently shown a reduction in stress, at least in the short term. The long-term effects still remain largely unknown (Rathbone, Clarry & Prescott 2017). It has also been established that mHealth apps can suitably administer CBT psychoeducation. Mindfulness techniques have recently gained significant traction, and are evidenced to effectively use self-regulation and awareness methods to alleviate stress. This method of training has been found particularly useful in helping university students deal more effectively with their anxiety and depression (Mak et al, 2015). A review looking into the efficacy and feasibility of such mental health applications or SMS services, over several years, segregated feasibility into target categories (Rathbone, Clarry & Prescott 2018). They found that 3 apps from those included in their review had a significant effect on the reduction of stress and anxiety, as well as depression, with a small effect on anxiety and a profound effect on stress. Extended use of the apps presented more effective results. Overall, phone based mental health applications were found to be efficacious and viable. In a psychotherapy context, a review, also focusing on examining the efficacy of these apps, revealed that there was not sufficient evidence present for any specific application to be considered as evidence-based (Lui, Marcus, & Barry, 2017). It also discusses the presence of several methodological matters that could possibly complicate the final conclusion regarding the effectiveness and treatment outcomes. The authors understand the importance of the implication of these mobile apps and discuss problems for practitioners, and discuss the cautiousness that must be employed by clinicians in recommending these to their patients or incorporating them in treatment. Another review supporting this finding had been conducted by Griffiths and colleagues as early as 2006 This review focused only on anxiety and depression targeting applications.

A recent paper studying the efficacy of smartphone apps for subthreshold depression (Takahashi, Takada, & Hirao, 2018) created an app designed to increase self-confidence. This study methodology was the first of its kind, since it developed a stop-motion picture reproducing app. The results credited the safety of this application. Results also revealed that the app could prove an effective method in reducing symptoms of subthreshold depression in young adults owing to the positive words used in the video.

Michaela Sprenger, Tobias Mettler & Jorge Osma (2017) conducted a study to explore the perspectives of healthcare professionals, an objective much like that of the current study. The authors explored four aspects: “(1) the intention of health professionals to use and recommend e- mental health applications, (2) how this intention of health professionals might be influenced, (3) which group of health professionals might be most accessible to promote e-mental health applications for maternal depression, and (4) for which tasks they rate them to be most useful.” (Sprenger, Mettler & Osma, 2017). On analyzing the collected data, they found that mental health professionals would generally recommend the use of mobile applications, but their attitudes towards these varied depending on each specific case as well as the usefulness of the app in the treatment process. Another such study by Matthews M., and Doherty G., (2011) revealed that therapists often expressed concerns regarding security and privacy and increased access to the user’s confidential information. Therapists also indicated a lack of confidence in technology (as cited in Grist, R., Porter, J., & Stallard, P., 2017).

Given that an extensive proportion of the population spend most of their time online, especially the younger generation, as well as the high rates of anxiety and depression there is an increasing need for psychiatrists and counsellors to become familiar with the concept of e-mental health and associated applications (Carli, et al., 2014). This is an important element to better understand the perspectives of their client and their life context. People have a tendency to portray themselves online in a way that is inconsistent with their real lives; and the use of apps presents an opportunity to indulge in avoidance behaviours such as avoiding social interactions (Morriss, 2015). Secondly, the rapid increase in the number of mHealth apps, and the need for the development of more effective, empirically supported apps has been highlighted in earlier sections of this review (Sucala et al., 2017). The familiarization of practitioners with mental health applications would aid in development of evidence-based apps. Apps can also be utilized and incorporated into therapists’ personal practices in a positive way. For example, there are applications which allow an individual to record their moods and behaviours through the week, which can be used by practitioners to track their progress and add to the various ways patients can self-manage or reinforce their treatments (Morriss, 2015; Kroenke, Spitzer, & Williams, 2001; Powell, 2011). However, prior to implementation of these apps into treatment, aspects such as clinical workflow, the culture and socioeconomic class of the patient and their day-to-day life must be considered. Chan S. et al. (2017) suggest that apps should be reviewed from the perspectives of clinical, business and information systems and development or implementation teams should interview potential users and identify their needs and perceptions. The American Psychiatric Association (2017) has also recently published an evaluation model specifically listing guidelines and discussing the importance of evaluating mHealth apps which should be considered by developers or implementors.

To summarize, through this review of literature it can be assumed that the popularity and awareness of phone-based interventions varies across the world. However, all applications seem to be marketed similarly and tend to normalize the prevalence of psychological issues. The numerous studies and reviews conducted testing the efficacy, feasibility and viability of these app-based programs all seem to support the general findings highlighted above. The implementation of such apps into therapy can prove to be a valuable addition if specific aspects such as culture are considered. Therapist familiarity with such platforms is increasingly important. However, only two studies were found indicating the opinions and perspectives of health care practitioners.

Methods

Study Design

The current study attempts to explore the central research question which is:Do mental health practitioners believe in the efficacy and reliability of mobile based mental health applications?

This would provide deeper insight into the realistic use of this platform, since practitioners may have encountered clients who have used these applications, or recommended the use/disuse of the same. Given the exploratory scope of the study, all mobile based applications involved with mental health were included, without differentiation based on their function. For the purpose of this study, mental health apps were considered as any applications which could be downloaded onto mobile devices, and involved or addressed some mental health issue in any possible way, under the classification by U.S. National Institute of Mental Health (NIMH, 2017).

This is a pilot study, and thus does not include any specific variables. Since previous studies have all been largely quantitative in nature, this study wishes to provide a more qualitative point of view in understanding popular opinion. The study was conducted through semi-structured interviews, exploring opinions regarding certain general aspects such as awareness, recommendations/ incorporation into their personal practices, attitudes and beliefs and concerns. The questions were used as prompts to further the conversation, and participants were allowed to freely discuss their thoughts and opinions, fully guiding the direction of the interview. All interviews were conducted over the phone, and were recorded following participant consent. These were later transcribed and the data obtained was analysed. The responses were analysed in the same order as the questions listed below.

Participants

This study employed a total sample size of 7 mental health practitioners, between the ages of 28-60 years. The term mental health practitioners included counsellors and therapists, excluding psychiatrists. All participants were from the Mumbai-Pune area, and were included on the basis of convenience, the only requirements being that they were currently active in their field and had been working in the mental health care field for a minimum of 3 years.  Participants were given numerical labels from P1-P7.

 Interview Questions

1) Are counsellors aware about the existence of these mobile based apps?

a. If yes, are there any specific applications they know of?

2)  Have they previously used/ recommended this app?

3)  Would they incorporate these applications in their therapy sessions?

4)  Mobile health apps vs. traditional therapies

5)  Mobile apps as first aid measures (example- for anxiety symptoms/attacks)

6)  Any concerns practitioners may have (example- misuse, misleading tendencies)

7) General attitudes towards the apps.

Results

This section reveals the responses obtained from the interviews conducted, and attempts to answer the research question. It aims to build on the existing studies which have proved the effectiveness of these applications as treatment interventions for a number of conditions, by also providing insight into the safety or benefits or demerits of using the apps, as per counsellor suggestions. The results could also provide insight to app developers, by highlighting concerns the counsellors may have about various aspects such as privacy or using evidence-based therapies.

Question 1 explored counsellor awareness with regards to the existence of mobile based mental health applications. Of the 7 participants, only one participant (P1) was unaware of any kind of mHealth apps. 5 participants were well versed in these apps, whereas one participant had only heard about the presence of mHealth apps but was unfamiliar.

Some apps that were mentioned by participants were Headspace, Reflect.me, BetterHelp, wayForward and What’s Up. Additionally, 6 participants mentioned their knowledge of journaling apps and reminder apps.

Question 2 attempted to inquire about participants’ personal use of the app, or any recommendations they might have made to their clients. Since P1 was completely unaware this question was redundant. 4 participants had recommended some kinds of mHealth applications to their clients, with one participant having personally used an app, while 2 participants reported never using or recommending them.

The participants who recommended the use of applications mainly suggested relaxation, journaling or reminder-based apps. Participant 7, who had personally used this platform, spoke about Cognitive Behaviour Therapy based apps as well as symptom recognition apps. One participant mentioned that only preventative apps would be recommended, not curative apps. One of the participants (P3) who had not recommended the use of this platform also stated that journaling apps did not seem to be “extra useful, pushing the person to do those things.”

Question 3 examined the likelihood of participants to incorporate mHealth apps into their practice in the future. Participant 1 stated that they were highly unlikely to incorporate any mobile based apps. Another participant (P3) reasoned that they would not incorporate such apps since they practice process-based therapy and “don’t require much from … clients apart from journaling.”

Four participants responded that they were likely incorporate some type of mHealth app, with one participant (P7) adding that they would do so “only with the younger, tech-savvy generation”. The last participant had mixed opinions, stating that they might be inclined to incorporate mobile apps only if the client did not have access to traditional therapy.

Question 4 investigated participant opinions regarding traditional therapy versus mobile based apps, and whether mHealth apps could act as a replacement for therapy. All participants strongly believed that applications could not be used to replace traditional therapy. Participants who had previously showed support towards such apps stated that they may be used as an “adjunct” or “additive” to traditional forms of therapy, or even as a “starting point”. Some participants also mentioned that they could be powerful tools if incorporated correctly, but held that apps could not replace face to face therapy. For example, Participant 6 stated that “they are useful for writing down your thoughts when you’re in the train, or waiting somewhere, because people don’t have time these days. So they can be supportive in nature, but they aren’t therapeutic as such.”  Three participants also added that the use of apps as the only intervention would not be advantageous since important non-verbal cues cannot be recorded, which is a key aspect in understanding client conditions.

Question 5 attempted to explore whether participants believed that mHealth apps could be used as first aid or immediate response measures. Almost all participants indicated that while they could not be fully used to completely eliminate symptoms or as first-aid treatment, they could provide distractions. One participant (P2) stated that “they cannot be used as first aid, but may be used as wellness tools or precautions before, say, anxiety attacks. I do believe that they may or may not work for everyone.”

Responses also indicated that participants believed in the use of mindfulness-based apps as they provided relaxation. One participant (P4) stated:

I would imagine if I am working with a client, then I and the client would have worked out possible ways that they could help themselves outside sessions, I do not believe that they would need to access an app.

Question 6 aimed at discussing any concerns the participants had regarding the existence of such mobile based mental health platforms, mainly pertaining to concerns about misuse or possibly being misled by apps. Participants responded that privacy was one of the main concerns revolving around such platforms, since personal data could be easily accessed by other people. One participant mentioned the risks of apps that connect individuals to therapists, since the real qualifications of the “so-called professional” could not be determined. Five participants also highlighted marketing issues, relating that apps could mislead consumers into believing that they may “achieve amazing results” and that “apps are aids not solutions and they should be put out like that”. One participant expressed concerns about subliminal messages that could possibly exist within these apps and may mislead and be potentially harmful to consumers.

Question 7 explored general attitudes of the participants towards these applications, finding that only 2 participants displayed strong negative attitudes. While 4 participants did not indicate negative attitudes, they did not strongly support mHealth apps either. One participant (P2) responded that “behaviour tracking, journaling and daily health routine-based apps are useful. These apps can also be used by people not seeking any treatment, or those who do not have any mental health problems on a daily basis.”

Another participant (P3) also reflected that there is a need for “proper guidance when it comes to using apps like this.” Only one participant seemed to have a stronger positive outlook, believing that apps could be “a bridge to connect people to the right source and they have so much potential, especially for the younger generation, they just have to be used with caution” (P4).

Since this was a semi-structured interview, participants made a few additional comments in the form of recommendations. One participant (P5) emphasized “cautious use of apps” and said that “there is a need to understand and know the origins of an app, in order to understand the reliability and safety.” Another participant (P6) also mentioned that“these apps, if used correctly, can increase scalability. Through such platforms we as therapists can reach so many more people, and help those who for some reason or the other cannot access a therapist”. The participant also cautioned that since apps are based on algorithms, which prevents them from being able to adequately respond to individual issues, they “do not consider the infinite possibilities of human conditions, and so they can only be used as additives.”

Discussion

Results clearly reflect the opinions of the interviewed therapists, and whether they support the use of these applications as supplementary therapeutic measures, along with their recommendations for use. This finding was also highlighted in studies reviewed previously (Becker, 2016) in terms of the marketing of apps as supportive tools or aides. Overall, results revealed that while some participants support the use of mobile based mental health applications as an addition to therapy, or were hesitant, there was a unanimous conclusion that they could not be used in place of traditional methods. Most participants also endorse apps that were mindfulness based or involved journaling and reminders, due to their ease of access when “on the go” (participant 6).  Additionally, Mak W.W. et al. (2015) highlighted the existence of empirical evidence supporting mindfulness techniques, and the current study substantiates those results. Apps do not consider the duration of the condition, and tend to leave out several important data points including the “tone of voice and intensity of the problem” (participant 6). However, applications have the potential to greatly reduce certain anxieties that normally arise such as self-identity, etc. by allowing an increase in scalability. Given the increased use of and reliance on technology, the use of mHealth apps may also prove to be motivational, in terms of encouraging journaling behaviour and homework suggested by the practitioners. The use of apps as supplements to traditional therapy, as indicated by participants, is supported by previous findings stating the efficacy of apps in enhancing the well- being of individuals through happiness-seeking techniques such as physical activity (Macias et al., 2015).

 Their concerns regarding an online platform such as this also converged, with privacy being the most common apprehension. Privacy concerns included aspects such as misuse of personal health information, lack of knowledge of the identity and background of developers, identity of ‘counsellor’ with respect to interactive apps, etc. This finding was consistent with previous studies (Becker, 2016; Matthews, & Doherty, 2011) which reported the importance of privacy despite the easy usage of apps. An additional concern was the possible lack of expertise or psychological background of app developers, which may lead to negative consequences for users. This concern was also reflected in a study by Sucala et al. (2017).

With regard to awareness, only one participant was completely uninformed about mHealth, indicating that there is sufficient knowledge and familiarity within the Indian context regarding the presence of mobile applications. Mobile based apps are versatile and have the potential to incorporate several features that are used in traditional therapies, however, the results will always be limited. The integration of this platform into therapy practices requires careful consideration of both patient and clinical workflows (Chan et al. 2017), accessibility and several cultural factors. Before recommending the use of mHealth apps, practitioners should thoroughly investigate the ethics and terms and conditions of the application, in order to protect client privacy. Additionally, they should attempt to find culturally relevant apps to ensure maximum effectiveness. Practitioners must also consider the possible isolation and avoidance behaviours that may arise. For example, participant 4 pointed out that “with an app it is just being further isolated in the comfort of your bed or couch or whatever, and you’re not really meeting or seeing anybody, so that I think would be the issue for me.” Practitioners should also monitor and guide patient activity on these apps, and ensure that there are sufficient benefits for the patient.

Conclusion

This research was a small-scale exploratory study, serving as a starting point, to understand the views of professionals in the mental healthcare field about the role technology plays in therapy. The study can provide insight to practitioners and clients, as well as app designers, regarding the efficacy of app-based interventions. It also indicates factors regarded as important by practitioners when considering mHealth apps, such as culture and privacy. The study highlights the potential of mental health apps, when used in tandem with traditional therapy, and under the guidance of therapists or counsellors. It revealed that while practitioners may be slightly apprehensive, the majority do believe that they are in fact, effective to a certain degree, and support their validity. Research in this field is imperative, especially in the Indian context, due to the vast population and increasing frequency of smartphone users.  

Limitations

One of the major drawbacks of this study is the restricted sample size, since the results obtained cannot be generalised to a large population. The study also does not differentiate between the various types of mental health apps available, clubbing them under the general umbrella term. Few studies have been previously conducted which were based on perspectives and opinions of mental health practitioners, which restricted the study from being detailed, owing to the limited availability of resources to draw from. Lastly, all interviews were conducted over the phone, which may have impacted the responses of participants, since there may have been disturbances in their environment.

Future Research

Future studies should attempt to segregate mental health apps by category based on function, in order to thoroughly understand the perspectives of counsellors, and the kind of apps they would be more likely to integrate into their practices. Larger sample populations must also be employed in order to allow for generalization. Studies should also consider focusing on specific cultures and differences in accessibility, as well as exploring the viability of apps in varying dialects to better support cultural backgrounds.  In the Indian context, studies should also attempt to establish the number of individuals engaging with mHealth apps, and gain a deeper understanding of the effect of prejudice against traditional therapy.

Acknowledgements

This paper was written under the mentorship of Tonmoy Halder and Sandip Ravindra, and revised with the assistance of Nikita Kumar and Srushti Borkar.

References

American Psychiatric Association (2017). Mental health apps. American Psychiatric Association. Retrieved from https://www.psychiatry.org/psychiatrists/practice/mental-health-apps. Accessed 27 Feb 2019.

Radovic A., Vona, P.L., Santostefano, A.M., Ciaravino, S., Miller, E., & Stein, B.D. (2016) Smartphone applications for mental health. Cyberpsychology, Behavior, and Social Networking.  19(7):465-70.

Anthes, E. (2016). Mental health: There’s an app for that.Nature, 532 (7597), 20-23. 

Barak, A., & Grohol, J.M. (2011) Current and Future Trends in Internet-Supported Mental Health Interventions. Journal of Technology in Human Services, 29:3, 155-196. 

Becker, D. (2016). Acceptance of Mobile Mental Health Treatment Applications. Procedia Computer Science,98, 220-227. doi:10.1016/j.procs.2016.09.036

Carli V, Hoven CW, Wasserman C, Chiesa F, Guffanti G, Sarchiapone M,…Wasserman D (2014). A newly identified group of adolescents at ‘invisible’ risk for psychopathology and suicidal behavior: findings from the SEYLE study. World Psychiatry, 13, 78–86.

Chan S, Godwin H, Gonzalez A, Yellowlees P.M., & Hilty D.M. (2017). Review of use and integration of mobile apps into psychiatric treatments.” Current Psychiatry Reports, 19:  (12):96.

Griffiths, K. M., & Christensen, H. (2006). Review of randomised controlled trials of internet interventions for mental disorders and related conditions. Clinical Psychologist,10(1), 16-29. doi:10.1080/13284200500378696

Grist, R., Porter, J., & Stallard, P. (2017). Mental health mobile apps for preadolescents and adolescents: A systematic review. Journal of Medical Internet Research, 19(5), e176. 

Howells, A., Ivtzan, I., & Eiroa-Orosa, F. J. (2014). Putting the ‘app’ in happiness: A randomised controlled trial of a smartphone-based mindfulness intervention to enhance wellbeing. Journal of Happiness Studies,17(1), 163-185. doi:10.1007/s10902-014-9589-1

Kenny, R., Dooley, B., & Fitzgerald A. (2014). Developing mental health mobile apps: Exploring adolescents’ perspectives. Health Informatics Journal, 22(2), 265-275.

Kroenke K., Spitzer R.L., & Williams J.B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–13.

Lui, J. H., Marcus, D. K., & Barry, C. T. (2017). Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Professional Psychology: Research and Practice,48(3), 199-210. doi:10.1037/pro0000122

Luxton D. D., McCann R. A., Bush N. E., Mishkind M. C. & Reger G. M (2011). “MHealth for Mental Health: Integrating Smartphone Technology in Behavioral Healthcare.” Professional Psychology: Research and Practice,42(6),  505–512., doi:10.1037/a0024485.

Macias, C., Panch, T., Hicks, Y. M., Scolnick, J. S., Weene, D. L., Öngür, D., & Cohen, B. M. (2015). Using smartphone apps to promote psychiatric and physical well-being. Psychiatric Quarterly,86(4), 505-519. doi:10.1007/s11126-015-9337-7

Mak, W. W., Tong, A. C., Yip, S. Y., Lui, W. W., Chio, F. H., Chan, A. T., & Wong, C. C. (2018). Efficacy and moderation of mobile app–based programs for mindfulness-based training, self- compassion training, and cognitive behavioral psychoeducation on mental health: Randomized controlled noninferiority trial. JMIR Mental Health,5(4). doi:10.2196/mental.8597

Matthews M, & Doherty G. (2011, May 7-12) In the mood: engaging teenagers in psychotherapy using mobile phones. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 2947-2956.

Merikangas, K. R., He, J. P., Burstein, M., Swanson, S. A., Avenevoli, S., Cui, L., … Swendsen, J. (2010). Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication–Adolescent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry, 49(10), 980–989. doi:10.1016/j.jaac.2010.05.017

Morriss, R. (2015). Do apps have anything to offer mental health?: Commentary on Smartphone Apps in Mental Healthcare. BJPsych Advances, 21(5), 359-360. 

National Institute of Mental Health (2017). Technology and the Future of Mental Health Treatment. Retrieved from https://www.nimh.nih.gov/health/topics/technology-and-the-future-of-mental-health-treatment/index.shtml

Parker, L., Bero, L., Gillies, D., Raven, M., Mintzes, B., Jureidini, J., & Grundy, Q. (2018). Mental health messages in prominent mental health apps. Annals of Family Medicine, 16(4), 338-342. 

Powell J., Inglis N., Ronnie J., & Large S. (2011). The characteristics and motivations of online health information seekers: cross-sectional survey and qualitative interview study. Journal of Medical Internet Research, 13, e20.

Rathbone, A. L., & Prescott, J. (2018). The use of mobile apps and SMS messaging as physical and mental health interventions: Systematic review. Focus,16(4), 456-465. doi:10.1176/appi.focus.16406
Rathbone, A. L., Clarry, L., & Prescott, J. (2017). Assessing the efficacy of mobile health apps sing the basic principles of cognitive behavioral therapy: Systematic review. Journal of Medical Internet Research, 19(11), e 339. doi:10.2196/preprints.8598

Sprenger, M., Mettler, T., & Osma, J. (2017). Health professionals perspective on the promotion of e-mental health apps in the context of maternal depression. Plos One,12(7). doi:10.1371/journal.pone.0180867

Sucala, M., Cuijpers, P., Muench, F., Cardoș, R., Soflau, R., Dobrean, A., . . . David, D. (2017). Anxiety: There is an app for that. A systematic review of anxiety apps. Depression and Anxiety,34(6), 518-525. doi:10.1002/da.22654

Takahashi, K., Takada, K., & Hirao, K. (2018). Feasibility and preliminary efficacy of a smartphone application intervention for subthreshold depression. Early Intervention in Psychiatry,13(1), 133- 136. doi:10.1111/eip.12540

WHO (2015. September 15). Comprehensive mental health action plan 2013–2020. . Retrieved from https://www.who.int/mental_health/action_plan_2013/en/

Zelmer J., van Hoof K., Notarianni M., van Mierlo T., Schellenberg M., T& annenbaum C. (2018). An assessment framework for e-mental health apps in Canada: Results of a modified Delphi Process. JMIR Mhealth Uhealth;6(7), 10016.