Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
Nature Mental Health (2026)
Many adolescents seek mental health advice from general-purpose artificial intelligence (AI) chatbots, despite limited evaluations against youth-relevant clinical standards. Here we developed the Adolescent Mental Health AI Response Tool (AMHART), a clinician-informed instrument for auditing the presence of guideline-informed response content across seven domains, and applied it to four major AI chatbots. Candidate items were reviewed by clinicians for their evidence base, objectivity and clinical importance, yielding a subset with strong content validity and inter-rater reliability. We then configured a three-model large language model evaluator ensemble, assessed its agreement with human reference labels and scored multiturn conversations in which adolescent personas (varying by sex, diagnosis and symptom severity) engaged with ChatGPT-5, ChatGPT-4, Claude 4.5 and Gemini 2.5. AMHART scores indicated greater coverage of applicable criteria for anxiety than depression and for moderate than severe presentations. The findings suggest AMHART offers a reliable, clinician-informed benchmark for evaluating chatbot responses to adolescent anxiety and depression disclosures.
This is a preview of subscription content, access via your institution
Subscribe to this journal
Receive 12 digital issues and online access to articles
Buy this article
39,95 €
Prices may be subject to local taxes which are calculated during checkout
All data are available via GitHub at https://github.com/RANDCorporation/AMHART. The study appendices contain additional data collection and analytic materials—including clinical vignettes and AMHART training protocol.
All code is available via GitHub at https://github.com/RANDCorporation/AMHART. This includes the LLM evaluator script and analytic script for processing outputs from the LLM evaluator when applied to each AI chatbot, as well as a README file.
McBain, R. K. et al. Growth and distribution of child psychiatrists in the United States: 2007–2016. Pediatrics 144, e20191576 (2019).
Article PubMed PubMed Central Google Scholar
CDC. U.S. Teen Girls Experiencing Increased Sadness and Violence https://www.cdc.gov/media/releases/2023/p0213-yrbs.html (Centers for Disease Control and Prevention, 2025).
Kieling, C. et al. Worldwide prevalence and disability from mental disorders across childhood and adolescence: evidence from the global burden of disease study. JAMA Psychiatry 81, 347–356 (2024).
Article PubMed PubMed Central Google Scholar
Mental Illness https://www.nimh.nih.gov/health/statistics/mental-illness (National Institute of Mental Health, 2024).
McBain, R. K. et al. AI chatbot use and disclosure for mental health among US adolescents and young adults. JAMA Pediatr. 180, 884–890 https://doi.org/10.1001/jamapediatrics.2026.2015 (2026).
Article PubMed PubMed Central Google Scholar
Talk, trust, and trade-offs: how and why teens use AI companions. Common Sense Media https://www.commonsensemedia.org/research/talk-trust-and-trade-offs-how-and-why-teens-use-ai-companions (2025).
Ettman, C. K. & Galea, S. The potential influence of ai on population mental health. JMIR Ment. Health 10, e49936 (2023).
Article PubMed PubMed Central Google Scholar
Siddals, S., Torous, J. & Coxon, A. ‘It happened to be the perfect thing’: experiences of generative AI chatbots for mental health. npj Ment. Health Res. 3, 48 (2024).
Article PubMed PubMed Central Google Scholar
Malik, T., Ambrose, A. J. & Sinha, C. Evaluating user feedback for an artificial intelligence-enabled, cognitive behavioral therapy-based mental health app (Wysa): qualitative thematic analysis. JMIR Hum. Factors 9, e35668 (2022).
Article PubMed PubMed Central Google Scholar
Olawade, D. B. et al. Enhancing mental health with artificial intelligence: current trends and future prospects. J. Med. Surg. Public Health 3, 100099 (2024).
Article Google Scholar
Zuckerbrot, R. A. et al. Guidelines for adolescent depression in primary care (GLAD-PC): part I. Practice preparation, identification, assessment, and initial management. Pediatrics 141, e20174081 (2018).
Article PubMed Google Scholar
Walter, H. J. et al. Clinical practice guideline for the assessment and treatment of children and adolescents with anxiety disorders. J. Am. Acad. Child Adolesc. Psychiatry 59, 1107–1124 (2020).
Article PubMed Google Scholar
Walter, H. J. et al. Clinical practice guideline for the assessment and treatment of children and adolescents with major and persistent depressive disorders. J. Am. Acad. Child Adolesc. Psychiatry 62, 479–502 (2023).
Article PubMed Google Scholar
OSG. Social Media and Youth Mental Health: The U.S. Surgeon General’s Advisory (Office of the Surgeon General, US Department of Health and Human Services, 2023).
Schlicht, I. B., Zhao, Z., Sayin, B., Flek, L. & Rosso, P. Do LLMs provide consistent answers to health-related questions across languages? Preprint at https://doi.org/10.48550/arXiv.2501.14719 (2025).
Obradovich, N. et al. Opportunities and risks of large language models in psychiatry. NPP Digit. Psychiatry Neurosci. 2, 8 (2024).
Article PubMed PubMed Central Google Scholar
Lawrence, H. R. et al. The opportunities and risks of large language models in mental health. JMIR Ment. Health 11, e59479 (2024).
Article PubMed PubMed Central Google Scholar
Health advisory: artificial intelligence and adolescent well-being. American Psychological Association https://www.apa.org/topics/artificial-intelligence-machine-learning/health-advisory-ai-adolescent-well-being (2025).
Arora, R. K. et al. HealthBench: evaluating large language models towards improved human health. Preprint at https://doi.org/10.48550/arXiv.2505.08775 (2025).
Wagner, M. et al. Using large language models as a scalable mental status evaluation technique. NPP Digit. Psychiatry Neurosci. 3, 27 (2025).
Article PubMed PubMed Central Google Scholar
Archiwaranguprok, C., Albrecht, C., Maes, P., Karahalios, K. & Pataranutaporn, P. Simulating psychological risks in human–AI interactions: real-case informed modeling of AI-induced addiction, anorexia, depression, homicide, psychosis, and suicide. Preprint at https://doi.org/10.48550/arXiv.2511.08880 (2025).
Dwyer, B. et al. Mindbench.ai: an actionable platform to evaluate the profile and performance of large language models in a mental healthcare context. NPP Digit. Psychiatry Neurosci. 3, 28 (2025).
Article PubMed PubMed Central Google Scholar
Bighelli, I. et al. Implementation of treatment guidelines for specialist mental health care. Cochrane Database Syst. Rev. 2016, CD009780 (2016).
Google Scholar
Aboraya, A. The reliability of psychiatric diagnoses: point-our psychiatric diagnoses are still unreliable. Psychiatry 4, 22–25 (2007).
PubMed PubMed Central Google Scholar
Rocha Neto, H. et al. Inter-rater reliability between structured and non-structured interviews is fair in schizophrenia and bipolar disorders—a systematic review and meta-analysis. Diagnostics 13, 526 (2023).
Article PubMed PubMed Central Google Scholar
Brown, J. E. H. & Halpern, J. AI chatbots cannot replace human interactions in the pursuit of more inclusive mental healthcare. SSM Ment. Health 1, 100017 (2021).
Article Google Scholar
Khawaja, Z. & Bélisle-Pipon, J.-C. Your robot therapist is not your therapist: understanding the role of AI-powered mental health chatbots. Front. Digit. Health 5, 1278186 (2023).
Article PubMed PubMed Central Google Scholar
Vanhoffelen, G., Vandenbosch, L. & Schreurs, L. Teens, tech, and talk: adolescents’ use of and emotional reactions to Snapchat’s My AI Chatbot. Behav. Sci. 15, 1037 (2025).
Article PubMed PubMed Central Google Scholar
McBain, R. K. et al. Evaluation of alignment between large language models and expert clinicians in suicide risk assessment. Psychiatr. Serv. 6, 944–950 https://doi.org/10.1176/appi.ps.20250086 (2025).
Pichowicz, W., Kotas, M. & Piotrowski, P. Performance of mental health chatbot agents in detecting and managing suicidal ideation. Sci. Rep. 15, 31652 (2025).
Article PubMed PubMed Central Google Scholar
Campbell, L. O., Babb, K., Lambie, G. W. & Hayes, B. G. An examination of generative ai response to suicide inquires: content analysis. JMIR Ment. Health 12, e73623 (2025).
Article PubMed PubMed Central Google Scholar
Terlizzi, E. P. & Zablotsky, B. Symptoms of anxiety and depression among adults: United States, 2019 and 2022. Natl Health Stat. Rep. 7, CS353885 (2024).
Google Scholar
APA. Depression Treatments for Children and Adolescents (American Psychological Association, 2023).
NICE. Depression in Children and Young People: Identification and Management (National Institute for Health and Care Excellence, 2019).
WHO. Guidelines on Mental Health Promotive and Preventive Interventions for Adolescents: Helping Adolescents Thrive (World Health Organization, 2020).
Shirali, G., Shekari, M. & Angali, K. A. Assessing reliability and validity of an instrument for measuring resilience safety culture in sociotechnical systems. Saf. Health Work 9, 296–307 (2018).
Article PubMed Google Scholar
Download references
We thank A. Joshi for conceptual feedback on the development of this manuscript.
This work was supported by a grant from the National Institute of Mental Health (1R01MH132551).
RAND, Arlington, VA, USA
Ryan K. McBain
Brigham and Women’s Hospital, Boston, MA, USA
Ryan K. McBain
Harvard Medical School, Boston, MA, USA
Ryan K. McBain, Fang Zhang & Hao Yu
RAND, Santa Monica, CA, USA
Jonathan H. Cantor, Brooke Hollister, Nicholas Kong, Adrian Salas & Li Ang Zhang
RAND, Pittsburgh, PA, USA
Joshua Breslau, Shannon Donofry & Bradley D. Stein
Harvard Pilgrim Health Care Institute, Boston, MA, USA
Fang Zhang, Alyssa Burnett & Hao Yu
Boston Children’s Hospital, Boston, MA, USA
Benjamin Rader
RAND, Boston, MA, USA
Jessica L. Sousa
MIT Media Lab, Cambridge, MA, USA
Pat Pataranutaporn
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
R.K.M., B.H. and N.K. had full access to the data in the study and take responsibility for data integrity and data analysis. Concept and design by R.K.M. and J.H.C. Drafting of the manuscript by R.K.M. and J.H.C. Critical review of the manuscript by R.K.M., B.H., N.K., A.S., J.H.C., L.A.Z., F.Z., S.D., J.L.S., B.R., A.B., J.B., B.D.S., P.P. and H.Y. Statistical analysis by R.K.M. Funding obtained by H.Y., J.H.C. and R.K.M. Administrative, technical and material support by P.P. Supervision by J.B., B.D.S., A.S., P.P. and H.Y.
Correspondence to Ryan K. McBain.
The authors declare no completing interests. NIMH was not involved in the design or conduct of this study, the analysis or interpretation of the data or the decision to publish this Article.
Nature Mental Health thanks Amber W. Childs, Desmond Ong, Anna Viduani and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Appendices e1–e6, including eight clinical persona cards, five supplementary tables, AMHART scoring instructions, clinical-domain rationale and model configuration details.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Reprints and permissions
McBain, R.K., Cantor, J.H., Hollister, B. et al. Development, validation and AI chatbot evaluation of the Adolescent Mental Health AI Response Tool. Nat. Mental Health (2026). https://doi.org/10.1038/s44220-026-00753-y
Download citation
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s44220-026-00753-y
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Advertisement
Nature Mental Health (Nat. Mental Health)
ISSN 2731-6076 (online)
© 2026 Springer Nature Limited
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.