Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
2 min. read
A new Penn-led randomized controlled trial has found that AI-powered chatbots can make vaccine-hesitant parents more likely to say they will immunize their children against human papillomavirus (HPV), but no more than standard written public health materials.
The findings raise questions about when, how and to what extent AI enhances public health communications. “Comparing a chatbot to nothing isn’t really a fair test. The interesting question is whether it does better than what public health agencies already have out there. In our study, it didn’t,” says Sharath Chandra Guntuku, research associate professor in computer and information science (CIS) and the study’s senior author. Guntuku directs the Computational Social Listening Lab and holds a secondary appointment in the Annenberg School of Communication (ASC). Co-authors include Alison M. Buttenheim, professor of nursing and health policy in family and community health at Penn Nursing; Neil Sehgal, a doctoral student in CIS; CIS professor Lyle Ungar and postdoctoral fellow Sunny Rai; Anish Agarwal of Penn Medicine; Joseph Cappella of Penn LDI and ASC; and Melanie Kornides of Penn LDI and Penn Nursing.
Described in a new paper in JAMA Network Open, the trial—which includes nearly 1,300 participants in the United States, the United Kingdom, and Canada—finds that skeptical parents who interacted with the chatbots were more likely than those who received no intervention to say they intended to immunize their children.
But spending a few minutes reading standard written materials provided online about the benefits of the HPV vaccine from governmental health agencies like the Centers for Disease Control and Prevention (CDC) produced essentially the same effect.
“It would have been easy to compare an AI chatbot with no intervention or a very weak control condition and find a positive result,” says Sehgal, the study’s first author. “But we wanted to know whether the chatbot added value beyond what public health agencies already provide.”
Two years ago, one study found that an AI-powered chatbot could reduce beliefs in conspiracy theories. But the designs of those studies and others like them often make it difficult to assess the effectiveness of the chatbots.
In contrast, the Penn team set strict guidelines to make the chatbot as comparable as possible with existing public health materials, and to assess its effects over time: participants assigned to the chatbot or written materials groups were exposed to each for the same minimum time of 3 minutes, and all participants were assessed for their intention to vaccinate their children 15 and 45 days after the interventions.
Under these stricter conditions, the innovative features of the chatbots—including their ability to converse with parents in real time—essentially provided no additional benefit, even though parents spent more time engaging with the chatbots than the written materials.
“While chatbot conversations can move intentions immediately,” says Ungar, “their advantage disappeared when compared with well-designed public health materials and did not persist over time.”
“AI chatbots are promising, but they should not be assumed to outperform existing tools simply because they are newer or more interactive,” adds Guntuku. “A short read of a CDC webpage held up at least as well as a chatbot conversation, and the effect actually lasted longer.”
Ultimately, the researchers hope their work encourages a more evidence-based approach to using AI for public health. “We need to evaluate AI tools against realistic alternatives,” says Guntuku. “It’s time to shift the conversation from, ‘Can AI persuade people?’ to more granular questions, like ‘When does AI add meaningful value, for whom and under what conditions?’”
Read more at Penn Engineering.
Ian Scheffler
Image: Jessica Kourkounis / Stringer via Getty Images
Apr 30, 2026
(Image: Lance Nelson)
May 13, 2026
Image: shih-wei via Getty Images
May 5, 2026
A bioengineered bean gum from the lab of Penn Dental’s Henry Daniell is found to reduce the levels of three microbes associated with head and neck squamous cell cancer to almost zero, without affecting the beneficial bacteria normally found in the mouth.
(Image: Kevin Monko/Penn Dental Medicine)
Apr 20, 2026