Welcome to the forefront of conversational AI as we explore the fascinating world of AI chatbots in our dedicated blog series. Discover the latest advancements, applications, and strategies that propel the evolution of chatbot technology. From enhancing customer interactions to streamlining business processes, these articles delve into the innovative ways artificial intelligence is shaping the landscape of automated conversational agents. Whether you’re a business owner, developer, or simply intrigued by the future of interactive technology, join us on this journey to unravel the transformative power and endless possibilities of AI chatbots.
You have full access to this open access article
834
Explore all metrics
Rapidly growing technology has enabled real-time digital services in healthcare, and new opportunities for sexual and reproductive health care. As assessing patient needs and communication effectively on digital platforms can be challenging, healthcare students must practise written communication, such as chat dialogues. Despite its increasing use in education, the way in which generative AI can enhance chat interactions between healthcare providers and patients remains poorly understood. The aim of this cross-sectional study was to explore contraception topics in chat dialogues between healthcare students and AI patients during AI simulations.
The AI application simulated a written chat dialogue between student and AI patient, using the CurreChat interface, to enable students to practise clinical skills and communication in digital health service chat dialogues. Purposive sampling was used to collect the data from fifth-year medical students (n = 24) and graduating midwifery students (n = 20) in higher education institutions of medicine and midwifery. Data were collected in August and October 2024.
The data consisted of chat dialogues between healthcare students and generative AI patients. Natural language processing (NLP) and automated text analysis examined the contraception topics in the dialogues. The analysis software was based on pre-taught (self-supervised learning), industry-specific language models that detect meanings and their semantics in given texts.
The most significant result was that the students discussed essential aspects of contraception in the dialogues with the AI patients. Several topics in the students’ part of the dialogues were similar to those in MeSH terminology and to work-related topics. The students’ dialogues covered essential topics such as contraindications (114 times), contraceptive methods (93 times), and smoking (80 times), aligning with the Current Care Guideline.
Generative AI chat simulations can enhance the education of healthcare professionals globally in contraception issues by improving educational outcomes. To fully utilise the advantages of AI chat interactions, effective prompting is essential. NLP was an appropriate method for analysing the conversations and could be utilised more in future research across diverse healthcare settings.
• AI chat simulation developed for healthcare education on contraception.
• Novel use of NLP to analyse chat dialogues in healthcare education.
• Gap analysis identifies alignment with Current Care Guideline.
• Students engage deeply with AI, enhancing understanding of clinical guidelines.
• AI-driven virtual simulations boost healthcare communication skills.
Avoid common mistakes on your manuscript.
The rapid expansion of technology has profoundly transformed healthcare delivery into real-time digital services. Digital tools and platforms are instrumental in improving the availability of information and services, particularly in remote areas and among vulnerable populations [1]. Telemedicine encompasses an extensive array of services including remote diagnostics, patient monitoring, virtual consultations and comprehensive digital health platforms [2]. Digital health services, such as patient portals, chat services, artificial intelligence (AI) -powered chatbots, and remote consultations, offer multiple advantages, such as better patient outcomes and broader access to healthcare across diverse environments and populations [2, 3]. Such technologies are available in 81% of the member states of the World Health Organization´s (WHO) European region countries [4]. The incorporation of digital technologies into healthcare systems not only facilitates access but also promotes early intervention and preventive care, aiming to reduce disparities in health outcomes [2].
Telehealth services are particularly relevant in the field of sexual and reproductive health. The WHO emphasises the importance of equitably providing comprehensive contraceptive information and services to all individuals, free from discrimination and advocates for integrating mobile telehealth services to improve access to contraceptive information [5]. The Countdown 2030 Europe Consortium similarly highlights that digitalisation as means to advance sexual and reproductive health and rights (SRHR), including contraception and family planning. Digital sexual health interventions primarly occur on websites and mobile applications, and no physical presence is required for interactions between healthcare professionals and patients. Additionally, data collected through digitalisation improves the planning and execution of these services, improving the understanding of population needs and significantly improving the accessibility and quality of sexual and reproductive health services [1].
Digital platforms are increasingly utilised to enhance diagnosis, consultation and treatment processes for patients [6]. Chat services facilitate straightforward written communication between patients and healthcare professionals [7, 8]. On these platforms, symptoms can be discussed, questions can be asked, and active participation in decision-making processes can be encouraged, thereby fostering a collaborative approach to healthcare [2]. Positive patient experiences of chat services have been reported, indicating that allowing ample time for reflection before responding reduces patient stress [7, 8].
However, the lack of specialised expertise and challenges in discerning patient needs through digital means create obstacles for healthcare professionals conducting remote consultations [9, 10]. The absence of nonverbal cues complicates the accurate assessment of patients’ needs [10]. For digital transformation to enhance evidence-based treatment decision-making [11], healthcare professionals must be able to remotely assess clinical status, evaluate treatment needs, and collaboratively establish objectives with patients [12].
To address these challenges, it has been proposed that healthcare professionals improve their communication skills [9] and their clinical assessment abilities through chat dialogues that rely on written interaction [10]. Developing written communication competencies for chat services through targeted training is essential [9]. Furthermore, regular, competency-based training should begin during already the undergraduate education of healthcare students, to establish foundational skills early in their professional development [13].
AI-driven virtual simulations offer a promising method for practising communication skills. Virtual simulations and AI are a notable innovations that have shown significant potential to improve cognitive abilities, clinical decision-making and evidence-based practice [14,15,16,17]. AI chatbots, such as ChatGPT, have shown considerable promise in the field of education, highlighting the versatility and applicability of AI technologies across various domains [18, 19]. Large language models can be beneficial in healthcare training, as the use of AI can improve education through tailored learning experiences and simulation exercises [20].
The usefulness of AI in education comes from its capacity to provide feedback, offer guided learning pathways and reduce costs [21]. AI enables realistic interactions, making learning more meaningful, personalised and efficient [22,23,24,25]. It can also create a less stressful communication environment for students than traditional classroom settings [26]. For instance, by assigning generative AI the role of a patient in course-specific simulations, healthcare students can practice engaging in dialogues with clients [27]. Students have a positive attitude towards AI technologies and are eager to incorporate tools like ChatGPT into their learning processes. They especially appreciate the usefulness of these technologies in offering unique insights and find them user-friendly due to their 24/7 availability [28].
Furthermore, anthropomorphising AI – by incorporating human-like characteristics such as personalised pronouns – can deepen the processing of learning materials and boost intrinsic motivation to learn [29]. University teachers also see AI as providing an opportunity to not only facilitate the teaching of critical skills, such as interpersonal communication, but also to act as a bridge, connecting theoretical knowledge with real-world applications through simulations of real-life scenarios [30].
One promising opportunity to improve student performance, particularly in patient interviewing, is the creation of generative AI patients through optimised prompt engineering [27]. Effective prompt engineering plays a vital role in this process – revising and retesting prompts is essential for achieving desired outcomes. This requires a thorough understanding of the user’s goals, the strengths and weaknesses of AI, and the nuances of language [31].
Despite the growing integration of digital healthcare services and the increasing use of generative AI in training programmes, a significant gap remains in our empirical understanding of how these technologies can improve the chat dialogue between healthcare providers and patients. Specifically, research is lacking on the topics discussed in chat-based interactions between healthcare students and generative AI patients. Considering the role of Natural Language Processing (NLP) in digital healthcare, particularly in enhancing interactions and the analyses of patient information [2], studies specifically employing NLP methods to analyse the dialogue within chat discussions are still lacking. This lack of insight hinders the development of effective training methods that utilise generative AI and could ultimately improve the communication skills essential for successful remote consultations. Therefore, the aim of this study was to explore the contraception-related topics in chat dialogues between healthcare students and generative AI patients during AI patient simulations. The specific research question guiding this investigation is: How do these dialogues reflect the alignment between Medical Subject Headings (MeSH) terminology, work-related topics, and the Finnish Current Care Guideline for Contraception [32].
This study employed a cross-sectional descriptive design to gather data from a single group at one specific time point [33]. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement [34] was used to increase the transparency of the study procedures.
The AI patient simulation used in this study has been developed for educational purposes, through multidisciplinary collaboration between the Faculty of Medicine and the Global Campus project at the University of Helsinki in 2023 and 2024 [35]. The AI patient simulation employs the CurreChat interface, developed by the University of Helsinki and Toska, the application development academy at the University of Helsinki. CurreChat is based on generative AI and utilises the GPT-4 language model developed by OpenAI.
The AI application simulates a chat dialogue on a computer. In the simulation, healthcare students assume the role of a healthcare professional, and communication occurs through written messages. The AI application features various characters and scenarios created by clinical experts. Each simulated chat dialogue adapts to the learner’s actions, making every conversation unique. This approach creates an authentic interaction scenario, allowing the learner to practice remote consultations and prepare more thoroughly for real-life situations. The AI patient scenarios are designed to reflect situations commonly encountered in digital healthcare services. In this study, the clients sought advice on contraception and the students were advised to base their recommendations and treatment choices on the Finnish Current Care Guideline on Contraception [32]. The Finnish Current Care Guidelines are treatment guidelines that provide current recommendations for managing various diseases and health issues. They are based on scientific research and expert opinions, and offer up-to-date, evidence-based advice on medical treatments, therapeutic interventions and lifestyle changes.
A patient scenario prompt for an AI patient was created through a multidisciplinary effort of medical and midwifery specialists, educators, digital learning designers, and AI experts. The development process included:
Defining Learning Objectives: The first step involved determining what the students should learn or practise through this patient case [14, 36]. This included specific medical topics such as assessment of patient needs and wishes, evaluation of suitable contraceptive methods and potential contraindications, and patient counselling based on the Current Care Guideline on Contraception. (Table 1)
Designing Patient Profiles for Realistic AI Patients: Detailed profiles were developed for the AI patients: their age, gender, medical history, current symptoms, lifestyle habits (e.g., smoking) and any other relevant information representing typical clients of contraception clinics.
Writing the Patient Scenario: All the information was combined into a coherent scenario. This served as the basis for the AI patients, with distinct medical history and contraceptive needs, responses and interactions with the students.
Prompt Engineering & AI Fine-tuning: Initially, a base prompt for patient #1 was established, after which layers of complexity, contextual richness and linguistic nuances consistent with the patient’s distinct ‘persona’ were progressively incorporated and refined through iterative testing. This ensured that the simulated patient responded accurately and conversationally, and aligned closely with the pedagogical objectives. Continuous adjustments and fine-tuning improved the coherence, medical accuracy and educational effectiveness of the interactions.
In total, four patient scenarios were developed. Patients #1 and #2 shared the same base prompt with minor variations in medical context, and patients #3 and #4 shared another distinct base prompt with some medical differences. However, for the purposes of this study, a single patient scenario was utilised. This approach enabled a focused examination of the interactions and discussions surrounding the topic of contraception within the specified context.
Testing: Before the patient case was used, it was tested with students and health care professionals. Feedback was gathered on the basis of which necessary adjustments were made to ensure that the scenario was realistic and that it met the learning objectives. In addition, healthcare students participated in sessions to test the interactions, refine the prompts, and provide feedback on any learning experiences.
Purposive sampling [37] was used to recruit medical and midwifery students, whose curriculum includes education on reproductive health and contraception. The data were collected between August and October 2024 from fifth-year medical students (n = 24) and graduating midwifery students (n = 20). Students participated in an AI patient simulation on a computer at their educational institution. AI patient simulation was integrated into teaching but participation in the study was voluntary. Prior to the simulation medical students had a lecture on hormonal contraception and for midwifery students, the topic had been covered in previous studies. In addition, students were instructed to read the Finnish Current Care Guidelines for Contraception [32] as preparatory material. During the simulation, students were given a short background story (Table 2) to introduce the situation of the chat dialogue. The AI patient simulation took about 20 min.
The data consisted of chat dialogues between AI patients and students on the CurreChat interface. The participants saved their dialogue by clicking the ‘Save as Email’ button, which sent the dialogue to the student’s own email. The participants were instructed to forward the dialogue to a member of the research team, who then pseudonymised the data. Any direct or indirect personal identifiers that appeared in the chat dialogues (e.g., introducing oneself to the AI with one’s real name) were removed at this stage. The pseudonymised data were stored on a secure, backed-up network drive, after which the emails were deleted.
NLP and automated text analysis developed by HeadAI Inc. was utilised for the analysis. The analysis software, HeadAI Graphmind, is based on pre-taught (self-supervised learning), industry-specific language models that detect meanings and their semantics from given texts. This enables an effective, balanced performance in analyses, and the results (industry-relevant words) are computationally meaningful compound words or phrases. In this study, the selected industry-specific language models were 1) work-related language model (covering e.g., topics focusing on ESCO, ONET, ISCO, upskilling, training, human resources or higher education) and 2) MeSH terminology-focused language models. These industry-specific language models were selected because work-related models represent language in education and work in general, and MeSH represents language used in the health care domain. Headai’s secure computing architecture is explained in Fig. 1. The main idea is, that all the data is fully controlled by the user (host system). All computing is accessed via APIs, while the computing itself is isolated from the internet. This means, Headai’s processes are not using external resources, nor storing the data during the analysis. The result sets are accessible with API (behind user management) and the results can be shown in the system user/host system prefers.
High-level HeadAI Cloud architecture. This diagram illustrates the overall structure of the HeadAI Cloud system, highlighting the data input process through a REST API, the isolated analysis workflow, and the output generated in JSON graph format
A total of 44 dialogues between students and AI patients were converted into two knowledge graphs: 1) Students’ part of the dialogue in MeSH language models, 2) Students’ part of the dialogue in work-related topics. Knowledge graph is a classical data structure representing information as a network of interconnected entities, their properties, and relationships. The Finnish Current Care Guideline on Contraception was also turned into two knowledge graphs: 3) Current Care Guidelines in work-related topics and 4) Current Care Guidelines in MeSH language models.
The methodology for building individual knowledge graphs and subsequently merging them to identify and analyse gaps between different datasets is illustrated in Fig. 2. Altogether five knowledge graphs were constructed. The further analysis, similarities and gaps, are performed to these knowledge graphs.
The process of developing and integrating knowledge graphs through application of multiple language models with the specific purpose of identifying and analysing gaps in information (gap analysis)
After this, the knowledge graphs were cleaned by 1) removing single observations (i.e. all the topics that appeared at least twice), 2) removing topics that were not relevant from the education perspective and 3) removing clear/obvious misinterpretations. To address potential sources of bias, the data cleaning process was focused on, which involved the removal of misleading information to avoid wrong conclusions and biased results. Subjective rejection was performed by a research team of professionals based on the rule that at least two persons had to agree on what was removed and that none of the members opposed the removal. Such words were, for example, ‘university’, ‘teacher’, ‘professor’, ‘data base’ and ‘communication’. HeadAI Cloud visualisation tools were used to visualise the knowledge graphs.
After the single knowledge graphs were constructed, the work-related and MeSH-based knowledge graphs were merged, and a gap analysis was conducted of the students’ knowledge graph and the Finnish Current Care Guideline on Contraception knowledge graph (Fig. 3). A gap analysis reveals topics that are found in both knowledge graphs (similarity), and the topics that are found on one but not the other knowledge graph, i.e. the gap between the documents. The result of a gap analysis of two data models shows similar topics as one group and the gaps as two separate groups. Basically, the result is a combination of a Venn diagram and a correlation matrix. HeadAI gap/similarity maps can be read like correlation matrices in statistics. When Data 1 is the goal and Data 2 is measured, the gap/similarity map shows: 1) what topics are covered (match), 2) what is not covered (Gap 1) and 3) what extra topics exist in Data 2 (Gap 2).
Gap and similarity analysis of two knowledge graphs. This illustration displays neighbouring topics as adjacent hexagonal boxes within the model. A thick line between the topics indicates a weak connection suggesting that the inclusion of the topic is contrived to maintain consistency in the illustration
The Responsible Conduct of Research (RCR) guidelines of the Finnish National Board on Research Integrity TENK [38] were followed at all stages of the study. Ethical approval was received from the University of Helsinki Research Ethics Committee in the Humanities and Social and Behavioural Sciences on June 4, 2024. Research permits were obtained from the participating organisations. All the participants who enrolled signed an informed consent form.
The ethical considerations regarding generative AI in this study pertain to data analysis, the outputs of language models, and the uploading and storage of data. For language models, the developers of the generative AI application provided by the University of Helsinki, which is compliant with GDPR guidelines, guided the AI patient using pre-designed prompts. The application does not collect any personal information on the user, and the service administrators cannot see the chat dialogues. The information input into the service is not used to further train the language model. All data remains within the EU.
All the analyses were based on anonymised data (students) and public data (recommendations). Furthermore, the HeadAI technology does not save or learn from the data it analyses and it does not recognise topics such as gender or ethnic background or a persons’ religion, sexual orientation, age or other GDPR-defined strictly personal topic. From the ethics perspective, this secures the study participants. From the AI Act ethical perspective, the analysis software is fully transparent, and the decision chain can be traced if needed. Furthermore, all the technology is ‘low-energy-computing’ technology, which consumes significantly less energy than, for example, models in high-performance computing environments.
The analysis began with single knowledge graphs to assess whether the MeSH terminology and work-related topics covered the key topics of the Current Care Guidelines for Contraception. The analysis revealed the MeSH topics, their frequences and their relations found in the Finnish Current Care Guidelines on Contraception, including contraindications (50), condoms (36), sterilisation (27), IUD (25) and contraception (25) (Fig. 4). The number in parentheses indicates the frequency of these specific topics in the Finnish Current Care Guidelines on Contraception—not necessarily as the exact word, but as a word with the same meaning.
MeSH topics, their frequencies and their relations to the Current Care Guideline on Contraception. Dark colours represent the most frequent topics, while lighter colours show less frequent topics. The most frequent topic is in the centre. Every neighbouring topic in the figure correspond to actual relationships in the real world, except those with a thick black line. These thick black lines denote non-optimal neighbours, included to maintain consistency in the illustration. The colour schemes and line widths are consistent across Figs. 4–8. (AUB = Abnormal Uterine Bleeding, BMI = Body Mass Index, CDC = Centers for Disease Control, IUD = Intrauterine Devices, PID = Pelvic Inflammatory Disease, SLE = Systemic lupus erythematosus, THL = The Finnish Institute for Health and Welfare, WHO = World Health Organization)
A similar analysis was then conducted of work-related topics and the Current Care Guideline, demonstrating overlaps in contraindications (50), COCs, (48), contraceptive methods (47), contraception (46), smoking (26) and overweight (24). (Fig. 5).
Work-related topics, their frequences, and their relations to the Current Care Guideline on Contraception. The significance of the colour schemes and line thicknesses are consistent with Fig. 4. (COCs = Combined Oral Contraceptives, IUD = Intrauterine Devices, OTC = Over-the-counter)
Subsequently, a comparative analysis was conducted on the dialogue between the students and the AI patient, focusing on their alignment with the MeSH terminology. Notable similarities were identified between several topics in the students’ part of the dialogues and the MeSH terminology, including menstruation (52), hormonal IUD (25), side effects (10), progesterone (9) progestin (8), IUD (8), birth control pills (8), contraceptive methods (8), visual disturbance (8) (Fig. 6).
Comparative analysis of dialogues between students and an AI patient, emphasizing their alignment with MeSH terminology. This figure highlights areas of strong alignment as well as discrepancies in the dialogues, illustrating how effectively the conversations correspond to established MeSH terminology. The colours schemes and line thicknesses are explained in Fig. 4. (IUD = Intrauterine Devices, s.c. = subcutaneous)
Next, the dialogue between the students and the AI patient were evaluated in relation to the work-related topics. The topics most frequently mentioned were smoking (54), migraine (39), medical practitioner (27), COCs (18) and contraceptive methods (16) (Fig. 7), indicating that the dialogue between the students and the AI patient frequently covered work-related topics.
Comparative analysis on the dialogue between the students and the AI patient focusing on their alignment with the work-related topics. The colour schemes and line thicknesses are consistent with the explanations provided in Fig. 4. (COCs = Combined Oral Contraceptives, IUD = Intrauterine Devices)
Finally, the students’ dialogues, originally analysed for MeSH terminology and work-related topics, were merged and analysed in conjunction with the Current Care Guideline for Contraception (Fig. 8). The gap analysis showed that the most frequently covered topics that aligned with the Current Care Guideline (depicted in green) were contraindications (114), contraceptive methods (93), contraception (89), smoking (80) and COCs (66).
Integration and analysis of students’ dialogues using MeSH terminology and work-related topics in conjunction with the Current Care Guideline for Contraception. The color-coded system illustrates the alignment of discussed topics with the guidelines. Green represents the topics that the students discussed that are aligned with the Current Care Guideline. Pink represents topics that are valid in terms of MeSH and work-related topics and were discussed but are not relevant in terms of the Current Care Guideline. Blue represents topics that are relevant in terms of the Current Care Guideline but were not mentioned in the chat dialogues. (AUB = Abnormal Uterine Bleeding, BMI = Body Mass Index, CDC = Centers for Disease Control, COCs = Combined Oral Contraceptives, PID = Pelvic Inflammatory Disease, SLE = Systemic lupus erythematosus, s.c. = subcutaneous, THL = The Finnish Institute for Health and Welfare, WHO = World Health Organization)
The numbers in the gap analysis show the sum of the numbers in the two knowledge graphs: the numbers in the gap area are the same as those in the single knowledge graphs, and in the matching areas, it is number in A + number in B. For example, smoking was found 26 times in the Current Care Guidelines and 54 times in the students’ dialogues. In the gap analysis, because the topic was in both the knowledge graphs, the value was 26 + 54 = 80.
On the other hand, the most frequently mentioned topics of the students, which were not relevant in the terms of the Current Care Guideline (depicted in pink), were medical practitioner (27), birth control pills (8), visual disturbance (8), contraceptive ring (6), contraceptive clinic (6), contraceptive pills (5) and medicine (5). This indicates that the students’ dialogues with the AI patient covered many important topics that are also included in the Current Care Guideline for Contraception.
In addition, the topics not discussed by the students in the dialogues, but that were relevant in terms of the Current Care Guidelines (depicted in blue), were sterilisation (27), drug-drug interactions (24), women (23), overweight (24) and diabetes (16).
The blue area contains the topics that are important from the Current Care Guidelines perspective.
The aim of this study was to explore contraception-related topics in chat dialogues between healthcare students and generative AI patients during AI patient simulations. This investigation was guided by the research question on the reflection of these dialogues in terms of alignment with MeSH terminology, work-related topics, and the Current Care Guideline for Contraception. By addressing this question, the study sought to clarify the extent to which the discussions among the students aligned with the established guidelines and relevant medical terminology, thereby enabling a deeper understanding of the educational potential of AI for healthcare training.
The most significant result of this study was that the students engaged in discussing the essential aspects of contraception in the dialogues with the AI patients. They discussed a range of pertinent topics, including indications for contraceptive use, contraindications and practical application. The AI-enhanced, interactive learning environment enabled the students to engage in realistic patient interactions, supporting the findings of Heston and Khun [27].
The results of this study also show that MeSH terminology and work-related topics broadly cover the key subjects of the Current Care Guideline for Contraception. This enables students to use established terminology when discussing contraception, which deepens their understanding of care guidelines. It also fosters deeper discussions and critical thinking, which are essential for developing communication skills and expertise in digital services, as proposed by Laukka et al. [9]. The ability of students to interact with AI patients not only enhances their understanding of contraception; it also facilitates a shift towards integrating digital tools into healthcare education. This finding aligns with the broader context highlighted by the Countdown 2030 Europe Consortium [1], which emphasises that digitalisation presents new opportunities for advancing SRHR, including contraception and family planning.
The learning objective of the AI patient simulations was to understand and apply the national Current Care Guideline on Contraception [32] in a simulated patient interaction. Assessment of the chat dialogues between the students and the AI patients showed that the students were not only familiar with the clinical guideline but also actively incorporated its key concepts into their dialogues with the generative AI patients. This indicates that the students had successfully internalised the guideline’s fundamental principles and applied this knowledge in the simulated chat discussions. While students effectively discussed lifestyle factors such as smoking, the topic of overweight was not addressed, as highlighted in the gap analysis. This oversight underscores the need for future simulations to ensure comprehensive coverage of all relevant lifestyle factors, reflecting an awareness of how these factors can impact contraceptive efficacy and patient health. These findings highlight the advantages of AI chat simulations for improving students’ understanding of contraception while also reflecting on the broader trend in healthcare towards digital tools, including chat services, which can lead to improved health outcomes and a more equitable healthcare system [2, 39].
The results indicate a significant alignment between healthcare students’ dialogue topics and MeSH terminology. Integrating MeSH terminology into chat services can effectively address the challenges posed by the absence of nonverbal cues in digital interactions, as discussed by Laukka et al. [9]. By employing precise, standardised terms, healthcare students and professionals can communicate more clearly about patients’ needs and symptoms, thereby reducing misunderstandings. Within these simulations, the students engaged in discussions on symptoms and posed questions, as also noted by Adeghe et al. [2]. They also enhanced their clinical assessment skills [10] and actively participated in decision-making processes [15, 16]. These experiences emphasise the benefits of AI for education, many of which have been documented by previous studies [21,22,23,24,25,26]. Kranz and Abele [40] have suggested that these benefits are particularly relevant in the context of midwifery education.
The topics that the students frequently discussed were contraindications, contraceptive methods, contraception, smoking, and combination contraceptives, which indicates their understanding of these key areas. However, the gap analysis revealed that certain important topics, such as overweight, were not addressed by the students. This oversight suggests areas for improvement in the educational approach, ensuring that future AI simulations comprehensively cover all relevant topics from the Current Care Guideline for Contraception. By enhancing the coverage of these topics, we can further strengthen students’ understanding and application of the guidelines in clinical practice. However, the students’ dialogues also emphasised certain topics that were not so central in the guideline. This suggests that students might prioritise certain practical or personally significant topics more than the guideline does. This difference can provide opportunities for improving education, to ensure that the students’ knowledge and practical skills align even more closely with the guideline’s content.
Regarding the use of generative AI for simulating chat dialogues, the results support the current understanding that AI systems can and should support each other. Generative AI was used to produce patient behaviour and analytical AI was used to analyse the outcome and reveal ideas for improvement. This kind of analysis between large text masses would require considerable human resources, whereas with AI, the automatable part took only a few minutes. Naturally, the researchers had to understand the method and interpret the results.
From the perspective of prompt engineering, the analysis of the student dialogues illuminated several significant topics that were not addressed but are nonetheless critical according to the Current Care Guideline. This finding underscores the potential for developing the chat patient by integrating these overlooked themes into future iterations, as the inclusion of additional topics can significantly enrich the dialogue. By employing structured prompting methods, as highlighted by Heston and Khun [27], educators can create more effective AI patient scenarios that align closely with educational objectives and clinical guidelines.
However, to fully leverage AI’s capabilities to enhance critical skills, it is essential to design scenarios that are informed by specialised knowledge in education, healthcare content and prompt engineering. This requires interdisciplinary collaboration among educators, healthcare professionals and specialists who are well-versed in generative AI interactions. This scalability enables educators to create new patient cases, thereby expanding AI-based training to other areas of healthcare education. Ultimately, by refining prompt engineering practices, the use of AI in educational settings can be optimised, fostering a more comprehensive and effective learning environment.
NLP was effectively employed in this research to analyse the chat dialogues utilising MeSH terminology and work-related topics that were well aligned with the Current Care Guideline. This innovative approach transformed the results into a quantitative model, allowing us to focus on interpreting the findings rather than on manually annotating the text. This highlights the efficiency and effectiveness of this new method for studying chat dialogues.
The study has several limitations. Firstly, the use of a novel generative AI chat simulation may have posed challenges to user familiarity and adaptability, and the variability of the AI interactions may have affected the consistency of the learning experience. Secondly, the sample consisted of a small number of healthcare students, potentially limiting the generalizability of the results to the broader healthcare student population. Thirdly, the data were based solely on dialogues within the CurreChat interface, which may have caused us to overlook other factors influencing student learning and interaction. Fourthly, the use of a single AI patient scenario may not fully reflect the diversity and complexity of real-world clinical interactions, potentially limiting the applicability of the findings. Incorporating diverse patient profiles and clinical settings into AI simulations could offer a more comprehensive learning experience, better mirroring the variability encountered in actual practice. Finally, the absence of a control group or pre/post intervention assessments restricts our ability to evaluate the educational effectiveness of AI simulations rigorously. Implementing these methodological elements could provide a more robust evaluation of learning outcomes and clearer insights into the benefits of AI-enhanced education. Lastly, as the application of NLP in this context is novel, it introduces uncertainties regarding its accuracy in this context.
Future research could employ a comparative study design to compare the knowledge and competencies of the students who utilise AI patient simulations with those of students who do not use such tools within specific fields of healthcare. In the future, as this application is further developed based on the descriptive findings of this study, it may be possible to assess effectiveness using appropriate measures. This research could involve quantitative measures, such as pre- and post-tests, alongside qualitative feedback from the participants to gain a comprehensive understanding of the impact of AI simulations on learning outcomes. Additionally, exploring diverse patient profiles and clinical complexities within AI simulations could provide a more comprehensive learning experience, better reflecting the variability of real-world clinical interactions. This approach could be tested across various healthcare fields, allowing for a broader understanding of its effectiveness and adaptability in different medical specialties and settings. One potential area of focus could be developing and optimising prompt engineering practices in the context of AI patients by examining how different prompts affect students’ learning outcomes and user experiences, and how students and educators perceive the clarity and relevance of the prompts. It could also be worthwhile investigating how NLP techniques can enhance the analysis of student interactions with AI-driven educational tools more broadly.
The study highlighted the potential of generative AI chat simulations to enhance the education of future healthcare professionals globally. Using individualised learning paths, these simulations improved educational outcomes and equipped students with essential knowledge on contraception and sexual health for informed decision-making across diverse healthcare settings.
As innovative educational tools, AI chat simulations can enrich teaching practices and learning environments. Effective prompting is crucial, as well-structured questions enhance AI’s ability to provide relevant responses, especially on sensitive topics such as contraception. Engaging with chat services also helps students develop vital communication skills, enabling them to interact empathetically and clearly with diverse populations worldwide.
A unique contribution of this study is the application of NLP. It facilitates the nuanced analysis of dialogue content, enabling educators to gain insights into student learning patterns and areas for improvement. By quantifying these interactions, NLP offers insights into the effectiveness of AI chat simulations in delivering relevant information and support, thereby enhancing educational outcomes. The integration of generative AI simulations, effective prompting, and NLP not only creates new educational opportunities but also prepares future healthcare professionals to effectively navigate digital services, ultimately improving their capacity to deliver sensitive information equitably and without discrimination across different cultural and in multiple healthcare contexts.
No datasets were generated or analysed during the current study.
Artificial Intelligence
Application Programming Interface
Abnormal Uterine Bleeding
Body Mass Index
Centers for Disease Control
Combined Oral Contraceptives
European Skills, Competences, Qualifications and Occupations
Generative Pre-trained Transformer
International Standard Classification of Occupations
Intrauterine Devices
JavaScript Object Notation
Medical Subject Headings
Natural Language Processing
Occupational Information Network
Over-the-counter
Pelvic Inflammatory Disease
Systemic lupus erythematosus
Subcutaneous
Sexual and Reproductive Health and Rights
Strengthening the Reporting of Observational Studies in Epidemiology
The Finnish Institute for Health and Welfare
World Health Organization
Countdown 2030 Europe. How Digitalisation and Sexual and Reproductive Health and Rights Can Strengthen One Another. 2020 (referred 2025, March 2). https://www.countdown2030europe.org/storage/app/media/uploaded-files/DSW%20IPPF%20Factsheet%20SRHR%20and%20digitalisation%20FINAL.pdf.
Adeghe EP, Okolo CA, Ojeyinka OT. A review of emerging trends in telemedicine: healthcare delivery transformations. Int J Life Sci Res Arch. 2024;6(1):137–47. https://doi.org/10.53771/ijlsra.2024.6.1.0040.
Article Google Scholar
Ezeamii VC, Okobi OE, Wambai-Sani H, Perera GS, Zaynieva S, Okonkwo CC, et al. Revolutionizing healthcare: how telemedicine is improving patient outcomes and expanding access to care. Cureus. 2024;16(7):e63881. https://doi.org/10.7759/cureus.63881.
Article Google Scholar
World Health Organization (WHO). The ongoing journey to commitment and transformation: digital health in the WHO European Region. Copenhagen: WHO Regional Office for Europe; 2023. Licence: CC BY-NC-SA 3.0 IGO. https://iris.who.int/bitstream/handle/10665/372051/9789289060226-eng.pdf.
World Health Organization (WHO). Ensuring human rights in the provision of contraceptive information and services: guidance and recommendations. Geneva: World Health Organization; 2014. https://iris.who.int/bitstream/handle/10665/102539/9789241506748_eng.pdf.
Senbekov M, Saliev T, Bukeyeva Z, Almabayeva A, Zhanaliyeva M, Aitenova N, et al. The recent progress and applications of digital technologies in healthcare: a review. Int J Telemed Appl. 2020;2020:8830200. https://doi.org/10.1155/2020/8830200.
Article Google Scholar
Karisalmi N, Kaipio J, Kujala S. Encouraging the use of eHealth services: a survey of patients’ experiences. Stud Health Technol Inform. 2019;257:206–11 (PMID: 30741197).
Google Scholar
Nilsson E, Sverker A, Bendtsen P, Eldh AC. A human, organization, and technology perspective on patients’ experiences of a chat-based and automated medical history-taking service in primary health care: interview study among primary care patients. J Med Internet Res. 2021;23(10):e29868. https://doi.org/10.2196/29868.
Article Google Scholar
Laukka E, Huhtakangas M, Heponiemi T, Kujala S, Kaihlanen A, Gluschkoff K, et al. Health care professionals’ experiences of patient-professional communication over patient portals: systematic review of qualitative studies. J Med Internet Res. 2020;22(12):e21623. https://doi.org/10.2196/21623.
Article Google Scholar
Jarva E, Oikarinen A, Andersson J, Tuomikoski AM, Kääriäinen M, Meriläinen M, et al. Healthcare professionals’ perceptions of digital health competence: a qualitative descriptive study. Nurs Open. 2022;9(2):1379–93. https://doi.org/10.1002/nop2.1184.
Article Google Scholar
World Health Organization (WHO). Global strategy on digital health 2020–2025. Geneva: World Health Organization; 2021. Licence: CC BY-NC-SA 3.0 IGO. https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf.
Jarva E, Oikarinen A, Andersson J, Tomietto M, Kääriäinen M, Mikkonen K. Healthcare professionals’ digital health competence and its core factors; development and psychometric testing of two instruments. Int J Med Inform. 2023;171:104995. https://doi.org/10.1016/j.ijmedinf.2023.104995.
Article Google Scholar
Brown J, Pope N, Bosco AM, Mason J, Morgan A. Issues affecting nurses’ capability to use digital technology at work: an integrative review. J Clin Nurs. 2020;29:2801–19. https://doi.org/10.1111/jocn.15321.
Article Google Scholar
Koivisto J-M, Buure T, Engblom J, Rosqvist K, Haavisto E. Association between game metrics in a simulation game and nursing students’ surgical nursing knowledge – a quasi-experimental study. BMC Nurs. 2024;23(1):16. https://doi.org/10.1186/s12912-023-01668-0.
Article Google Scholar
Kovalainen T, Pramila-Savukoski S, Kuivila H-M, Juntunen J, Jarva E, Rasi M, et al. Utilising artificial intelligence in developing education of health sciences higher education: an umbrella review of reviews. Nurse Educ Today. 2025;106600. https://doi.org/10.1016/j.nedt.2025.106600.
Article Google Scholar
Havola S, Haavisto E, Mäkinen H, Engblom J, Koivisto J-M. The effects of computer-based simulation game and virtual reality simulation in nursing students’ self-evaluated clinical reasoning skills. Comput Inform Nurs. 2021;39(11):725–35. https://doi.org/10.1097/CIN.0000000000000748.
Article Google Scholar
Cant R, Cooper S, Ryan C. Using virtual simulation to teach evidence-based practice in nursing curricula: a rapid review. Worldviews Evid Based Nurs. 2022;19(5):415–22. https://doi.org/10.1111/wvn.12572.
Article Google Scholar
Aggarwal A, Tam CC, Wu D, Li X, Qiao S. Artificial intelligence-based chatbots for promoting health behavioral changes: systematic review. J Med Internet Res. 2023;25:e40789. https://doi.org/10.2196/40789.
Article Google Scholar
Chakraborty C, Pal S, Bhattacharya M, Dash S, Lee SS. Overview of chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science. Front Artif Intell. 2023;6:1237704. https://doi.org/10.3389/frai.2023.1237704.
Article Google Scholar
Liu J, Liu F, Fang J, Liu S. The application of chat generative pre-trained transformer in nursing education. Nurs Outlook. 2023;71(6):102064. https://doi.org/10.1016/j.outlook.2023.102064.
Article Google Scholar
Chan K, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5(1):e13930. https://doi.org/10.2196/13930.
Article Google Scholar
Adiguzel T, Kaya MH, Cansu FK. Revolutionizing education with AI: exploring the transformative potential of ChatGPT. Contemp Educ Technol. 2023;15(3):ep429. https://doi.org/10.30935/cedtech/13152.
Article Google Scholar
Gray M, Baird A, Sawyer T, James J, DeBroux T, Bartlett M, et al. Increasing realism and variety of virtual patient dialogues for prenatal counseling education through a novel application of ChatGPT: exploratory observational study. JMIR Med Educ. 2024;10:50705. https://doi.org/10.2196/50705.
Article Google Scholar
Stamer T, Steinhäuser J, Flägel K. Artificial intelligence supporting the training of communication skills in the education of health care professions: scoping review. J Med Internet Res. 2023;25(8):e43311. https://doi.org/10.2196/43311.
Article Google Scholar
Suárez A, Adanero A, Díaz-Flores García V, Freire Y, Algar J. Using a virtual patient via an artificial intelligence chatbot to develop dental students’ diagnostic skills. Int J Environ Res Public Health. 2022;19(14):8735. https://doi.org/10.3390/ijerph19148735.
Article Google Scholar
Ahmad SF, Rahmat MK, Mubarik MS, Alam MM, Hyder SI. Artificial intelligence and its role in education. Sustainability. 2021;13(22):12902. https://doi.org/10.3390/su132212902.
Article Google Scholar
Heston TF, Khun C. Prompt engineering in medical education. Int Med Educ. 2023;2(3):198–205. https://doi.org/10.3390/ime2030019.
Article Google Scholar
Chan CKY, Hu W. Students’ voices on generative AI: perceptions, benefits, and challenges in higher education. Int J Educ Technol High Educ. 2023;20(1):43. https://doi.org/10.1186/s41239-023-00411-8.
Article Google Scholar
Liew TW, Wei Ming P, Chew L, Tan S-M. Anthropomorphizing malware, bots, and servers with human-like images and dialogues: the emotional design effects in a multimedia learning environment. Smart Learn Environ. 2022;9. https://doi.org/10.1186/s40561-022-00187-w.
Hemminki-Reijonen U, Hassan NMAM, Huotilainen M, Koivisto J-M, Cowley B. Design of generative ai-powered pedagogy for virtual reality environments in higher education. NPJ Sci Learn. 2025;10:13. https://doi.org/10.1038/s41539-025-00326-1.
Cain W. Prompting change: exploring prompt engineering in large language model AI and its potential to transform education. TechTrends. 2024;68:47–57. https://doi.org/10.1007/s11528-023-00896-0.
Article Google Scholar
Working group set up by the Finnish Medical Society Duodecim, the finnish society of obstetrics and gynaecology and the finnish association for general practice. Current Care Guidelines. Contraception. 2022 (referred 2025, March 3). Helsinki: The Finnish Medical Society Duodecim. https://www.kaypahoito.fi/hoi50104#K1.
Gray JG. Quantitative Methodology: Noninterventional Designs and Methods. In: Grey JR, Grove SK, editors. Burn’s & Grove’s The practice of Nursing Research. Appraisal, Synthesis, and Generation of Evidence, 9th Edition. Elsevier; 2021. p. 249. ISBN: 9780323673174.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–7. https://doi.org/10.1136/bmj.39335.541782.AD. (PMID: 18064739).
Article Google Scholar
Vilanti T, Koivisto J-M. Creating AI-Patients Eeva and Liisa. 2024. (referred 2025, February 15). https://blogs.helsinki.fi/globalcampus/2024/10/18/creating-ai-patients-eeva-and-liisa/.
Kaul V, Morris A, Chae JM, Town JA, Kelly WF. Delivering a novel medical education “Escape Room” at a national scientific conference: first live, then pivoting to remote learning because of COVID-19. Chest. 2021;160. https://doi.org/10.1016/j.chest.2021.04.069.
Article Google Scholar
Gray JG. Qualitative Research Methods. In: Grey JR, Grove SK, editors. Burn’s & Grove’s The practice of Nursing Research. Appraisal, Synthesis, and Generation of Evidence, 9th Edition. Elsevier; 2021. p. 314–357. ISBN: 9780323673174.
Finnish National Board on Research Integrity TENK. The Finnish Code of Conduct for Research Integrity and Procedures for Handling Alleged Violations of Research Integrity in Finland 2023. Guideline of the Finnish National Board on Research Integrity TENK 2023. 2023. ISBN 978–952–5995–88–6. https://tenk.fi/sites/default/files/2023-11/RI_Guidelines_2023.pdf.
Secinaro S, Calandra D, Secinaro A. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak. 2021;21:125. https://doi.org/10.1186/s12911-021-01488-9.
Article Google Scholar
Kranz A, Abele H. The impact of artificial intelligence (AI) on midwifery education: a scoping review. Healthcare. 2024;12(11):1082. https://doi.org/10.3390/healthcare12111082.
Article Google Scholar
Download references
We want to thank Jussi-Pekka Järvinen and the educational technology services at the University of Helsinki as well as the senior university lecturer Matti Luukkainen and the Toska software development academy from the Department of Computer Science at the University of Helsinki for their contributions in the development of the CurreChat features needed for this research. Special thanks to Noha M.A.M. Hassan for her contribution with the prompt engineering of the AI patient simulation. Open access funded by Helsinki University Library.
Not applicable.
Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital). This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Teaching and Learning Services, University Services, Faculty of Medicine, University of Helsinki, Haartmaninkatu 8, Helsinki, 00014, Finland
Titta Vilanti
Department of Obstetrics and Gynecology, Clinicum, Faculty of Medicine, University of Helsinki, Haartmaninkatu 2, Helsinki, 00014, Finland
Kaisu Luiro
Gynecology and Obstetrics Unit, Reproductive Medicine Unit, HUS, Helsinki University Hospital, Mannerheimintie 164a, Helsinki, 00019, Finland
Kaisu Luiro
Department of Public Health, Clinicum, Faculty of Medicine, University of Helsinki, Tukholmankatu 8 B, Helsinki, 00014, Finland
Inari Dahlqvist, Jutta Piipponen & Jaana-Maija Koivisto
Faculty of Educational Sciences, University of Helsinki, Siltavuorenpenger 5, Helsinki, 00014, Finland
Ulla Hemminki-Reijonen & Sasa Tkalcan
Department of Computer Science, Faculty of Science, University of Helsinki, Pietari Kalmin Katu 5, Helsinki, 00014, Finland
Ulla Hemminki-Reijonen & Sasa Tkalcan
Services for Digital Education and Continuous Learning (DOJO)/Educational Technology Services (OTE), University of Helsinki, Unioninkatu 38, Helsinki, 00014, Finland
Ulla Hemminki-Reijonen
HeadAI, Rautatienpuistokatu 7, Pori, 28130, Finland
Harri Ketamo
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
The authors contributed as followed: T.V.: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review and editing, Visualization K.L.: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review and editing, Visualization I.D.: Formal analysis, Investigation, Data curation, Writing – review and editing, Visualization J.P.: Formal analysis, Investigation, Data curation, Writing – review and editing, Visualization U.H-R.: Resources, Writing – original draft, Writing – review and editing S.T.: Software, Resources, Data curation, Writing – original draft, Writing – review and editing, Visualization H.K.: Conceptualization, Methodology, Software, Validation, Formal analysis, Resources, Data curation, Writing – original draft, Writing – review and editing, Visualization J-M.K.: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Writing – original draft, Writing – review and editing, Project administration, Supervision.
Correspondence to Titta Vilanti.
The Responsible Conduct of Research (RCR) guidelines of the Finnish National Board on Research Integrity TENK [38] were followed at all stages of the study. Ethical approval was received from the University of Helsinki Research Ethics Committee in the Humanities and Social and Behavioural Sciences on June 4, 2024. Research permits were obtained from the participating organisations. All the participants who enrolled signed an informed consent form.
Not applicable. The data collected were pseudonymised before analysis.
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Reprints and permissions
Vilanti, T., Luiro, K., Dahlqvist, I. et al. Contraception-related topics in chat dialogues between healthcare students and generative AI patients: a natural language processing analysis. BMC Med Educ 25, 1458 (2025). https://doi.org/10.1186/s12909-025-08032-7
Download citation
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1186/s12909-025-08032-7
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Avoid common mistakes on your manuscript.
Advertisement
148.135.140.220
Not affiliated
© 2025 Springer Nature