Simulation Gets Smarter

The Gordon Center’s innovative use of simulation in medical training and education stretches back nearly 50 years, and its intellectual underpinnings even further. The designated Center of Excellence owes much of its international reputation to Michael S. Gordon, M.D., Ph.D., a practicing cardiologist who, in the late 1960s, recognized the potential of simulation for creating realistic training scenarios for aspiring physicians.

Dr. Gordon was a pioneer in the early integration of technology and medical education, a techie before tech was cool. In 1968, he unveiled Harvey®, a cardiology patient simulator whose direct descendants continue to be a vital training tool for medical education.

Dr. Issenberg and his colleagues are continuing Dr. Gordon’s forward-thinking work. Patient simulators are now produced in different genders, ages and skin colors, with increasingly sophisticated computer-programmed responses that better mimic human medical conditions. In addition, 3D printing is used to create anatomically accurate models for surgical planning and training. It allows for customization of patient-specific anatomy, enhancing the realism and relevance of surgical rehearsals. Bringing health care to the metaverse, wearable technology devices such as smart glasses or haptic feedback gloves enhance simulation training, as they provide a more immersive experience and simulate more realistically the audiovisual and tactile feedback of medical procedures.

“Performance in a simulation can predict real-world behaviors,” said Ross Scalese, M.D., the Gordon Center’s director of educational technology development and professor in the Miller School’s Department of Medicine. “The more authentic the simulations are, the more likely the training exercises will translate to better patient care in actual clinical environments.”

A new program takes one group of caregivers a step further: The community paramedicine program teaches paramedics how not only to recognize and evaluate medical problems, but also to identify patients’ social determinants of health, such as housing or access to medicine, and refer them to services as needed. The goal is to reduce hospital readmissions of people whose living situations keep them coming back because they never really get well.

Dr. Issenberg believes that AI will be most useful to simulation training in evaluating performance for procedure-based tasks. Surgery is an obvious match, as it relies on a specific procession of steps that lead to a successful outcome. The current process often involves faculty reviewing lengthy videos of surgical trainees to determine if those steps were followed correctly. Absent technological intervention, that approach takes a long time and can lead to waning concentration after a few hours. This is where AI — augmented intelligence, in this case, as it’s helping rather than replacing a person — can prove beneficial.

“The procedure’s order and steps are well defined, so we can develop algorithms for an AI application and then feed the application training videos,” Dr. Issenberg said. “The application learns from those videos, and after a relatively short period of time, it’s as accurate as an expert.”

The expertise the AI application develops enables it to pinpoint errors and reduces the time needed for review from hours to minutes. AI also provides the opportunity to assess the effectiveness of the curriculum itself by identifying where trainees struggle or perform without proficiency. That assessment allows faculty instructors to review and revise training materials as soon as an issue surfaces, rather than waiting for the end of the semester.

“AI can be used to identify the components of the case people are having difficulty with,” Dr. Issenberg said. “And we can make tweaks in real time. AI will enable us to have a much more individualized curriculum and adaptive learning.”

In many ways, the transition from medical student to physician is a movement from theory to practice. Students absorb theories from textbooks to learn how to address medical problems and put that knowledge into practice when the “M.D.” formally appends their name. Problems arise when patients deviate from the textbook.

“Medical students read about one or two prototypical cases in a textbook,” Dr. Issenberg said, “and then on their first day at UHealth Tower or Jackson Memorial Hospital, they see a patient who doesn’t look anything like what they read about in a textbook or learned during a lecture.”

Simulation training strives to bridge the gap between theory and practice by accounting for deviations from the norm and presenting as varied a curriculum as possible. But time, that relentless obstacle, can make that challenging.

“Having a large number of training cases is important,” Dr. Issenberg said, “but it takes a lot of time for faculty to come up with these cases.”

AI, however, doesn’t need much time to synthesize a large volume of de-identified patient notes and capture all the variables in that data set.

“In a matter of minutes, we can create a large library of patient simulation cases using AI,” Dr. Issenberg said. “Faculty can then go in and correct any mistakes, any biases that may occur.”

Vivian Obeso, M.D. ’00, an associate professor of medical education, is spearheading a project that does just that. Dr. Obeso is in the early experimental phase of using ChatGPT to generate realistic, varied patient scenarios that can be used for clinical training and assessment. Currently, Miller School faculty develop those scenarios, which are translated to conversational scripts that actors use in face-to-face encounters with medical students. It can be arduous work.

“The scripts need to be developed carefully,” Dr. Obeso said. “It usually takes at least a couple of faculty members to write them and get the appropriate feedback. The scripts must be accurate enough to portray a realistic patient scenario. It can take hours, sometimes days.”

Not with ChatGPT and similar generative natural language processing applications. Dr. Obeso is posing and refining prompts that include information about de-identified patient symptoms, health history and physical examination findings, then asking ChatGPT to produce plausible scripts that support a given differential diagnosis. The preliminary results demonstrate that AI can do a pretty decent imitation of a real patient scenario with a carefully developed prompt.

“I started with broad questions but found that ChatGPT does better with more detailed inquiry,” Dr. Obeso said. Given the proper specifics, “it does a good job generating patient cases that include pertinent positive and negative findings, as well as a differential diagnosis,” she added.

That’s not good enough to forego physician oversight, but Dr. Obeso plans to organize a study that will ask faculty physicians to review 100 or more ChatGPT-generated scenarios for accuracy.

“We’ll be able to identify trends and see what ChatGPT is doing really well,” she said, “and address why it’s struggling in certain areas.”

AI then will be able to alleviate a good portion of the labor-intensive scenario-creation burden. The faculty physicians who endorse the training curriculum will focus on fine-tuning the AI output with subtle variations to the questions that ChatGPT “consumes.”

This approach should be especially useful in the Miller School’s monthlong Transition to Residency course — nicknamed “Residency Boot Camp” — that is now required for all fourth-year students.

“We feel this type of simulation training in acute medical emergencies is critical for every graduating student,” said Gauri Agarwal, M.D. ’00, associate dean for curriculum and associate professor of medicine. “Students are trained not only to recall critical pieces of medical knowledge, but also to practice their composure, communication, decision-making and teamwork skills with patients and colleagues.”

While AI’s anticipated impact on medicine is exciting, “overreliance on AI in training simulations could potentially diminish the trainees’ ability to develop critical thinking and decision-making skills,” Dr. Issenberg said. “They might become too dependent on AI recommendations and analysis. As educators, we need to guard against that effect and demonstrate the serious implications of relying too much on this new technology.”

As Dr. Obeso learned in her first forays into ChatGPT, AI is only as good as its source material. Bad data means inadequate or even harmful training.

“AI-driven simulations are heavily reliant on the quality of data they are fed,” Dr. Issenberg said. “If the data are incomplete, outdated or biased, it can lead to inaccurate simulations, which may misguide trainees rather than helping them.”

And as patient data become potentially more accessible, privacy will gain even more importance. Academic medical centers like the Miller School are bound by the strict dictates of the Health Insurance Portability and Accountability Act of 1996, commonly referred to as HIPAA, which protects sensitive patient information. Dr. Issenberg recommends the strongest measures in ensuring confidentiality and hopes patient communications will evolve to include explicit details about the use of patient data for research and education.

Dr. Motola agrees: “Data security is going to be huge, because with facial recognition, for instance, identity means access to an additional level of the patient’s medical information,” she said. “One of the challenges is to ensure it is protected. How individuals consent to have their data used for both training and clinical care purposes will have to be integrated into patient care interactions, and data security will need to be at the highest level and monitored vigilantly.”