News
Study: Generative AI Outperforms People in Screening Patients for Clinical Trial
Mass General Brigham, a U.S.-based academic healthcare system, recently released a study showing that artificial intelligence may be more efficient at screening trial patients than the researchers who normally accomplish the task.
The study, published on June 17 in the New England Journal of Medicine, AI edition, found that a system based on OpenAI's GPT 4 model, which they named RECTIFIER, performed "better than the study staff in determining symptomatic heart failure, with an accuracy of 97.9% versus 91.7%."
And RECTIFIER also saved money, reducing the cost of the process to as much as two cents per patient.
In other words, the researchers found that automating the screening process via a generative AI both increased accuracy and saved the organization money, according to the study's authors Ozan Unlu, M.D, Jiyeon Shin, A.L.M., Charlotte J. Mailly, B.A, Michael F. Oates, Alexander J. Blood, M.D., M.Sc, et.al.
However, the researchers state, there are still concerns:
"Examples of potential hazards associated with automating the clinical trial screening process include loss of nuanced patient context, operational hazards due to system downtime, overlooked critical clinical details, potential inequity, and performance sensitivity to changes in data capture and clinical processes…Although RECTIFIER did not display statistically significant differences across racial or ethnic groups, directional differences both between groups and relative to study staff could prove statistically significant at scale. Given these potential hazards, the integration of GPT-4 into clinical trial screening necessitates a careful balance between embracing technological advancements and mitigating the associated risks."
The researchers recommend any systems involving patients having plenty of checks and doublechecks in place.
Co-lead author Ozan Unlu, MD, a fellow in Clinical Informatics at Mass General Brigham and a fellow in Cardiovascular Medicine at Brigham and Women's Hospital, commented in a press release announcing the research results, "Screening of participants is one of the most time-consuming, labor-intensive, and error-prone tasks in a clinical trial," highlighting the need of such systems.
"If we can accelerate the clinical trial process, and make trials cheaper and more equitable without sacrificing safety, we can get drugs to patients faster and ensure they are helping a broad population," co-researcher Alexander J. Blood, M.D.,M.Sc commented.
A press release summarizing the paper's findings can be found here.
About the Author
Becky Nagel serves as vice president of AI for 1105 Media specializing in developing media, events and training for companies around AI and generative AI technology. She also regularly writes and reports on AI news, and is the founding editor of PureAI.com. She's the author of "ChatGPT Prompt 101 Guide for Business Users" and other popular AI resources with a real-world business perspective. She regularly speaks, writes and develops content around AI, generative AI and other business tech. Find her on X/Twitter @beckynagel.