Please note: this is a preprint study, which means it has not yet been peer reviewed.
Large language models such as ChatGPT are increasingly being explored as tools to support clinical decisions, but there are concerns that they may reproduce gender and racial biases. This study tested whether ChatGPT 3.5 showed such biases when recommending how to manage acute coronary syndrome, a group of conditions that includes heart attacks and unstable angina.
When patients were described as female, African American or Hispanic, ChatGPT recommended guideline-based diagnosis, treatment and symptom management less often. The largest gaps were in recommending coronary angiography or stress testing, used to diagnose the condition and plan further treatment, and high-intensity statins. These differences mirror biases already observed in clinical practice, which have been linked to worse outcomes for women and racial minorities with heart disease. The gaps were largest for unstable angina, where clinical guidelines are less explicit.
The researchers also found that asking ChatGPT to explain its reasoning before giving an answer improved its clinical accuracy and reduced gender and racial bias. According to the authors, this is among the first studies to show that the biases of these models affect clinical management, and that existing strategies to improve their performance can also help reduce bias.
Learn more about this study here: https://doi.org/10.48550/arXiv.2311.14703
Reference:
Zhang, A., Yuksekgonul, M., Guild, J., Zou, J., & Wu, J.C. (2023). ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management. arXiv preprint arXiv:2311.14703.