When AI is used to support clinical decisions, bias in the model can lead to poorer care and deepen existing health disparities. This review explains how bias can arise and build up at every stage of an AI tool’s life, from the data used to build it to the way it is used in practice and even how research is published.
The authors identify several sources of bias. In the data, too few patients from some groups can lead to weaker performance and underestimated risk for them, and missing information, such as on social determinants of health, can also distort results. The labels used to train models may carry clinicians’ unconscious biases or reflect substandard care. Relying on overall performance figures can hide poorer results for specific groups, and a model’s performance may fall, sometimes unevenly, when it is used with patients different from those it was trained on. Finally, where models are developed and published, and by whom, shapes the priorities of future research.
To address this, the authors recommend larger and more diverse datasets, statistical methods to reduce bias, thorough evaluation, a focus on making models interpretable, and standardised reporting of bias. They also stress that models should be tested in rigorous clinical trials before being used in real-world care.
Learn more about this review here: https://doi.org/10.1371/journal.pdig.0000651
Reference:
Cross, J.L., Choma, M.A., & Onofrey, J.A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS Digital Health, 3(11), e0000651.