Most AI tools in healthcare learn from large datasets, but many groups of people have long been missing or misrepresented in medical data. When AI is trained on data that does not reflect the whole population, it can reinforce existing biases and lead to misdiagnoses and even fatal outcomes, particularly for gender and ethnic minorities.
The authors give several examples. AI tools for classifying skin lesions are often trained on datasets in which only about 5% to 10% of images come from Black patients, and when tested on images of Black patients they reach about half the accuracy their creators claimed. This matters because Black patients have the highest mortality from melanoma. In cardiology, heart attacks are often misdiagnosed in women, yet some models that predict heart attacks were trained mostly on data from men. Many datasets also record gender only as male or female, making it difficult to build AI that includes LGBTQ+ people or to test it for bias.
To move towards fairer AI, the authors propose open science practices: involving patients and communities in the development of AI, sharing data responsibly while protecting privacy, using inclusive data standards, and sharing code. Sharing code and algorithms allows researchers around the world to test whether a tool developed with data from one population also works for patients elsewhere.
Learn more about this article here: https://doi.org/10.1016/j.patter.2021.100347
Reference:
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021). Addressing bias in big data and AI for health care: A call for open science. Patterns, 2(10), 100347.