Biased AI algorithms can result in some groups of patients not receiving appropriate care, potentially leading to significant harm. In one example, an algorithm designed to identify patients who needed extra care used healthcare costs to estimate their needs. Because Black patients face barriers in accessing care, less was spent on them, and the algorithm may have more than halved the number of Black patients identified for extra care.
In April 2024, the US Department of Health and Human Services issued a rule that explicitly prohibits discriminatory outcomes from tools that support clinical decisions, including AI. Health care facilities now have to identify and reduce the risk of discrimination when using these tools. However, the authors argue that guidelines, resources and shared responsibilities are still lacking. Algorithms need to be monitored locally over time, which may be possible for well-resourced facilities but is a major challenge for under-resourced ones. These facilities may either avoid AI tools that could benefit their communities, or adopt tools that produce biased results without being aware of it.
The authors propose a model of shared responsibility between health care facilities, AI developers and regulators. Developers should be transparent, showing that diverse data were used, describing how they tested their algorithms for bias, and disclosing any limitations, with this information stored in a public central repository. Existing federal requirements for AI transparency should also be aligned with the new anti-discrimination rule.
Learn more about this article here: https://doi.org/10.1001/jama.2024.13486
Reference:
Ratwani, R. M., Sutton, K., & Galarraga, J. E. (2024). Addressing AI algorithmic bias in health care. JAMA, 332(13), 1051–1052.