AI is transforming medicine, but it is mostly developed by men and trained on data that prioritises men’s bodies and health needs. In this commentary, researchers from the George Institute for Global Health argue that this embeds sex and gender biases that can put women and nonbinary patients at risk. For example, natural language processing models used in psychiatry have shown significant gender biases, and tools that screen for suicide risk may make mistakes if trained mainly on text written by men, since men and women express distress differently.
The authors also argue that AI could help reduce these inequities if sex and gender are considered deliberately. One example is the GRACE heart attack risk score, which was updated using AI models that analyse male and female data separately to account for sex-specific characteristics of the disease. Another is a chatbot developed by the George Institute to give guideline-based pregnancy advice in rural India, created together with community health workers, clinicians and women.
They call for sex and gender bias to be actively addressed in the data used to train AI, for tools to be co-designed with the communities they serve, for work on bias to go beyond the English language, and for sex and gender to be considered at every stage of research and clinical use.
Learn more here: https://www.unsw.edu.au/newsroom/news/2024/10/can-ai-fight-sex-and-gender-bias-in-healthcare-
Reference:
Haupt, S., Graham, B., & Hirst, J. (2024, October 21). Can AI fight sex and gender bias in healthcare? 360info.