Tag: Racial Bias

  • Addressing AI Algorithmic Bias in Health Care

    Addressing AI Algorithmic Bias in Health Care

    Biased AI algorithms can result in some groups of patients not receiving appropriate care, potentially leading to significant harm. In one example, an algorithm designed to identify patients who needed extra care used healthcare costs to estimate their needs. Because Black patients face barriers in accessing care, less was spent on them, and the algorithm may have more than halved the number of Black patients identified for extra care.

    In April 2024, the US Department of Health and Human Services issued a rule that explicitly prohibits discriminatory outcomes from tools that support clinical decisions, including AI. Health care facilities now have to identify and reduce the risk of discrimination when using these tools. However, the authors argue that guidelines, resources and shared responsibilities are still lacking. Algorithms need to be monitored locally over time, which may be possible for well-resourced facilities but is a major challenge for under-resourced ones. These facilities may either avoid AI tools that could benefit their communities, or adopt tools that produce biased results without being aware of it.

    The authors propose a model of shared responsibility between health care facilities, AI developers and regulators. Developers should be transparent, showing that diverse data were used, describing how they tested their algorithms for bias, and disclosing any limitations, with this information stored in a public central repository. Existing federal requirements for AI transparency should also be aligned with the new anti-discrimination rule.

    Learn more about this article here: https://doi.org/10.1001/jama.2024.13486


    Reference:

    Ratwani, R. M., Sutton, K., & Galarraga, J. E. (2024). Addressing AI algorithmic bias in health care. JAMA, 332(13), 1051–1052.

  • Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change?

    Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change?

    Clinical algorithms are risk calculators and decision tools that doctors use every day, and many of them take a patient’s race into account. In Boston, for example, paediatrician Alexandra Epee-Bounya used a calculator to assess young children’s risk of a urinary tract infection that added two points if the child was a girl and one point if the child was not Black. The more points, the higher the estimated risk, and the more likely the child was to receive follow-up testing.

    This article opens STAT’s “Embedded Bias” series, which spent more than a year investigating why race, a social construct rather than a biological fact, is still so widely used in medicine. Race-adjusted calculators, used for example to assess kidney function, lung capacity and the risk of urinary tract infections, have been shown to underdiagnose and harm patients of colour. Although some hospitals and medical organisations have started to change, many of these algorithms remain in use. Across the series, the reporters identified 48 race-based clinical algorithms still in use.

    Some argue that race can still provide useful predictive information, for example in breast cancer or surgical risk calculators. Critics say this reinforces the false idea that race is a biological category.

    Learn more here: https://www.statnews.com/2024/09/03/embedded-bias-investigation-health-equity-clinical-algorithms/


    Reference:

    Palmer, K., & McFarling, U. L. (2024, September 3). Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change? STAT.

  • Common heart failure may be underdiagnosed in Black patients, AI-based study finds

    Common heart failure may be underdiagnosed in Black patients, AI-based study finds

    Heart failure with preserved ejection fraction (HFpEF) accounts for around half of heart failure cases in the UK. In this condition, the heart pumps blood normally but does not fill properly, causing symptoms such as breathlessness, fatigue and dizziness. Researchers at King’s College London, funded by the British Heart Foundation, studied how this condition is diagnosed in patients of different ethnicities.

    The team used a natural language processing algorithm, an AI tool that reads and interprets medical text, to analyse electronic health records from a large London hospital. It identified 1,973 patients who met European guidelines for HFpEF: 64% White, 29% Black and 7% Asian. Compared with White patients, Black patients were younger at diagnosis, more often had obesity, diabetes and high blood pressure, and less often had atrial fibrillation. When the researchers compared these results with routine care, they found that Black and Asian patients were more likely to have been missed in routine practice than by the algorithm.

    The researchers believe this is linked to the H2FPEF score, which clinicians partly use to diagnose HFpEF and which gives more weight to atrial fibrillation. Black patients were less likely than White patients to reach a score indicating likely HFpEF (26% vs 44%), so relying on this score may lead to more Black patients being missed. The authors conclude that established markers appear to underdiagnose HFpEF in Black patients and that more research is needed to improve diagnostic tools.

    Learn more here: https://www.bhf.org.uk/what-we-do/news-from-the-bhf/news-archive/2024/july/ai-reduces-underdiagnosis-of-common-heart-failure-in-black-patients-new-research-finds
    Read the study here: https://doi.org/10.1016/j.jacadv.2024.101064


    References:

    British Heart Foundation. (2024, July). AI reduces underdiagnosis of common heart failure in Black patients, new research finds.

    Brown, S., Biswas, D., Wu, J., Ryan, M., Bernstein, B. S., Fairhurst, N., Kaye, G., Baral, R., Cannata, A., Searle, T., Melikian, N., Sado, D., Lüscher, T. F., Teo, J., Dobson, R., Bromage, D. I., McDonagh, T. A., Vazir, A., Shah, A. M., & O’Gallagher, K. (2024). Race- and ethnicity-related differences in heart failure with preserved ejection fraction using natural language processing. JACC: Advances, 3(8), 101064.

  • Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis

    Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis

    HbA1c is a blood test that reflects average blood sugar levels over the previous months, and it is widely used to diagnose diabetes using a single cut-off value for everyone. However, HbA1c levels appear to differ between ethnic groups. Because most earlier studies included people with diabetes, whose blood sugar varies more, this systematic review focused only on people without diabetes.

    The review combined the results of 12 studies with a total of 49,238 adults. Compared with White participants, HbA1c levels were significantly higher in Black participants (by 0.26 percentage points), Asian participants (by 0.24 points) and Latino participants (by 0.08 points).

    The authors conclude that, although these differences are small, they may affect the use of a single HbA1c threshold to diagnose diabetes in all ethnic groups.

    Learn more about this review here: https://doi.org/10.1371/journal.pone.0171315


    Reference:

    Cavagnolli, G., Pimentel, A.L., Freitas, P.A.C., Gross, J.L., & Camargo, J.L. (2017). Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis. PLOS ONE, 12(2), e0171315.

  • Race, Racism, and Risk Prediction for Cardiovascular Disease

    Race, Racism, and Risk Prediction for Cardiovascular Disease

    To estimate a patient’s risk of heart disease and stroke, many clinicians have used a tool called the Pooled Cohort Equations, which had separate equations for Black and White people. In recent years, concerns have grown that this approach may reinforce harmful stereotypes and suggest that race itself has a biological basis for disease. In this commentary, two cardiologists from the American Heart Association explain why the association’s new risk calculator, PREVENT, no longer uses race.

    Instead of race, PREVENT considers factors such as kidney and metabolic health, which affect Black Americans more often, and a social deprivation index based on postcode, reflecting the social conditions that drive racial disparities in cardiovascular disease. It was developed and validated using data from more than 3 million people each, with diverse races and ethnicities and equal numbers of women and men, and proved accurate across all racial and ethnic groups studied. According to the authors, leaving race out also helps to make clear that race is associated with disease, but does not cause it.

    The authors stress, however, that a more accurate calculator does not eliminate unconscious bias or the injustices that people of certain races and ethnicities face in healthcare. It is the effects of racism, not race itself, that are linked to disease risk. They describe PREVENT as a first step towards fairer tools in cardiovascular care, and note that research is under way to test whether it works equally well for everyone, regardless of how they identify, how much they earn or where they live.

    Learn more here: https://newsroom.heart.org/news/race-racism-and-risk-prediction-for-cardiovascular-disease


    Reference:

    Churchwell, K., & Khan, S.S. (2024, March 12). Race, Racism, and Risk Prediction for Cardiovascular Disease. American Heart Association Newsroom.

  • Assessing racial bias in type 2 diabetes risk prediction algorithms

    Assessing racial bias in type 2 diabetes risk prediction algorithms

    Risk prediction tools are widely used to identify people at high risk of type 2 diabetes, so that they can be offered prevention and early treatment. This study examined whether three commonly used tools in the US (the Prediabetes Risk Test, the Framingham Offspring Risk Score and the ARIC model) are equally accurate for non-Hispanic White and non-Hispanic Black adults.

    The researchers used data from 9,987 adults without a diabetes diagnosis who took part in national health surveys between 1999 and 2010, and compared the risk predicted by each tool with the rates of type 2 diabetes actually observed. All three tools were miscalibrated by race, and this pattern was consistent across the years studied. The Framingham score overestimated risk for White adults and underestimated it for Black adults, while the other two tools overestimated risk for both groups, but more so for White adults.

    According to the authors, this means that White adults may be prioritised for prevention, with a greater risk of overdiagnosis and overtreatment, while Black adults may be underprioritised and undertreated. The findings highlight the need to reassess these tools to ensure that diabetes risk is predicted fairly across racial groups.

    Learn more about this study here: https://doi.org/10.1371/journal.pgph.0001556


    Reference:

    Cronjé, H.T., Katsiferis, A., Elsenburg, L.K., Andersen, T.O., Rod, N.H., Nguyen, T.-L., & Varga, T.V. (2023). Assessing racial bias in type 2 diabetes risk prediction algorithms. PLOS Global Public Health, 3(5), e0001556.

  • The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations

    The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations

    Diabetes outcomes differ across ethnic and racial groups, yet AI tools for diabetes management are often developed without taking this into account. To assess how well these groups are represented, the authors re-analysed 141 articles from an earlier review of AI in diabetes care.

    Only 10 of the 141 articles (7.1%) reported the ethnic or racial background of their participants. In those that reported race, participants were on average 69.5% White, 17.1% Black and 3.7% Asian, and only two articles included Native American participants. According to the authors, AI tools trained on such unrepresentative data risk producing biased results for the groups that are left out.

    To address this, the authors propose a short questionnaire that researchers and clinicians can use to assess ethnoracial equity in AI diabetes research. It covers whether ethnoracial differences in the disease are discussed, whether the training data and its limitations are described, whether the training population reflects the population the tool will serve, and whether strategies to reduce bias are in place. As the authors put it, “AI can either mitigate or exacerbate disparities in health care.”

    Learn more about this article here: https://doi.org/10.2196/22320


    Reference:

    Pham, Q., Gamble, A., Hearn, J., & Cafazzo, J.A. (2021). The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations. Journal of Medical Internet Research, 23(2), e22320.

  • When “Neutral” Data Isn’t: Gender and Racial Bias in Biomedical AI

    When “Neutral” Data Isn’t: Gender and Racial Bias in Biomedical AI

    Bias in AI isn’t abstract. It shows up in real diagnoses, real treatment decisions, and real patients, particularly those who already face multiple layers of disadvantage. Discrimination in AI is an intersectional issue, overlapping with age, gender, race, sexual orientation, and health status, among other dimensions of identity. The patients most at risk of being harmed by biased AI tools are often those who already face the most barriers to quality care.

    Decades of biased medical data

    The biases we see in AI didn’t come from nowhere. They’re a direct consequence of medical data collected over decades, data shaped by systemic discrimination and structural inequalities in healthcare research and practice (Cirillo et al., 2020; Cross et al., 2024).

    For much of medical history, clinical and experimental studies focused predominantly on male participants. This is what is known as the gender health gap. In 2020, only 5% of global health research funding went to women’s health research. This was split into 4% for women’s cancers and 1% for all other women-specific health conditions, with 25% of that further limited to fertility research (Nature Reviews Bioengineering, 2024). 

    It’s worth noting this isn’t a one-way problem: in some areas, such as depression, men are underrepresented in clinical data, largely because they’re less likely to seek care, report symptoms, or receive a diagnosis (Smith et al., 2018).

    Racial bias is equally well documented

    Racial bias in medicine is also extensively documented, particularly in the United States where research shows that, for instance, Black patients and other minority groups receive fewer medical procedures, lower rates of surgical intervention, and fewer referrals to specialists than white patients, regardless of clinical need (Bowser, 2001; Williams & Wyatt, 2015). When AI systems are trained on data that reflects decades of existing inequalities in healthcare access and treatment, they risk encoding and perpetuating those inequalities at a much larger scale. While much of the evidence comes from the US, the structural conditions producing racial bias, including socioeconomic inequalities, barriers to access and underrepresentation in research, are present across Europe too.

    LGBTQIA+ patients face their own layer of risk

    It is also important to recognise the specific situation of LGBTQIA+ individuals, who experience discrimination in healthcare and are subject to stereotypes that affect the care they receive. These social and cultural factors perpetuate discrimination and have a measurable impact on health and healthcare. Research has shown that 16% of LGBTQIA+ individuals report discrimination in healthcare encounters, and 18% avoid seeking care altogether due to fear of mistreatment (Chang et al., 2025). Another large-scale study found that in emergency department scenarios, AI recommendations for LGBTQIA+ patients included mental health interventions six to seven times more often than was clinically appropriate (Chang et al., 2025).

    How this plays out in three key areas

    In cardiovascular care, women’s symptoms often differ from the “classic” presentation described in medical textbooks, itself largely based on male patient data (Fatunde et al., 2025). As a result, women are offered fewer diagnostic tests, less medication, and fewer specialist referrals (Al Hamid et al., 2024). Racial bias compounds the problem: pulse oximeters, devices routinely used to measure blood oxygen saturation, have been shown to produce less accurate readings for patients with darker skin tones (Sjoding et al., 2020), a bias that then feeds directly into AI-based triage systems.

    In diabetes care, racial bias is extensively documented. Studies have shown that African American patients present systematically higher A1c values than white patients with the same average blood glucose (Karter et al., 2023). If an AI system uses A1c as a proxy for glycaemic control without accounting for this, it risks producing incorrect diagnoses for Black patients.

    In depression, both gender and racial bias carry significant weight, especially in tools using natural language processing for screening. Men and women tend to express psychological distress differently (Pennebaker et al., 2003), so a system trained predominantly on one gender’s language patterns may screen inaccurately for the other. Most mental health AI tools also still operate on binary gender assumptions, excluding non-binary, transgender, and gender non-conforming individuals from both the data and the populations these tools are meant to serve (Hafner et al., 2024).

    Why this matters

    These biases translate into delayed diagnoses, inappropriate treatments, and unequal care, every day, for real patients. Understanding these patterns is exactly what equips healthcare professionals to ask better questions, challenge assumptions, and advocate for the patients most at risk, which is precisely what the AEQUITAS training is designed to do.

    References:

    Chang, C.T., Srivathsa N., Bou-Khalil, C., Swaminathan, A., Lunn, M.R., Mishra, K., Koyejo, S,. Daneshjou, R. (2025). Evaluating anti-LGBTQIA+ medical bias in large language models. PLOS Digit Health 4(9): e0001001. https://doi.org/10.1371/journal.pdig.0001001

    Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A.A., Valencia, A., Rementeria, M.J., Chadha, A.S., & Mavridis, N. (2020). Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. npj Digit. Med. 3(81). https://doi.org/10.1038/s41746-020-0288-5

    Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS Digital Health, 3(11), e0000651. https://doi.org/10.1371/journal.pdig.0000651

    Funding research on women’s health. (2024). Nature Reviews Bioengineering, 2, 797–798. https://doi.org/10.1038/s44222-024-00253-7

    Hafner, F.S., Valdivia, A., Rocher, L. 2025. Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory. FAccT ’25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 1677–1695. https://doi.org/10.1145/3715275.3732112

    Karter, A. J., Parker, M. M., Moffet, H. H., & Gilliam, L. K. (2023). Racial and Ethnic Differences in the Association Between Mean Glucose and Hemoglobin A1c. Diabetes Technology & Therapeutics, 25(10), 697–704. https://doi.org/10.1089/dia.2023.0153

    Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological Aspects of Natural Language Use: Our Words, Our Selves. Annual Review of Psychology, 54(1), 547–577. https://doi.org/10.1146/annurev.psych.54.101601.145041

    Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020). Racial Bias in Pulse Oximetry Measurement. New England Journal of Medicine, 383(25), 2477–2478. https://doi.org/10.1056/NEJMc2029240

    Smith, D.T., Mouzon, D.M., & Elliott, M. (2018). Reviewing the assumptions about men’s mental health: An exploration of the gender binary. American Journal of Men’s Health, 12(1), 78–89. https://doi.org/10.1177/1557988316630953

    Bowser, R. (2001). Racial bias in medical treatment. Dick. L. Rev., 105(3), 365.

    Williams, D. R., & Wyatt, R. (2015). Racial Bias in Health Care and Health: Challenges and Opportunities. JAMA, 314(6), 555. https://doi.org/10.1001/jama.2015.9260

  • The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

    The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

    AI health algorithms are increasingly used to predict risk and support treatment decisions, often outperforming humans in diagnostic accuracy. However, they can also underestimate risk for marginalised and underserved patients, delaying the treatment they need. This scoping review analysed 23 sources published between 2020 and 2024 to examine racial and ethnic bias in these algorithms, the role of oversight and stakeholders, and how this bias can be reduced.

    A common explanation for this bias is that algorithms are trained on data in which majority populations are overrepresented, making them less reliable for others. Black, Native American, Asian and Latinx populations are often underrepresented in health datasets. Bias can also come from people: racial stereotypes absorbed during medical training can affect how clinicians assess patients, and these assessments end up in the health records used to train algorithms. The authors also describe a feedback loop: when patients from marginalised groups distrust the health system and use it less, data about them becomes even less representative, and the algorithms less accurate for them.

    To address this, the review highlights the need for continuous monitoring of algorithms once they are in use, in a similar way to how medicines are monitored after approval, and for retraining or withdrawing those that create inequities. It also stresses that simply including or removing race from an algorithm does not solve the problem, and that responsibility should be shared by developers, health systems, clinicians, regulators and patients. The authors conclude that, without this oversight, AI in healthcare risks entrenching health inequities.

    Learn more about this review here: https://doi.org/10.1080/13557858.2024.2422848


    Reference:

    Hussain, S.A., Bresnahan, M., & Zhuang, J. (2025). The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review. Ethnicity & Health, 30(2), 197–214.

  • Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases

    Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases

    AI models are increasingly used in cardiovascular medicine, but there are concerns that they may not work equally well for people of different racial and ethnic backgrounds. This systematic review examined how accurate and clinically applicable AI models for diagnosing heart disease and predicting outcomes are across racial and ethnic groups.

    Of 1,704 references screened, 11 studies were included, covering diagnosis, prognosis and the analysis of heart images. Nine of them (82%) found racial or ethnic bias, while two found no differences between groups. For example, an AI model predicting heart failure from electrocardiograms was less accurate for young Black patients than for young Asian or White patients, and AI tools analysing cardiac MRI scans were more accurate for White people than for people from minority ethnic groups.

    The authors conclude that AI models can perform well for some groups and less accurately for others, and that bias should be addressed at every stage, from data collection to clinical use. They recommend methods to reduce bias, standardised ways of measuring fairness, considering social determinants of health rather than race alone, and greater transparency through the sharing of code and data.

    Learn more about this review here: https://doi.org/10.1016/j.ejrad.2024.111867


    Reference:

    Cau, R., Pisu, F., Suri, J.S., & Saba, L. (2024). Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases. European Journal of Radiology, 111867.

  • Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity

    Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity

    AI has been seen as a tool that could help detect and reduce the racial and ethnic inequities that persist in cardiovascular care. However, AI can also carry and spread biases of its own, which means it must be approached with care. This review looks at the role of AI in cardiology in identifying and addressing these inequities, and in promoting diversity in research, as well as the concerns its use raises and strategies for using it fairly.

    The authors describe how AI has already helped reduce some inequities. In one study, an AI tool identified patients with severe heart valve disease and alerted their doctors, improving follow-up care for patients of all races. Even so, Black patients still faced greater delays in receiving appropriate care than White patients, showing that AI is not a “silver bullet” for eliminating inequities. AI can also perpetuate existing biases, due to limited diversity in training data, biases within the datasets themselves, and inadequate mechanisms for detecting and monitoring bias. For example, an AI model trained mostly on data from White patients to predict heart failure from electrocardiograms performed worse for Black patients, and particularly for Black women.

    To use AI in cardiovascular care in a way that addresses racial and ethnic inequities, the authors stress the need for careful design, starting with meticulous data collection and a thorough review of training data. They also call for rigorous testing and continuous monitoring of algorithms to ensure they perform equally well for all groups, as well as greater diversity in the AI workforce and the involvement of stakeholders.

    Learn more about this review here: https://doi.org/10.1007/s12170-024-00745-6


    Reference:

    Amponsah, D., Thamman, R., Brandt, E., James, C., Spector-Bagdady, K., & Yong, C.M. (2024). Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity. Current Cardiovascular Risk Reports, 18(11), 153–162.

  • Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias

    Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias

    AI has transformed diabetes management, particularly through tools that predict glucose levels and support clinical decisions. However, there are concerns that these technologies may not work equally well for different racial, ethnic and socioeconomic groups. This systematic review assessed how fair AI-based glucose prediction models are, focusing on differences in performance between groups, how fairness is reported, and biases linked to physiological measures.

    Of 1,243 articles initially identified, only 10 met the criteria for inclusion. Just one of them explicitly evaluated whether AI model performance differed by race. Reporting on fairness was rare: only 7% of AI diabetes studies documented ethnic or racial data, and almost none carried out fairness audits. The review also highlights known physiological differences: for the same average glucose level, African American patients tend to have higher HbA1c values than White patients. If HbA1c is used as a reference to train AI models, this difference can be built into them. Yet none of the AI studies reviewed took this into account.

    The authors also note that traditional diabetes risk scores overestimate risk for White individuals and underestimate it for Black individuals, a bias that AI systems trained on these scores could inherit. They conclude that, without systematic efforts to reduce bias, AI-based diabetes technologies risk worsening existing health disparities, and call for future research to prioritise transparency, representativeness and fairness.

    Learn more about this review here: https://doi.org/10.61186/ist.202502.05.04


    Reference:

    Alipour, M., & Alipour, A. (2025). Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias. InfoScience Trends, 2(5), 33–46.

  • Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes

    Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes

    Continuous glucose monitors, small sensors worn on the body that track blood sugar levels throughout the day, are increasingly combined with AI models that predict where glucose levels are heading. Because White people are overrepresented in medical research, these models may perform worse for other groups if they are trained mainly on White patients’ data. This study tested this using data from 205 adults and children with type 1 diabetes in the US, 101 White and 104 Black.

    The researchers trained AI models to predict glucose levels 60 minutes ahead, using datasets with different proportions of White and Black participants, from 0% to 100% White. As the proportion of White participants in the training data increased, prediction errors grew slightly more for Black participants than for White participants. The difference was small but statistically significant.

    When the models were then personalised to each individual, using a technique called transfer learning, this difference became smaller and was no longer statistically significant. The authors note that the differences may not be large enough to matter clinically on their own, but conclude that diverse training data is important for fair prediction models, and that personalisation may help build fairer tools.

    Learn more about this study here: https://doi.org/10.1371/journal.pdig.0000918


    Reference:

    Thomsen, H.B., Li, L.Y., Isaksen, A.A., Lebiecka-Johansen, B., Bour, C., Fagherazzi, G., van Doorn, W.P.T.M., Varga, T.V., & Hulman, A. (2025). Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes. PLOS Digital Health, 4(6), e0000918.

  • Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia

    Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia

    Diabetic retinopathy, damage to the retina caused by diabetes, can lead to vision loss if it is not detected early. This study tested an AI system for diabetic retinopathy screening in real-world conditions, in two endocrinology clinics in Melbourne and three Aboriginal medical services in Western Australia, and also gathered the views of patients and health professionals.

    Of the 236 people screened, 203 were included in the main analysis. Overall, the system identified diabetic retinopathy requiring referral with a sensitivity of 96.9% and a specificity of 87.7%, and over 93% of participants were satisfied with the service. However, specificity was substantially lower among participants recruited from Aboriginal medical services, meaning that more people without the condition were wrongly flagged for referral. The study used clinic location as an approximation of Indigenous status.

    The authors note that the AI system had been trained exclusively on images from Chinese adults, and that differences in retinal pigmentation may affect its accuracy. They also point to other possible factors, such as poorer image quality, older age and other eye conditions. These explanations were not tested in the study.

    Learn more about this study here: https://doi.org/10.1038/s41598-021-94178-5


    Reference:

    Scheetz, J., Koca, D., McGuinness, M., Holloway, E., Tan, Z., Zhu, Z., O’Day, R., Sandhu, S., MacIsaac, R.J., Gilfillan, C., Turner, A., Keel, S., & He, M. (2021). Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia. Scientific Reports, 11, 15808.

  • Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis

    Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis

    AI models developed using data from very similar populations may not work as well for other groups, and could reflect or even worsen existing racial and ethnic disparities in health care. This study tested whether this was the case for an AI model that uses electrocardiograms (ECGs) to detect a weakened heart pumping function, known as low left ventricular ejection fraction.

    The model had been developed with data from nearly 45,000 patients, 96.2% of whom were non-Hispanic White. When tested on a separate group of almost 53,000 patients, it performed consistently well across all the racial and ethnic groups analysed: non-Hispanic White, Asian, Black/African American, Hispanic/Latino and American Indian/Native Alaskan. A separate AI model was able to identify patients’ racial group from the ECG to some extent, showing that ECG characteristics do vary by race, but this did not affect the first model’s performance.

    The authors conclude that, in this case, racial differences in the ECG did not affect the model’s ability to detect the condition. Even so, they recommend that the performance of all new AI tools in medicine be reported across different ethnic, racial, age and sex groups, to ensure they are used responsibly.

    Learn more about this study here: https://doi.org/10.1161/CIRCEP.119.007988


    Reference:

    Noseworthy, P.A., Attia, Z.I., Brewer, L.C., Hayes, S.N., Yao, X., Kapa, S., Friedman, P.A., & Lopez-Jimenez, F. (2020). Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis. Circulation: Arrhythmia and Electrophysiology, 13(3), e007988.