Tag: AI in Healthcare

Artificial intelligence bias in the prediction and detection of cardiovascular disease

AI can help identify people at risk of cardiovascular disease, detect it early and allow timely intervention. However, bias can arise at any step in the development, validation and use of these algorithms, causing them to perform poorly for historically marginalised groups. This article discusses the sources and consequences of AI bias in predicting and detecting cardiovascular disease, and presents strategies to reduce it.

Bias can come from training data that does not represent the population a tool is meant for, from measurements or devices that work less well for some groups, and from leaving out important factors such as social determinants of health. It can also reflect inequalities in care: because women and ethnic minority patients are more likely to have their cardiovascular disease missed or diagnosed late, algorithms trained on this data may be less accurate at detecting it in these groups. Unequal access to digital technologies can further limit how well some groups are represented in the data. The authors give several examples of AI models that performed worse for women, Black patients or older adults.

To reduce bias, the authors propose an AI health equity framework covering the whole lifecycle of an algorithm: diverse research teams, early involvement of patients, representative and balanced data, the inclusion of social determinants of health, input from clinicians, validation across different groups, transparent reporting of results and monitoring after implementation. They conclude that, without these measures, AI bias could worsen existing healthcare inequities and miss opportunities to predict and detect cardiovascular disease.

Learn more about this article here: https://doi.org/10.1038/s44325-024-00031-9

Reference:

Mihan, A., Pandey, A., & Van Spall, H.G.C. (2024). Artificial intelligence bias in the prediction and detection of cardiovascular disease. npj Cardiovascular Health, 1, 31.
The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

AI health algorithms are increasingly used to predict risk and support treatment decisions, often outperforming humans in diagnostic accuracy. However, they can also underestimate risk for marginalised and underserved patients, delaying the treatment they need. This scoping review analysed 23 sources published between 2020 and 2024 to examine racial and ethnic bias in these algorithms, the role of oversight and stakeholders, and how this bias can be reduced.

A common explanation for this bias is that algorithms are trained on data in which majority populations are overrepresented, making them less reliable for others. Black, Native American, Asian and Latinx populations are often underrepresented in health datasets. Bias can also come from people: racial stereotypes absorbed during medical training can affect how clinicians assess patients, and these assessments end up in the health records used to train algorithms. The authors also describe a feedback loop: when patients from marginalised groups distrust the health system and use it less, data about them becomes even less representative, and the algorithms less accurate for them.

To address this, the review highlights the need for continuous monitoring of algorithms once they are in use, in a similar way to how medicines are monitored after approval, and for retraining or withdrawing those that create inequities. It also stresses that simply including or removing race from an algorithm does not solve the problem, and that responsibility should be shared by developers, health systems, clinicians, regulators and patients. The authors conclude that, without this oversight, AI in healthcare risks entrenching health inequities.

Learn more about this review here: https://doi.org/10.1080/13557858.2024.2422848

Reference:

Hussain, S.A., Bresnahan, M., & Zhuang, J. (2025). The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review. Ethnicity & Health, 30(2), 197–214.
Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases

AI models are increasingly used in cardiovascular medicine, but there are concerns that they may not work equally well for people of different racial and ethnic backgrounds. This systematic review examined how accurate and clinically applicable AI models for diagnosing heart disease and predicting outcomes are across racial and ethnic groups.

Of 1,704 references screened, 11 studies were included, covering diagnosis, prognosis and the analysis of heart images. Nine of them (82%) found racial or ethnic bias, while two found no differences between groups. For example, an AI model predicting heart failure from electrocardiograms was less accurate for young Black patients than for young Asian or White patients, and AI tools analysing cardiac MRI scans were more accurate for White people than for people from minority ethnic groups.

The authors conclude that AI models can perform well for some groups and less accurately for others, and that bias should be addressed at every stage, from data collection to clinical use. They recommend methods to reduce bias, standardised ways of measuring fairness, considering social determinants of health rather than race alone, and greater transparency through the sharing of code and data.

Learn more about this review here: https://doi.org/10.1016/j.ejrad.2024.111867

Reference:

Cau, R., Pisu, F., Suri, J.S., & Saba, L. (2024). Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases. European Journal of Radiology, 111867.
Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity

AI has been seen as a tool that could help detect and reduce the racial and ethnic inequities that persist in cardiovascular care. However, AI can also carry and spread biases of its own, which means it must be approached with care. This review looks at the role of AI in cardiology in identifying and addressing these inequities, and in promoting diversity in research, as well as the concerns its use raises and strategies for using it fairly.

The authors describe how AI has already helped reduce some inequities. In one study, an AI tool identified patients with severe heart valve disease and alerted their doctors, improving follow-up care for patients of all races. Even so, Black patients still faced greater delays in receiving appropriate care than White patients, showing that AI is not a “silver bullet” for eliminating inequities. AI can also perpetuate existing biases, due to limited diversity in training data, biases within the datasets themselves, and inadequate mechanisms for detecting and monitoring bias. For example, an AI model trained mostly on data from White patients to predict heart failure from electrocardiograms performed worse for Black patients, and particularly for Black women.

To use AI in cardiovascular care in a way that addresses racial and ethnic inequities, the authors stress the need for careful design, starting with meticulous data collection and a thorough review of training data. They also call for rigorous testing and continuous monitoring of algorithms to ensure they perform equally well for all groups, as well as greater diversity in the AI workforce and the involvement of stakeholders.

Learn more about this review here: https://doi.org/10.1007/s12170-024-00745-6

Reference:

Amponsah, D., Thamman, R., Brandt, E., James, C., Spector-Bagdady, K., & Yong, C.M. (2024). Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity. Current Cardiovascular Risk Reports, 18(11), 153–162.
Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias

AI has transformed diabetes management, particularly through tools that predict glucose levels and support clinical decisions. However, there are concerns that these technologies may not work equally well for different racial, ethnic and socioeconomic groups. This systematic review assessed how fair AI-based glucose prediction models are, focusing on differences in performance between groups, how fairness is reported, and biases linked to physiological measures.

Of 1,243 articles initially identified, only 10 met the criteria for inclusion. Just one of them explicitly evaluated whether AI model performance differed by race. Reporting on fairness was rare: only 7% of AI diabetes studies documented ethnic or racial data, and almost none carried out fairness audits. The review also highlights known physiological differences: for the same average glucose level, African American patients tend to have higher HbA1c values than White patients. If HbA1c is used as a reference to train AI models, this difference can be built into them. Yet none of the AI studies reviewed took this into account.

The authors also note that traditional diabetes risk scores overestimate risk for White individuals and underestimate it for Black individuals, a bias that AI systems trained on these scores could inherit. They conclude that, without systematic efforts to reduce bias, AI-based diabetes technologies risk worsening existing health disparities, and call for future research to prioritise transparency, representativeness and fairness.

Learn more about this review here: https://doi.org/10.61186/ist.202502.05.04

Reference:

Alipour, M., & Alipour, A. (2025). Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias. InfoScience Trends, 2(5), 33–46.
Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions

Machine learning models can help predict which patients with diabetes are likely to be readmitted to hospital, but they need to work fairly for all patients. This study compared several of these models, including deep learning, generalised linear models, gradient boosting machines and naive Bayes, assessing not only their accuracy but also how fair their predictions were across age, gender and race.

The gradient boosting model performed best overall, with an accuracy of 82.2% and consistently low error rates for most racial and gender groups. However, across several of the models, African American and Hispanic patients had higher rates of false positives, meaning they were more likely to be wrongly predicted to be readmitted. No single model performed best for every group: some worked better for certain groups of patients and others for other groups.

The authors conclude that there is no one-size-fits-all solution, and that models need to be chosen and adjusted carefully to ensure they are both accurate and fair for all patients. According to them, fair algorithms can help reduce disparities and improve outcomes for patients with diabetes from all backgrounds.

Learn more about this study here: https://doi.org/10.1109/ICHI61247.2024.00105

Reference:

Al-Zanbouri, Z., Sharma, G., & Raza, S. (2024). Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions. In 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI) (pp. 660–669). IEEE.
Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study

Large language models such as GPT-4 are seen as promising tools in health care, from automating administrative tasks to supporting clinical decisions. However, they also risk perpetuating biases and giving incorrect diagnoses, with a direct impact on patient care. This study assessed whether GPT-4 shows racial and gender biases in four possible uses: medical education, diagnostic reasoning, treatment planning and assessments of patients.

The researchers used clinical cases from medical education resources and from published research on implicit bias in health care. When asked to create clinical cases, GPT-4 did not reflect the real demographic diversity of medical conditions, and consistently produced cases that stereotyped how conditions present in different groups. For the same clinical cases, its lists of possible diagnoses were more likely to include diagnoses that stereotype certain races, ethnicities and genders.

The treatment plans and assessments it produced also varied with patients’ demographic characteristics, including recommendations for more expensive procedures and differences in how patients were perceived. The authors conclude that tools like GPT-4 urgently need thorough and transparent bias assessments for each intended use before they are introduced into clinical care.

Learn more about this study here: https://doi.org/10.1016/S2589-7500(23)00225-X

Reference:

Zack, T., Lehman, E., Suzgun, M., Rodriguez, J.A., Celi, L.A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D.W., Abdulnour, R.-E.E., Butte, A.J., & Alsentzer, E. (2024). Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health, 6(1), e12–e22.
Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes

Continuous glucose monitors, small sensors worn on the body that track blood sugar levels throughout the day, are increasingly combined with AI models that predict where glucose levels are heading. Because White people are overrepresented in medical research, these models may perform worse for other groups if they are trained mainly on White patients’ data. This study tested this using data from 205 adults and children with type 1 diabetes in the US, 101 White and 104 Black.

The researchers trained AI models to predict glucose levels 60 minutes ahead, using datasets with different proportions of White and Black participants, from 0% to 100% White. As the proportion of White participants in the training data increased, prediction errors grew slightly more for Black participants than for White participants. The difference was small but statistically significant.

When the models were then personalised to each individual, using a technique called transfer learning, this difference became smaller and was no longer statistically significant. The authors note that the differences may not be large enough to matter clinically on their own, but conclude that diverse training data is important for fair prediction models, and that personalisation may help build fairer tools.

Learn more about this study here: https://doi.org/10.1371/journal.pdig.0000918

Reference:

Thomsen, H.B., Li, L.Y., Isaksen, A.A., Lebiecka-Johansen, B., Bour, C., Fagherazzi, G., van Doorn, W.P.T.M., Varga, T.V., & Hulman, A. (2025). Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes. PLOS Digital Health, 4(6), e0000918.
Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia

Diabetic retinopathy, damage to the retina caused by diabetes, can lead to vision loss if it is not detected early. This study tested an AI system for diabetic retinopathy screening in real-world conditions, in two endocrinology clinics in Melbourne and three Aboriginal medical services in Western Australia, and also gathered the views of patients and health professionals.

Of the 236 people screened, 203 were included in the main analysis. Overall, the system identified diabetic retinopathy requiring referral with a sensitivity of 96.9% and a specificity of 87.7%, and over 93% of participants were satisfied with the service. However, specificity was substantially lower among participants recruited from Aboriginal medical services, meaning that more people without the condition were wrongly flagged for referral. The study used clinic location as an approximation of Indigenous status.

The authors note that the AI system had been trained exclusively on images from Chinese adults, and that differences in retinal pigmentation may affect its accuracy. They also point to other possible factors, such as poorer image quality, older age and other eye conditions. These explanations were not tested in the study.

Learn more about this study here: https://doi.org/10.1038/s41598-021-94178-5

Reference:

Scheetz, J., Koca, D., McGuinness, M., Holloway, E., Tan, Z., Zhu, Z., O’Day, R., Sandhu, S., MacIsaac, R.J., Gilfillan, C., Turner, A., Keel, S., & He, M. (2021). Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia. Scientific Reports, 11, 15808.
Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis

AI models developed using data from very similar populations may not work as well for other groups, and could reflect or even worsen existing racial and ethnic disparities in health care. This study tested whether this was the case for an AI model that uses electrocardiograms (ECGs) to detect a weakened heart pumping function, known as low left ventricular ejection fraction.

The model had been developed with data from nearly 45,000 patients, 96.2% of whom were non-Hispanic White. When tested on a separate group of almost 53,000 patients, it performed consistently well across all the racial and ethnic groups analysed: non-Hispanic White, Asian, Black/African American, Hispanic/Latino and American Indian/Native Alaskan. A separate AI model was able to identify patients’ racial group from the ECG to some extent, showing that ECG characteristics do vary by race, but this did not affect the first model’s performance.

The authors conclude that, in this case, racial differences in the ECG did not affect the model’s ability to detect the condition. Even so, they recommend that the performance of all new AI tools in medicine be reported across different ethnic, racial, age and sex groups, to ensure they are used responsibly.

Learn more about this study here: https://doi.org/10.1161/CIRCEP.119.007988

Reference:

Noseworthy, P.A., Attia, Z.I., Brewer, L.C., Hayes, S.N., Yao, X., Kapa, S., Friedman, P.A., & Lopez-Jimenez, F. (2020). Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis. Circulation: Arrhythmia and Electrophysiology, 13(3), e007988.
Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases

Electrocardiography (ECG) enhanced by artificial intelligence is emerging as a promising tool for diagnosing and managing cardiovascular disease. This review explains how AI can detect subtle patterns in the ECG that human interpreters may miss, helping to identify problems such as arrhythmias and heart failure more quickly and consistently.

However, the authors also highlight important limitations, including systematic bias. When training datasets do not adequately represent different ages, sexes and racial groups, models can perform worse for those who are underrepresented, since ECG patterns vary with age, physiological differences between the sexes and genetic factors. To make these tools safe and fair in clinical practice, the review calls for more diverse datasets, bias mitigation strategies and continuous validation across populations.

Learn more about this review here: https://doi.org/10.1016/j.jelectrocard.2024.01.006

Reference:

Muzammil, M.A., Javid, S., Afridi, A.K., Siddineni, R., Shahabi, M., Haseeb, M., Fariha, F., Kumar, S., Zaveri, S., & Nashwan, A.J. (2024). Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases. Journal of Electrocardiology, 83, 30–4