Category: Research

Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models

Large multi-modal models (LMMs) are a fast-growing type of generative AI that can take in different types of data, such as text, images and video, and produce varied responses. The World Health Organization (WHO) has issued guidance on their use in health, with over 40 recommendations for governments, technology companies and healthcare providers. The guidance covers five broad uses: diagnosis and clinical care, patient-guided use such as checking symptoms, administrative tasks, medical and nursing education, and scientific research and drug development.

The WHO warns that LMMs can produce false, inaccurate, biased or incomplete answers, which could harm people making health decisions. They may be trained on poor-quality or biased data, including by race, ethnicity, ancestry, sex, gender identity or age. Because most of the data and research used to train them comes from high-income countries, their answers may also be biased towards a high-income perspective. In addition, health professionals may place too much trust in these tools and overlook their errors, a problem known as automation bias.

Among its recommendations, the WHO calls on governments to pass laws and regulations to ensure LMMs meet ethical and human rights standards, assign regulatory agencies to assess and approve them, and require independent audits and impact assessments after they are released. Developers should involve diverse stakeholders, including patients and health professionals, from the early stages of development, and design LMMs to carry out well-defined tasks accurately and reliably.

Learn more here: https://www.who.int/publications/i/item/9789240084759

Reference:

World Health Organization. (2024). Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models.
AI-based diabetes care: risk prediction models and implementation concerns

AI is increasingly being used to predict a person’s risk of developing type 2 diabetes, which could help delay or prevent the disease through earlier, more personalised prevention. This editorial discusses a scoping review of 40 studies on AI models for diabetes risk prediction. It found that models combining electronic health records with other types of data, such as genetic or medical imaging data, performed better than models using a single type of data.

However, the authors warn that these results should be approached with caution. Many of the models showed a pronounced imbalance in gender, ethnicity and age in the data used to train them, only five were tested on external data, and most were not evaluated across different demographic groups. Since Black and Mexican Americans have a higher prevalence of diabetes than White Americans, leaving certain groups out of training data could worsen health inequities for populations already at greater risk.

To address this, the authors call for mandatory representation criteria for the approval and adoption of these models, tools to assess their risk of bias, and external validation. They argue that AI cannot be adopted on a large scale in healthcare until problems with data quality and bias are addressed, and that this requires collaboration between clinicians, researchers, policymakers and entrepreneurs.

Learn more about this article here: https://doi.org/10.1038/s41746-024-01034-7

Reference:

Wang, S. C. Y., Nickel, G., Venkatesh, K. P., Raza, M. M., & Kvedar, J. C. (2024). AI-based diabetes care: Risk prediction models and implementation concerns. npj Digital Medicine, 7, 36.
UnitedHealth lawsuit over alleged AI-driven denials of care moves forward

In November 2023, the families of two deceased patients filed a proposed class action lawsuit in the US against UnitedHealth Group, one of the country’s largest health insurers, and its subsidiary naviHealth. They allege the companies used an AI tool called nH Predict to wrongfully deny elderly patients in Medicare Advantage plans coverage for care after a hospital stay, such as physical therapy or skilled nursing care. According to the complaint, Gene Lokken, 91, was covered for only 19 days of care after a fall, despite his doctors recommending more, and his family paid around $150,000 until his death.

The tool uses information such as age, living situation and physical function to compare patients with millions of past cases and estimate how long they should need care. The lawsuit alleges that the company applied these estimates even when they went against doctors’ recommendations, and that staff were pressured to keep patients’ stays close to the tool’s predictions. Over 90% of the denials that were appealed were reversed, but only about 0.2% of patients ever appealed. UnitedHealth denies the claims, saying the tool is used as a guide and not to make coverage decisions.

In February 2025, a federal court in Minnesota allowed part of the case to proceed, and in March 2026 a judge ordered UnitedHealth to hand over a wide range of documents on how the tool was implemented and used. The case is still ongoing and raises important questions about transparency and accountability when algorithms influence decisions about patients’ care.

Learn more here: https://www.statnews.com/2023/11/14/unitedhealth-class-action-lawsuit-algorithm-medicare-advantage/
Read the latest update here: https://www.yahoo.com/news/us/articles/lawsuit-says-unitedhealth-used-ai-173017952.html

References:

Ross, C., & Herman, B. (2023, November 14). UnitedHealth faces class action lawsuit over algorithmic care denials in Medicare Advantage plans. STAT.

Becker’s Payer Issues. (2026, March 11). Judge orders UnitedHealth to hand over documents in AI coverage denial case.

Barrientos, A. (2026, September 18). Lawsuit says UnitedHealth used AI with a 90% error rate to deny care. Yahoo News.
Fairness and bias correction in machine learning for depression prediction across four study populations

A study found that standard machine learning approaches often exhibit biased behaviours in predicting depression across different populations. It also demonstrated that both standard and novel post-hoc-bias mitigation techniques can effectively reduce unfair bias, though no single model achieves equality of outcomes.

The biases that were identified risk reinforcing structural inequalities in mental healthcare, particularly affecting underserved populations. This underscores the importance of analyzing fairness during model selection and transparently reporting the impact of debiasing interventions to ensure equitable healthcare applications.

Learn more about this study here: https://doi.org/10.1038/s41598-024-58427-7

Reference

Dang, V.N., Cascarano, A., Mulder, R.H. et al. Fairness and bias correction in machine learning for depression prediction across four study populations. Sci Rep 14, 7848 (2024).
Predicting cardiovascular events from routine mammograms using machine learning

Cardiovascular risk is often underassessed in women, and many women have screening mammograms in midlife, just when their risk of cardiovascular disease starts to rise. Researchers led by The George Institute for Global Health developed an AI model that predicts a woman’s risk of a major cardiovascular event using only routine mammogram images and her age.

The model was developed using data from 49,196 women in Australia, who were followed for a median of 8.8 years. It analyses a wide range of features in breast tissue, not just calcium deposits in the arteries, and performed as well as standard risk calculators that require medical history and other clinical information. Adding clinical factors improved its performance only slightly.

Because many women already have regular mammograms for breast cancer screening, this approach could offer a “two-for-one” check, without the need for extra appointments or medical records. The researchers note some limitations, such as differences between mammography machines, and say further work is needed before the tool can be used in practice.

Learn more about this study here: https://doi.org/10.1136/heartjnl-2025-325705
Read the news here: https://www.georgeinstitute.org/news-and-media/news/new-ai-algorithm-uses-mammograms-to-accurately-predict-cardiovascular-risk-in-women

References:

Barraclough, J. Y., Gandomkar, Z., Fletcher, R. A., Barbieri, S., Kuo, N. I.-H., Rodgers, A., Douglas, K., Poppe, K. K., Woodward, M., Gallego Luxan, B., Neal, B., Jorm, L., Brennan, P., & Arnott, C. (2026). Predicting cardiovascular events from routine mammograms using machine learning. Heart, 112(5), 261–269.

Wall, T. (2025, September 17). New AI algorithm uses mammograms to accurately predict cardiovascular risk in women. The George Institute for Global Health.
Can AI fight sex and gender bias in healthcare?

AI is transforming medicine, but it is mostly developed by men and trained on data that prioritises men’s bodies and health needs. In this commentary, researchers from the George Institute for Global Health argue that this embeds sex and gender biases that can put women and nonbinary patients at risk. For example, natural language processing models used in psychiatry have shown significant gender biases, and tools that screen for suicide risk may make mistakes if trained mainly on text written by men, since men and women express distress differently.

The authors also argue that AI could help reduce these inequities if sex and gender are considered deliberately. One example is the GRACE heart attack risk score, which was updated using AI models that analyse male and female data separately to account for sex-specific characteristics of the disease. Another is a chatbot developed by the George Institute to give guideline-based pregnancy advice in rural India, created together with community health workers, clinicians and women.

They call for sex and gender bias to be actively addressed in the data used to train AI, for tools to be co-designed with the communities they serve, for work on bias to go beyond the English language, and for sex and gender to be considered at every stage of research and clinical use.

Learn more here: https://www.unsw.edu.au/newsroom/news/2024/10/can-ai-fight-sex-and-gender-bias-in-healthcare-

Reference:

Haupt, S., Graham, B., & Hirst, J. (2024, October 21). Can AI fight sex and gender bias in healthcare? 360info.
Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations

AI systems now match expert performance in many medical imaging tasks, but there is growing concern that they may reflect and amplify human bias. This is especially worrying in the case of underdiagnosis, when AI wrongly labels a person who has a disease as healthy, which can delay their access to care. This study examined underdiagnosis by AI tools that analyse chest X-rays, using three large US datasets and a combination of all three, with more than 700,000 images from almost 130,000 patients.

The AI tools consistently and selectively underdiagnosed under-served groups: female patients, patients under 20, Black patients, Hispanic patients and patients with Medicaid, a public health insurance for people with low incomes. Underdiagnosis was even higher for patients who belonged to more than one of these groups, such as Hispanic female patients, and for young patients who were also female, Black or on Medicaid.

The authors warn that using AI tools with these biases could worsen existing inequalities in care and lead to unequal access to treatment, raising ethical concerns about using these models in clinical practice.

Learn more about this study here: https://doi.org/10.1038/s41591-021-01595-0

Reference:

Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182.
New-Onset Illness Anxiety Disorder After Helicobacter Pylori Infection: A Case Report

In this interesting case study from 2024, a 21-year-old woman developed illness anxiety disorder (IAD), formerly known as hypochondria, following a Helicobacter pylori infection.

In the case, even after the H. pylori infection was successfully treated, the patient’s anxiety persisted, affecting her daily life and leading to frequent medical visits and avoidance behaviors.

While the study also explored potential psychosocial risk factors that may contribute to developing IAD and discussed both pharmacological and psychological treatment options, this case underscores how physical health events, like infections, can sometimes trigger or exacerbate mental health conditions, emphasizing the importance of integrated care for both gut and mental health.

Learn more about this study here: https://doi.org/10.7759/cureus.52613

Reference

Labban S A, Murshid L, Yousef Alhazmi A, et al. (January 20, 2024) New-Onset Illness Anxiety Disorder After Helicobacter Pylori Infection: A Case Report. Cureus 16(1): e52613.
Bias Mitigation in Primary Health Care Artificial Intelligence Models: Scoping Review

AI models in primary care can help identify quickly and accurately the people who should receive care, but they risk perpetuating or amplifying biases against diverse groups. This scoping review looked at the strategies and methods used to reduce bias in primary care AI models, which groups were considered, and how these strategies affected both bias and model performance.

Of the 17 studies included, race or ethnicity was the attribute most often examined (12 studies), followed by sex or gender (10 studies), usually defined only as male or female. The approaches to reducing bias fell into four groups: modifying existing models or datasets, sourcing data from electronic health records, developing tools with human input (“human-in-the-loop”), and identifying ethical principles to guide decisions. Adjusting the data before training the model, for example by relabelling or reweighting it, and analysing doctors’ written notes with language-processing tools showed the greatest potential. Other methods sometimes increased errors or made the models less well calibrated.

The authors conclude that bias appears easier to reduce when data are open source, when multiple stakeholders are involved, and when bias is addressed before the model is trained. They call for further studies that include a wider range of groups, such as Indigenous peoples in Canada.

Learn more about this review here: https://doi.org/10.2196/60269

Reference:

Sasseville, M., Ouellet, S., Rhéaume, C., Sahlia, M., Couture, V., Després, P., Paquette, J.-S., Darmon, D., Bergeron, F., & Gagnon, M.-P. (2025). Bias mitigation in primary health care artificial intelligence models: Scoping review. Journal of Medical Internet Research, 27, e60269.
Addressing AI Algorithmic Bias in Health Care

Biased AI algorithms can result in some groups of patients not receiving appropriate care, potentially leading to significant harm. In one example, an algorithm designed to identify patients who needed extra care used healthcare costs to estimate their needs. Because Black patients face barriers in accessing care, less was spent on them, and the algorithm may have more than halved the number of Black patients identified for extra care.

In April 2024, the US Department of Health and Human Services issued a rule that explicitly prohibits discriminatory outcomes from tools that support clinical decisions, including AI. Health care facilities now have to identify and reduce the risk of discrimination when using these tools. However, the authors argue that guidelines, resources and shared responsibilities are still lacking. Algorithms need to be monitored locally over time, which may be possible for well-resourced facilities but is a major challenge for under-resourced ones. These facilities may either avoid AI tools that could benefit their communities, or adopt tools that produce biased results without being aware of it.

The authors propose a model of shared responsibility between health care facilities, AI developers and regulators. Developers should be transparent, showing that diverse data were used, describing how they tested their algorithms for bias, and disclosing any limitations, with this information stored in a public central repository. Existing federal requirements for AI transparency should also be aligned with the new anti-discrimination rule.

Learn more about this article here: https://doi.org/10.1001/jama.2024.13486

Reference:

Ratwani, R. M., Sutton, K., & Galarraga, J. E. (2024). Addressing AI algorithmic bias in health care. JAMA, 332(13), 1051–1052.
Enhancing Fairness and Accuracy in Diagnosing Type 2 Diabetes in Young Adult Population

Type 2 diabetes is most common in older people, but it is becoming more frequent among young adults, who face a higher risk of early and severe complications. Missing the diagnosis in this group can have serious consequences for their health. This study found that common machine learning models show what the authors call “digital ageism”: they miss diabetes more often in young adults aged 30 to 44 than in other age groups.

The researchers used data from a large US health survey with more than 400,000 participants, in which young adults are a minority of the people with diabetes. Instead of using a single model for everyone, they trained a separate model for each five-year age band (30–34, 35–39 and 40–44), adjusting the data balance within each group.

This approach improved the detection of diabetes in young adults by 26% to 40%, and performed better than seven commonly used techniques for balancing data. The authors note that a model trained for one age group did not work equally well for groups defined by gender or ethnicity, but that the same approach could be applied to other underrepresented groups.

Learn more about this study here: https://doi.org/10.1109/JBHI.2025.3616312

Reference:

Pias, T. S., Su, Y., Tang, X., Wang, H., Faghani, S., & Yao, D. (2026). Enhancing fairness and accuracy in diagnosing type 2 diabetes in young adult population. IEEE Journal of Biomedical and Health Informatics, 30(4), 3321–3330.
Inteligência Artificial no setor da saúde: desafios jurídicos e regulação

Artificial Intelligence in the health sector: legal challenges and regulation

AI can significantly improve the quality of health services, but its use also raises well-founded concerns. This paper, published by the Office for Strategy and Studies (GEE) of the Portuguese Ministry of Economy, analyses the current landscape of AI in the Portuguese health sector and proposes public policy solutions for Portugal.

The author focuses on three main challenges: the opacity of AI systems, which can undermine patients’ informed consent; bias and discrimination; and the protection of health data and privacy. On bias, she explains that it comes mostly from the data used to train AI, which can reflect existing social prejudices and be skewed by factors such as age, sex, race and socioeconomic background. The paper also analyses how these systems fit within the EU AI Act, with particular attention to the obligation of human oversight.

To reduce these risks, the author proposes measures such as ethical requirements in the development and choice of AI systems, impact assessments and independent data audits before approval, periodic evaluation with a mechanism for reporting failures, and training for health professionals so that they can understand AI and recognise its errors.

Learn more about this paper here: https://EconPapers.repec.org/RePEc:mde:wpaper:187

Reference:

Paixão Cansado, M. (2024). Inteligência Artificial no setor da saúde: desafios jurídicos e regulação (GEE Paper No. 187). Gabinete de Estratégia e Estudos, Ministério da Economia.
Sociodemographic biases in medical decision making by large language models

Large language models (LLMs), such as the AI behind popular chatbots, show promise in healthcare, but they may make recommendations influenced by patients’ sociodemographic characteristics. Researchers tested nine LLMs on 1,000 emergency department cases, half real and half synthetic. Each case was presented in 32 versions: one for each of 31 sociodemographic groups, plus a control version with no sociodemographic details. The clinical details were always the same. In total, they analysed more than 1.7 million AI-generated recommendations.

Compared with both a physician-defined baseline and the control cases, cases labelled as Black or unhoused or as LGBTQIA+ were more often directed towards urgent care, invasive interventions or mental health evaluations. For example, certain cases from LGBTQIA+ subgroups were recommended mental health assessments around six to seven times more often than clinically indicated. Cases labelled as high-income were more often recommended advanced imaging tests, such as CT or MRI, while low- and middle-income cases were often limited to basic tests or none at all.

These differences were not supported by clinical reasoning or guidelines and may reflect bias in the models, which could lead to health disparities. As they were found in both proprietary and open-source models, the authors stress the need for robust ways to evaluate and reduce bias, so that AI-driven medical advice remains fair and centred on patients.

Learn more about this study here: https://doi.org/10.1038/s41591-025-03626-6

Reference:

Omar, M., Soffer, S., Agbareia, R., Bragazzi, N. L., Apakama, D. U., Horowitz, C. R., Charney, A. W., Freeman, R., Kummer, B., Glicksberg, B. S., Nadkarni, G. N., & Klang, E. (2025). Sociodemographic biases in medical decision making by large language models. Nature Medicine, 31, 1873–1881.
Addressing bias in big data and AI for health care: A call for open science

Most AI tools in healthcare learn from large datasets, but many groups of people have long been missing or misrepresented in medical data. When AI is trained on data that does not reflect the whole population, it can reinforce existing biases and lead to misdiagnoses and even fatal outcomes, particularly for gender and ethnic minorities.

The authors give several examples. AI tools for classifying skin lesions are often trained on datasets in which only about 5% to 10% of images come from Black patients, and when tested on images of Black patients they reach about half the accuracy their creators claimed. This matters because Black patients have the highest mortality from melanoma. In cardiology, heart attacks are often misdiagnosed in women, yet some models that predict heart attacks were trained mostly on data from men. Many datasets also record gender only as male or female, making it difficult to build AI that includes LGBTQ+ people or to test it for bias.

To move towards fairer AI, the authors propose open science practices: involving patients and communities in the development of AI, sharing data responsibly while protecting privacy, using inclusive data standards, and sharing code. Sharing code and algorithms allows researchers around the world to test whether a tool developed with data from one population also works for patients elsewhere.

Learn more about this article here: https://doi.org/10.1016/j.patter.2021.100347

Reference:

Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021). Addressing bias in big data and AI for health care: A call for open science. Patterns, 2(10), 100347.
Exploring Gender Bias in AI for Personalized Medicine: Focus Group Study With Trans Community Members

Precision medicine aims to tailor healthcare to each person, but trans people are often left out of this research. Little is known about how AI tools in healthcare could be adapted to their specific health needs. This study explored the challenges, obstacles and possible solutions for using AI in personalised medicine for trans people.

The researchers worked together with trans community representatives to co-design the study and its data collection tools, and held three focus groups with 16 trans adults. Participants raised concerns about data privacy, algorithmic bias and the lack of health data that reflects trans experiences. They also worried that AI built on data models that assume everyone is cisgender could lead to misdiagnosis or inappropriate treatment.

At the same time, participants saw opportunities for AI to improve health outcomes. They proposed community-led data collection, more transparent algorithms, including trans-specific health markers in datasets, involving community members in AI development, and ethical frameworks that respect gender diversity. The authors conclude that precision medicine needs a trans-inclusive approach, with AI designed around the health needs and lived realities of trans people.

Learn more about this study here: https://doi.org/10.2196/72325

Reference:

Buslón, N., Cirillo, D., Rios, O., & Perera del Rosario, S. (2025). Exploring gender bias in AI for personalized medicine: Focus group study with trans community members. Journal of Medical Internet Research, 27, e72325.
Mitigating the risk of artificial intelligence bias in cardiovascular care

Digital health technologies produce large amounts of data that can be used to train AI, which is transforming cardiovascular care. However, AI can become biased when the data used to train it comes from a very similar group of people, or from healthcare processes that are themselves unequal. Bias can also be introduced while algorithms are developed, tested and put into practice, and even after they are already in use.

Biased algorithms can lead to missed diagnoses, misclassified disease, inaccurate risk predictions and inappropriate treatment recommendations, and these harms can fall disproportionately on marginalised groups. This paper, part of a series in The Lancet Digital Health, gives an overview of how AI is used in cardiovascular care, explains where bias can arise at each stage of an algorithm’s development, and presents examples of harm caused by biased algorithms.

The authors also propose strategies to reduce bias during the training, testing and implementation of algorithms, so that “all those at risk for or living with cardiovascular disease might benefit equally from AI.”

Learn more about this article here: https://doi.org/10.1016/S2589-7500(24)00155-9

Reference:

Mihan, A., Pandey, A., & Van Spall, H. G. C. (2024). Mitigating the risk of artificial intelligence bias in cardiovascular care. The Lancet Digital Health, 6(10), e749–e754.
Research Frameworks towards Health Equity

AI has great potential in healthcare, but existing algorithms risk perpetuating and worsening existing biases, particularly racial and ethnic disparities. This master’s thesis, from the Faculty of Engineering of the University of Porto and developed with researchers at MIT, looks at these issues through a health equity lens. It reviews challenges such as sharing clinical data, selection bias, biased measurements from medical devices, and common pitfalls in developing AI.

The thesis then proposes three research frameworks, tested with a large intensive care database. The first found that patients with sepsis from racial and ethnic minority groups were less likely than White patients to receive mechanical ventilation and medication to raise blood pressure. The second found no substantial differences between groups in the effect of these treatments on outcomes such as mortality. The third focused on pulse oximeters, which are less accurate in people with darker skin, and showed that machine learning can be used to correct their readings, although this still needs to be validated in other databases.

Distinguished in the 2023 IEEE Portugal Section Outstanding MSc Thesis Awards, this work highlights the need to place health equity at the forefront of health research.

Learn more about this thesis here: https://repositorio-aberto.up.pt/handle/10216/150994

Reference:

Matos, J. (2023). Research frameworks towards health equity [Master’s thesis, Faculdade de Engenharia da Universidade do Porto]. Repositório Aberto da Universidade do Porto.
Value of serum brain-derived neurotrophic factor and glial fibrillary acidic protein for detecting depression in patients with Helicobacter pylori infection

Depression is often associated with Helicobacter pylori and the success of its treatment.

In this recent research paper, it was shown that people infected with these bacteria have lower levels of BDNF (a brain health marker considered a promising biomarker of depression) and higher levels of GFAP (a protein known as a marker of astroglia pathology in depression) in their blood, a change that is strongly related to depression symptoms. The researchers also found that a combined marker including BDNF, GFAP, gut hormone levels, and gastrointestinal symptoms scores was able to accurately predict depression in infected patients.

These findings suggest that these markers could be used to detect depression in people with H. pylori infection.

Learn more about this study here: https://doi.org/10.1016/j.neulet.2024.137687

Reference

Zhao, E., Yu, Q., Wang, M., Wang, Z., Jiang, B., Ma, X., Zhou, B., Dai, Q., Li, J., Wang, S., Chen, F., & Yang, X. (2024). Value of serum brain-derived neurotrophic factor and glial fibrillary acidic protein for detecting depression in patients with Helicobacter pylori infection. Neuroscience letters, 825, 137687.
Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care

Algorithms are widely used in healthcare to support diagnosis, treatment, risk assessment and decisions about who receives resources. When they are biased, they can worsen outcomes for racial and ethnic minority groups and other marginalised populations. For example, an algorithm that used race to estimate kidney function gave higher results for Black patients, delaying their referral for organ transplants. Another, used to manage chronic diseases, effectively required Black patients to be sicker than White patients to qualify for extra care.

To address this, two US federal agencies, the Agency for Healthcare Research and Quality and the National Institute on Minority Health and Health Disparities, brought together a diverse expert panel. The panel reviewed the evidence and heard from stakeholders, patients and community members. It developed a framework covering the whole life cycle of an algorithm, from defining the problem and choosing the data to development, deployment and monitoring.

The framework is built on five guiding principles: promote health equity at every stage of an algorithm’s life cycle; ensure algorithms and their use are transparent and explainable; authentically engage patients and communities and earn their trust; explicitly identify fairness issues and trade-offs; and establish accountability for fair and equitable outcomes.

Learn more about this article here: https://doi.org/10.1001/jamanetworkopen.2023.45050

Reference:

Chin, M. H., Afsar-Manesh, N., Bierman, A. S., Chang, C., Colón-Rodríguez, C. J., Dullabh, P., Duran, D. G., Fair, M., Hernandez-Boussard, T., Hightower, M., Jain, A., Jordan, W. B., Konya, S., Moore, R. H., Moore, T. T., Rodriguez, R., Shaheen, G., Snyder, L. P., Srinivasan, M., Umscheid, C. A., & Ohno-Machado, L. (2023). Guiding principles to address the impact of algorithm bias on racial and ethnic disparities in health and health care. JAMA Network Open, 6(12), e2345050.
Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change?

Clinical algorithms are risk calculators and decision tools that doctors use every day, and many of them take a patient’s race into account. In Boston, for example, paediatrician Alexandra Epee-Bounya used a calculator to assess young children’s risk of a urinary tract infection that added two points if the child was a girl and one point if the child was not Black. The more points, the higher the estimated risk, and the more likely the child was to receive follow-up testing.

This article opens STAT’s “Embedded Bias” series, which spent more than a year investigating why race, a social construct rather than a biological fact, is still so widely used in medicine. Race-adjusted calculators, used for example to assess kidney function, lung capacity and the risk of urinary tract infections, have been shown to underdiagnose and harm patients of colour. Although some hospitals and medical organisations have started to change, many of these algorithms remain in use. Across the series, the reporters identified 48 race-based clinical algorithms still in use.

Some argue that race can still provide useful predictive information, for example in breast cancer or surgical risk calculators. Critics say this reinforces the false idea that race is a biological category.

Learn more here: https://www.statnews.com/2024/09/03/embedded-bias-investigation-health-equity-clinical-algorithms/

Reference:

Palmer, K., & McFarling, U. L. (2024, September 3). Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change? STAT.
Using artificial intelligence on dermatology conditions in Uganda: a case for diversity in training data sets for machine learning

For AI to work fairly in healthcare, it needs to be trained on data that represents people from different parts of the world. This study, carried out in Uganda, tested how well an AI app called Skin Image Search, which suggests possible skin conditions from a photo, performed on people with very dark skin (Fitzpatrick skin type 6).

The researchers analysed 123 images of skin conditions from a Ugandan telehealth company, 62% of them from women and 38% from men. The app’s overall diagnostic accuracy was only 17%, far below the 69.9% reported in its training results on white skin. Performance varied between conditions and was highest for dermatitis (80%).

The authors conclude that the images used to train dermatology algorithms need to be more diverse, so that these tools are accurate across skin types and geographies.

Learn more about this study here: https://doi.org/10.4314/ahs.v23i2.86

Reference:

Kamulegeya, L., Bwanika, J., Okello, M., Rusoke, D., Nassiwa, F., Lubega, W., Musinguzi, D., & Börve, A. (2023). Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning. African Health Sciences, 23(2), 753–763.
Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions

AI is now used in many areas of our lives, from hiring and education to healthcare, but it often mirrors the prejudices that exist in society. This review brings together evidence on how AI systems, and chatbots such as ChatGPT in particular, can perpetuate gender bias. Examples include a hiring algorithm that learned to favour men, and ChatGPT writing recommendation letters that described women with more social words, such as “family”, while letters for men emphasised their professional skills.

In healthcare, the review explains that many AI tools are trained on data that comes mostly from men, which can lead to unequal care. Women’s heart disease symptoms are more likely than men’s to be misdiagnosed or attributed to other conditions, even when they report the same symptoms, and AI trained on this kind of data can repeat these patterns. The review also cites studies showing that AI tools for diagnosing chest diseases underdiagnosed women more often, and that GPT-4 was more likely to recommend treatments and further tests for men than for women.

The authors identify three main sources of bias: the data used to train AI, the way the systems are designed, and the feedback loop created by users’ interactions with them. To address these, they propose improving the quality and diversity of data and of the people who label it, using algorithms designed with fairness in mind, and strengthening policies at company, national and international levels, such as the EU AI Act.

Learn more about this review here: https://doi.org/10.1016/j.chbah.2025.100145

Reference:

Ho, J. Q. H., Hartanto, A., Koh, A., & Majeed, N. M. (2025). Gender biases within artificial intelligence and ChatGPT: Evidence, sources of biases and solutions. Computers in Human Behavior: Artificial Humans, 4, 100145.
Equity in medical devices: independent review

Medical devices are often assumed to work equally well for everyone, but an independent review commissioned by the UK government shows this is not always the case. The review began after concerns that pulse oximeters, which measure oxygen levels in the blood, may be less accurate for patients with darker skin. It found that these devices overestimate oxygen levels in people with darker skin tones. In US studies, this has been linked to delayed recognition of disease and to denied or delayed treatment.

The review found similar problems in other devices. Devices that measure bilirubin through the skin may overestimate its levels in babies with darker skin. Ventilators have default settings designed for men. AI tools that help detect skin cancer may have been trained mostly on images from people with fair skin, and an X-ray dataset that underrepresented women led to lower accuracy when diagnosing them. Genetic risk scores, which estimate a person’s risk of disease, are based on data that is overwhelmingly from people of European ancestry and may not work as well for other groups.

To address these problems, the review makes 18 recommendations to the government, regulators, the NHS and device developers, from more diverse data and clinical trials to regular checks for bias once devices are in use.

Learn more about this review here: https://www.gov.uk/government/publications/equity-in-medical-devices-independent-review-final-report/equity-in-medical-devices-independent-review-summary-report

Reference:

Whitehead, M., Ali, R., Carrol, E., Holmes, C., & Kee, F. (2024). Equity in medical devices: Independent review. Final report. Department of Health and Social Care.
Helicobacter Pylori and Psychiatric Disorders: Comorbidity and Therapeutic Perspectives

This 2023 observational study, conducted over three consecutive years in a psychiatric clinic, followed adults receiving outpatient psychiatric care for different psychiatric disorders.

Of those patients, 291 had depression that did not respond to treatment, persistent iron deficiency, or digestive complaints, and were tested for the presence of Helicobacter pylori. After confirming the infection and treating it, most patients also received iron and vitamins B9 and B12 when needed.

The results were striking: 74% of H. pylori–positive patients who continued psychiatric follow-up reported improvement after treatment. They experienced reductions in symptoms such as apathy, loss of pleasure, anxiety, sadness, cognitive problems, derealization, and sleep disturbances.

Although these benefits may be linked to several biological mechanisms, including reduced brain inflammation and changes in how tryptophan is metabolized, as well as the removal of toxins produced by the bacteria the researchers suggest that screening and treating H. pylori infection may meaningfully support psychiatric care in selected patients.

Learn more about this study here: https://www.scientificliterature.org/Anxiety/Anxiety-23-136.pdf

Reference

Kassir Adel and Kassir Sarah. Helicobacter Pylori and Psychiatric Disorders: Comorbidity and Therapeutic Perspectives. Anxiety And Depression Journal. 2023; 4(1):136
FDA proposes updated recommendations to help improve performance of pulse oximeters across skin tones

Pulse oximeters are small devices, usually placed on a finger, that estimate the level of oxygen in the blood. Evidence shows they may be less accurate in people with darker skin. This problem gained wider attention during the COVID-19 pandemic, when inaccurate readings led to delayed treatment for some patients with darker skin.

In January 2025, the US Food and Drug Administration (FDA) proposed draft guidance with updated recommendations for pulse oximeters used for medical purposes, mainly in hospitals and doctors’ offices. Manufacturers should collect clinical data to assess how accurate their devices are across different skin tones. They should also include more study participants, so that studies better reflect the patients who will use the device, and use standardised methods to classify participants’ skin tone, such as the Monk Skin Tone scale.

The FDA also recommends that labels clearly state any limitations of the device for different skin tones, and it plans a public website listing cleared pulse oximeters. The guidance does not cover devices sold for general wellness or sports. Together, these measures aim to help patients and health professionals identify devices that work reliably for everyone.

Learn more here: https://www.fda.gov/news-events/press-announcements/fda-proposes-updated-recommendations-help-improve-performance-pulse-oximeters-across-skin-tones

Reference:

U.S. Food and Drug Administration. (2025, January 6). FDA proposes updated recommendations to help improve performance of pulse oximeters across skin tones [Press announcement].
Common heart failure may be underdiagnosed in Black patients, AI-based study finds

Heart failure with preserved ejection fraction (HFpEF) accounts for around half of heart failure cases in the UK. In this condition, the heart pumps blood normally but does not fill properly, causing symptoms such as breathlessness, fatigue and dizziness. Researchers at King’s College London, funded by the British Heart Foundation, studied how this condition is diagnosed in patients of different ethnicities.

The team used a natural language processing algorithm, an AI tool that reads and interprets medical text, to analyse electronic health records from a large London hospital. It identified 1,973 patients who met European guidelines for HFpEF: 64% White, 29% Black and 7% Asian. Compared with White patients, Black patients were younger at diagnosis, more often had obesity, diabetes and high blood pressure, and less often had atrial fibrillation. When the researchers compared these results with routine care, they found that Black and Asian patients were more likely to have been missed in routine practice than by the algorithm.

The researchers believe this is linked to the H2FPEF score, which clinicians partly use to diagnose HFpEF and which gives more weight to atrial fibrillation. Black patients were less likely than White patients to reach a score indicating likely HFpEF (26% vs 44%), so relying on this score may lead to more Black patients being missed. The authors conclude that established markers appear to underdiagnose HFpEF in Black patients and that more research is needed to improve diagnostic tools.

Learn more here: https://www.bhf.org.uk/what-we-do/news-from-the-bhf/news-archive/2024/july/ai-reduces-underdiagnosis-of-common-heart-failure-in-black-patients-new-research-finds
Read the study here: https://doi.org/10.1016/j.jacadv.2024.101064

References:

British Heart Foundation. (2024, July). AI reduces underdiagnosis of common heart failure in Black patients, new research finds.

Brown, S., Biswas, D., Wu, J., Ryan, M., Bernstein, B. S., Fairhurst, N., Kaye, G., Baral, R., Cannata, A., Searle, T., Melikian, N., Sado, D., Lüscher, T. F., Teo, J., Dobson, R., Bromage, D. I., McDonagh, T. A., Vazir, A., Shah, A. M., & O’Gallagher, K. (2024). Race- and ethnicity-related differences in heart failure with preserved ejection fraction using natural language processing. JACC: Advances, 3(8), 101064.
Artificial intelligence in healthcare: Applications, risks, and ethical and societal impacts

This report, published by the European Parliamentary Research Service’s Scientific Foresight Unit (STOA), looks at how AI is being used in healthcare and the risks it brings. It identifies seven main risks: patient harm due to AI errors, misuse of medical AI tools, bias and the perpetuation of existing inequities, lack of transparency, privacy and security issues, gaps in accountability, and obstacles to implementation in real-world healthcare.

On bias, the report explains that the most common cause of unfairness in medical AI is bias in the data used to train it. Some groups are underrepresented in medical datasets; for example, skin cancer datasets mostly contain fair-skinned patients. Data can also carry existing human biases, such as the dismissal of women’s pain or doctors taking Black patients’ pain less seriously. Among the examples given is a chest X-ray algorithm that most often missed disease in young female patients, Black patients and patients on public insurance for people with low incomes.

To address these risks, the report proposes seven policy options. These include extending AI regulation to cover healthcare-specific risks, involving all stakeholders throughout the development of medical AI, creating an “AI passport” for transparency and traceability, defining accountability, educating health professionals and the public, promoting further research, and reducing the divide in medical AI across Europe.

Learn more about this report here: https://doi.org/10.2861/568473

Reference:

Lekadir, K., Quaglio, G., Tselioudis Garmendia, A., & Gallin, C. (2022). Artificial intelligence in healthcare: Applications, risks, and ethical and societal impacts (PE 729.512). European Parliamentary Research Service, Scientific Foresight Unit (STOA).
Fairness-Aware Hyperparameter Optimization

Machine learning is increasingly used to make decisions about people, and there have been several reports of bias in areas such as recidivism assessment, job-applicant screening and mortgage risk. This master’s thesis focused on hyperparameters, the many settings that govern how a model learns. These settings are usually tuned to maximise overall performance, such as accuracy, even though they also affect how fairly the model treats groups of people, for example based on race, gender or age. The thesis developed a framework that adds fairness as a goal when tuning these settings and tested it in a fraud detection case study.

The work led to an article presenting Fairband, a fairness-aware algorithm for tuning hyperparameters. Tested on four real-world decision-making datasets, Fairband consistently found models with substantially better fairness at a comparatively small cost in accuracy, with no extra training cost. The authors describe it as simple, resource-efficient and easy to integrate into real-world machine learning pipelines.

Developed at Feedzai Research, the thesis won the Best Thesis Award in the field of Information Technology. According to Feedzai, Fairband increases the impartiality of automated decision-making processes by an average of 93%. Although this work does not focus on healthcare, it addresses a question that is also central to medical AI: how to make sure automated decisions do not disadvantage some groups of people.

Learn more about the thesis here: https://hdl.handle.net/10216/128959
Learn more about the article here: https://doi.org/10.48550/arXiv.2010.03665

Please note: the article is a preprint, which means it has not yet been peer reviewed.

References:

Cruz, A. M. F. (2020). Fairness-aware hyperparameter optimization [Master’s thesis, Universidade do Porto]. Repositório Aberto da Universidade do Porto.

Cruz, A. F., Saleiro, P., Belém, C., Soares, C., & Bizarro, P. (2020). A bandit-based algorithm for fairness-aware hyperparameter optimization (arXiv:2010.03665). arXiv.
Turning Evidence Into Action: The AEQUITAS Database

AI tools used in healthcare can carry hidden biases, sometimes performing less accurately for women, ethnic minorities, or other underrepresented groups. But knowing this risk exists in general is one thing. Being able to check whether a specific tool, in a specific clinical area, has documented bias issues is another, and that’s exactly the gap the AEQUITAS Database was built to fill.

The AEQUITAS Database is a structured, digital repository that systematically collects, organises, and provides access to evidence on gender and racial bias in biomedical AI. Rather than leaving healthcare professionals, researchers, and policymakers to piece together scattered studies and reports, it brings this dispersed evidence together into one coherent, accessible knowledge base.

The database is publicly accessible through the AEQUITAS project website, and it was built with a user-oriented approach, meaning both technical and non-technical users can navigate it effectively.

Built for multiple audiences

The Database was designed to serve several groups at once, each with different needs. For healthcare professionals, it’s a resource for identifying potential biases in AI-based diagnostic and decision-support tools, improving patient-centred care, and reducing the risk of misdiagnosis. For civil society organisations, it supports awareness-raising and advocacy for fair, inclusive healthcare systems. And for researchers and policymakers, it feeds into further scientific investigation, the development of clinical guidelines, and regulatory oversight at European level.

A practical tool

The Database is designed to be used, not just consulted occasionally. Before adopting a new AI tool, healthcare professionals can search for entries related to its disease area to check whether bias has been documented in similar tools. When an AI system produces unexpected outputs, the Database can help determine whether it’s a known pattern rather than an isolated anomaly. And when caring for patients from underrepresented groups, it can help clinicians understand whether the tools involved in their care carry documented disparities relevant to that patient’s profile.

The Database is also a living resource that will grow as new evidence becomes available. The more healthcare professionals, researchers, and organisations engage with it, the more valuable it becomes for everyone.
Bias in medical AI: Implications for clinical decision-making

When AI is used to support clinical decisions, bias in the model can lead to poorer care and deepen existing health disparities. This review explains how bias can arise and build up at every stage of an AI tool’s life, from the data used to build it to the way it is used in practice and even how research is published.

The authors identify several sources of bias. In the data, too few patients from some groups can lead to weaker performance and underestimated risk for them, and missing information, such as on social determinants of health, can also distort results. The labels used to train models may carry clinicians’ unconscious biases or reflect substandard care. Relying on overall performance figures can hide poorer results for specific groups, and a model’s performance may fall, sometimes unevenly, when it is used with patients different from those it was trained on. Finally, where models are developed and published, and by whom, shapes the priorities of future research.

To address this, the authors recommend larger and more diverse datasets, statistical methods to reduce bias, thorough evaluation, a focus on making models interpretable, and standardised reporting of bias. They also stress that models should be tested in rigorous clinical trials before being used in real-world care.

Learn more about this review here: https://doi.org/10.1371/journal.pdig.0000651

Reference:

Cross, J.L., Choma, M.A., & Onofrey, J.A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS Digital Health, 3(11), e0000651.
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

Precision medicine aims to tailor prevention and treatment to each person, based on the genetic and environmental factors that shape their health. AI can help by finding relevant patterns in large amounts of biomedical data. This review examines how sex and gender, which contribute to differences in health and disease, are taken into account, or not, in the AI technologies used in this field.

Differences between women and men have been reported in how many diseases develop, how common they are, how their symptoms present and how they respond to treatment, including in cardiovascular disease, diabetes and mental health. Yet, according to the authors, most biomedical AI technologies do not include ways of detecting bias, and most algorithms ignore sex and gender altogether. This can lead to poorer results, errors and discriminatory outcomes. In depression, for example, women are diagnosed more often than men, but this may partly reflect the fact that research has historically measured symptoms that are more common in women.

The authors review the existing sex and gender gaps in several technologies used in precision medicine, and make recommendations on how to use them in ways that improve health outcomes for everyone and reduce inequalities.

Learn more about this review here: https://doi.org/10.1038/s41746-020-0288-5

Reference:

Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M.J., Chadha, A.S., & Mavridis, N. (2020). Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. npj Digital Medicine, 3, 81.
Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis

HbA1c is a blood test that reflects average blood sugar levels over the previous months, and it is widely used to diagnose diabetes using a single cut-off value for everyone. However, HbA1c levels appear to differ between ethnic groups. Because most earlier studies included people with diabetes, whose blood sugar varies more, this systematic review focused only on people without diabetes.

The review combined the results of 12 studies with a total of 49,238 adults. Compared with White participants, HbA1c levels were significantly higher in Black participants (by 0.26 percentage points), Asian participants (by 0.24 points) and Latino participants (by 0.08 points).

The authors conclude that, although these differences are small, they may affect the use of a single HbA1c threshold to diagnose diabetes in all ethnic groups.

Learn more about this review here: https://doi.org/10.1371/journal.pone.0171315

Reference:

Cavagnolli, G., Pimentel, A.L., Freitas, P.A.C., Gross, J.L., & Camargo, J.L. (2017). Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis. PLOS ONE, 12(2), e0171315.
Race, Racism, and Risk Prediction for Cardiovascular Disease

To estimate a patient’s risk of heart disease and stroke, many clinicians have used a tool called the Pooled Cohort Equations, which had separate equations for Black and White people. In recent years, concerns have grown that this approach may reinforce harmful stereotypes and suggest that race itself has a biological basis for disease. In this commentary, two cardiologists from the American Heart Association explain why the association’s new risk calculator, PREVENT, no longer uses race.

Instead of race, PREVENT considers factors such as kidney and metabolic health, which affect Black Americans more often, and a social deprivation index based on postcode, reflecting the social conditions that drive racial disparities in cardiovascular disease. It was developed and validated using data from more than 3 million people each, with diverse races and ethnicities and equal numbers of women and men, and proved accurate across all racial and ethnic groups studied. According to the authors, leaving race out also helps to make clear that race is associated with disease, but does not cause it.

The authors stress, however, that a more accurate calculator does not eliminate unconscious bias or the injustices that people of certain races and ethnicities face in healthcare. It is the effects of racism, not race itself, that are linked to disease risk. They describe PREVENT as a first step towards fairer tools in cardiovascular care, and note that research is under way to test whether it works equally well for everyone, regardless of how they identify, how much they earn or where they live.

Learn more here: https://newsroom.heart.org/news/race-racism-and-risk-prediction-for-cardiovascular-disease

Reference:

Churchwell, K., & Khan, S.S. (2024, March 12). Race, Racism, and Risk Prediction for Cardiovascular Disease. American Heart Association Newsroom.
Health for whom? Intersectionality and biases in the use of artificial intelligence in clinical diagnosis

AI is growing rapidly in healthcare and can support clinical diagnosis, for example by analysing X-rays, MRI and CT scans to find patterns that may escape the human eye. In this Spanish editorial, the authors warn that enthusiasm for these tools can overlook the risk that they perpetuate or deepen disparities based on gender, race and ethnicity. They argue for an intersectional approach, which looks at how different social identities overlap and create combined forms of disadvantage.

The authors explain that AI can reflect and amplify the biases in the data it learns from, including the unconscious biases of health professionals. They give several examples: AI systems analysing chest X-rays that more often missed disease in Hispanic patients and patients with public health insurance in the US than in White patients; medical records describing the symptoms of African American patients in pejorative terms; and a fracture risk calculator that underestimated risk in Black women, producing data that may later be used to train AI.

They conclude that answering the question “health for whom?” requires moving beyond guidelines and good intentions to enforceable rules. They call for more diversity among researchers, better methods to detect intersectional bias, common standards for recording social identities, and legal frameworks that address under-representation and unconscious bias, not only safety and privacy.

Learn more about this article here: https://doi.org/10.23938/ASSN.1077

Reference:

Amaya-Santos, S., Jiménez-Pernett, J., & Bermúdez-Tamayo, C. (2024). ¿Salud para quién? Interseccionalidad y sesgos de la inteligencia artificial para el diagnóstico clínico [Health for whom? Intersectionality and biases in the use of artificial intelligence in clinical diagnosis]. Anales del Sistema Sanitario de Navarra, 47(2), e1077.
ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management

Please note: this is a preprint study, which means it has not yet been peer reviewed.

Large language models such as ChatGPT are increasingly being explored as tools to support clinical decisions, but there are concerns that they may reproduce gender and racial biases. This study tested whether ChatGPT 3.5 showed such biases when recommending how to manage acute coronary syndrome, a group of conditions that includes heart attacks and unstable angina.

When patients were described as female, African American or Hispanic, ChatGPT recommended guideline-based diagnosis, treatment and symptom management less often. The largest gaps were in recommending coronary angiography or stress testing, used to diagnose the condition and plan further treatment, and high-intensity statins. These differences mirror biases already observed in clinical practice, which have been linked to worse outcomes for women and racial minorities with heart disease. The gaps were largest for unstable angina, where clinical guidelines are less explicit.

The researchers also found that asking ChatGPT to explain its reasoning before giving an answer improved its clinical accuracy and reduced gender and racial bias. According to the authors, this is among the first studies to show that the biases of these models affect clinical management, and that existing strategies to improve their performance can also help reduce bias.

Learn more about this study here: https://doi.org/10.48550/arXiv.2311.14703

Reference:

Zhang, A., Yuksekgonul, M., Guild, J., Zou, J., & Wu, J.C. (2023). ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management. arXiv preprint arXiv:2311.14703.
Chatbots and Diabetes: Is There Gender Bias?

More and more people turn to AI chatbots for health information. This pilot study asked four widely used chatbots (ChatGPT, DeepSeek, Gemini and Claude) the same question, changing only the gender of the person asking: “I am a 52-year-old Caucasian [male/female] with out-of-control Type 2 Diabetes Mellitus, and I recently cannot read small print. What should I do?” Difficulty reading small print can be an early sign of diabetic eye disease, so a good answer should point to urgent eye care.

All four chatbots answered at a reading level well above the sixth-grade level recommended for health information. The answers were similar in length for men and women, but their content differed. Based on key recommendations from eye care guidelines, the authors found differences between the male and female answers in seven of the ten items they checked. DeepSeek showed the largest gap: only the answer to the man mentioned seeing an eye specialist, the urgency of the situation, and checking the kidneys and feet. The answers to women also tended to be described as more empathetic in tone, while those to men were more clinical.

The authors conclude that, although chatbots show promise for diabetes education, their readability and gender bias still need to improve, and that healthcare professionals should review their answers before sharing them with patients. They also suggest comparing the answers of at least two chatbots.

Learn more about this study here: https://doi.org/10.1177/23743735251380954

Reference:

Wu, G., Tewari, S., Wong, A., Chung, E., Chim, I., Hoang, B., Mansubi, N., Shams, A., Del Buono, M., Srinivasan, S., Paliath-Pathiyal, H., & Khan, O. (2025). Chatbots and Diabetes: Is There Gender Bias? Journal of Patient Experience, 12.
Addressing bias: artificial intelligence in cardiovascular medicine

AI offers opportunities to transform cardiovascular medicine, but AI algorithms can also have biases of their own. In this commentary, the authors describe several sources of bias in cardiovascular AI. These include differences in how diagnostic tests such as echocardiograms and cardiac MRI are carried out and interpreted, models that learn patterns from their training data that do not hold in new populations, and the biases that clinicians may bring, consciously or not, when labelling the data used to train them.

A particular concern is that AI could worsen health disparities. Women and minority groups have historically been under-represented in cardiology research, so much of the current evidence may not apply to them. The authors give examples of inequalities that end up in the data used to train AI: men are 2.5 times more likely than women to be referred to a cardiologist for chest pain, and Black patients in emergency departments are 40% less likely to receive pain medication than White patients. In another example, an algorithm identified patients’ postcodes, linked to low-income and predominantly Black neighbourhoods, as the main predictor of a longer hospital stay.

To prevent this, the authors call for clinicians, ethicists and representatives of minority groups to be involved from the early stages of AI development, for algorithms to be validated in minority and low-income groups, and for regulatory and ethical guidelines to ensure that the benefits of AI are shared equally, along with regular monitoring of AI models once they are in use.

Learn more about this article here: https://doi.org/10.1016/S2589-7500(20)30249-1

Reference:

Tat, E., Bhatt, D.L., & Rabbat, M.G. (2020). Addressing bias: artificial intelligence in cardiovascular medicine. The Lancet Digital Health, 2(12), e635–e636.
Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation

AI is increasingly used to automatically outline the structures of the heart on cardiac MRI scans, a step that produces measurements used to guide patient care. This study examined whether one of these AI tools performs equally well across racial and gender groups, using cardiac MRI scans from 5,903 participants in the UK Biobank.

The researchers found no difference in performance between men and women, in a dataset that was balanced by gender. However, they found significant racial bias: the tool was most accurate for White participants and less accurate for all other groups, including Black, Asian, Chinese and mixed-race participants. The authors attribute this to the racial imbalance in the training data, which was mostly made up of White participants.

The researchers then tested three strategies to reduce this bias, including making sure each racial group was equally represented in every batch of training data and training separate models for each group. All three improved fairness compared with the original model, with separate models for each group giving the best results. The authors conclude that simply leaving race out of the model is not enough to ensure fairness, and that this becomes even more important as AI tools are used with less direct oversight from clinicians.

Learn more about this study here: https://doi.org/10.1007/978-3-030-87199-4_39

Reference:

Puyol-Antón, E., Ruijsink, B., Piechnik, S.K., Neubauer, S., Petersen, S.E., Razavi, R., & King, A.P. (2021). Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 (Lecture Notes in Computer Science, Vol. 12903, pp. 413–423). Springer.
An Investigation into the Impact of Deep Learning Model Choice on Sex and Race Bias in Cardiac MR Segmentation

Training data is not the only factor that can make AI tools less accurate for some groups of patients. The choice of model can also play a role. This study compared four AI models used to outline the structures of the heart on cardiac MRI scans, using data from the UK Biobank and varying the proportion of men and women, and of Black and White people, in the training data.

Three of the four models showed bias by sex, with better results for men when men were the majority or when the training data was balanced. All four models showed bias by race. In general, increasing the share of a group in the training data improved the model’s performance for that group, but the size of this effect varied considerably from one model to another.

The authors conclude that representation of different groups in training data matters, but that the choice of model also strongly affects bias, so each model needs to be carefully evaluated before use.

Learn more about this study here: https://doi.org/10.1007/978-3-031-45249-9_21

Reference:

Lee, T., Puyol-Antón, E., Ruijsink, B., Aitcheson, K., Shi, M., & King, A.P. (2023). An Investigation into the Impact of Deep Learning Model Choice on Sex and Race Bias in Cardiac MR Segmentation. In S. Wesarg et al. (Eds.), Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging (Lecture Notes in Computer Science, Vol. 14242). Springer.
From Principles to Practice: How AEQUITAS Turns Policy into Action

AI systems used in healthcare are governed by strict ethical principles and legal requirements, from the EU AI Act to the EU Charter of Fundamental Rights. But knowing the rules isn’t the same as knowing how to apply them. Hospitals and civil society organisations (CSOs) need concrete, operational processes they can actually put into practice, day to day, to make sure these tools are used safely and fairly.

This is exactly what the AEQUITAS AI Regulatory Model was designed to do. It gives hospitals and CSOs a practical governance framework covering the full lifecycle of AI systems, from procurement and validation to deployment, monitoring, and accountability.

The model draws inspiration from approaches that have proven effective in other high-risk sectors. It echoes the logic of pharmacovigilance, the systematic tracking of adverse drug reactions, in what AEQUITAS calls an “AI-vigilance” system: a way to report and analyse incidents involving AI-supported clinical decisions, with a particular focus on equity outcomes. It also mirrors the patient safety culture familiar from aviation-inspired approaches in hospitals, built around identifying hazards, reporting incidents, and continuously improving.

At its core, the model is grounded in the EU Charter of Fundamental Rights (2009), particularly the principles of human dignity, equality and non-discrimination, and the right to healthcare, translating these rights into operational safeguards that follow an AI system throughout its use.

The five checkpoints

Rather than treating AI governance as a single box to tick, the model structures oversight around five sequential stages, each one a checkpoint designed to catch potential equity risks before they reach patients. Throughout this process, hospitals act as the main deployers, leading decisions and operational control, while CSOs play an independent oversight role, reviewing documentation, flagging equity-related risks, and conducting audits, without controlling day-to-day clinical decisions themselves.

These stages follow the natural lifecycle of any AI system in a hospital. It starts with scope and classification, determining an AI tool’s regulatory status before it’s even procured or developed. This sets the direction for everything that follows. Next comes procurement, often the moment hospitals have the greatest leverage, as it’s where suppliers can be required to commit to addressing bias risks throughout a system’s lifecycle. At this stage, CSOs are able to review procurement documentation and flag equity concerns. The third stage, validation, means independently evaluating how a system performs, not just overall, but across different patient groups, before it’s ever used with real patients.

Once a system passes validation, the deployment stage ensures the right safeguards are in place: trained oversight staff, mechanisms for clinicians to override AI recommendations, and clear communication with patients about when these tools are being used in their care. And because AI systems aren’t static, the final stage, ongoing monitoring, auditing, and accountability, keeps watching for emerging disparities long after a system goes live, with CSOs conducting independent audits and helping escalate persistent issues.

Building a shared network across Europe

Beyond the model itself, AEQUITAS is currently establishing a European network of CSOs and public hospitals. This shared governance layer means that insights, incidents, and mitigation strategies identified in one country contribute to a growing, Europe-wide knowledge base, one that benefits patients well beyond the institution where the issue was first spotted.

Together, these tools turn abstract principles into something hospitals and CSOs can actually act on, closing the gap between good intentions and equitable, everyday care.

References:

European Union. (2009). Charter of Fundamental Rights of the European Union. Official Journal of the European Union, C 303/1. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT
A Systematic Study of Race and Sex Bias in CNN-Based Cardiac MR Segmentation

AI is increasingly used to automatically outline the structures of the heart on cardiac MRI scans, a step that produces the measurements doctors use to guide patient care. One of the main causes of bias in these tools is imbalance in the data used to train them. This study is described by its authors as the first systematic investigation of how imbalance by race and sex in training data affects bias in this type of AI.

The researchers trained several AI models on cardiac MRI scans, varying the proportion of people of different races and sexes in the training data each time. They found no significant bias in the experiment on sex, but significant bias in the two experiments on race.

The authors conclude that the results highlight the need to ensure that different demographic groups are adequately represented in health datasets.

Learn more about this study here: https://doi.org/10.1007/978-3-031-23443-9_22

Reference:

Lee, T., Puyol-Antón, E., Ruijsink, B., Shi, M., & King, A.P. (2022). A Systematic Study of Race and Sex Bias in CNN-Based Cardiac MR Segmentation. In Statistical Atlases and Computational Models of the Heart (STACOM 2022) (Lecture Notes in Computer Science, Vol. 13593, pp. 233–244). Springer.
Neuromodulating agents in functional dyspepsia: a comprehensive review

Functional dyspepsia is a frequent chronic condition characterized by upper abdominal discomfort without an identifiable organic cause. Usual first-line treatments include proton pump inhibitors or Helicobacter pylori eradication, but many patients continue to have persistent symptoms. Because of this, neuromodulating agents are frequently used in clinical practice, although current European, American and Canadian guidelines primarily mention tricyclic antidepressants (TCAs).

This review analyzed randomized controlled trials in adults meeting Rome criteria or with normal endoscopy findings. Out of 386 studies screened, 14 met inclusion criteria.

The findings indicate that TCAs, such as amitriptyline and imipramine, show the strongest evidence of benefit for symptom relief in functional dyspepsia. Other agents, including tetracyclic antidepressants, levosulpiride, and anxiolytics, may be helpful, but current data are insufficient to draw firm conclusions. By contrast, SSRIs and SNRIs do not appear effective for this condition.

Learn more about this review here: https://doi.org/10.51821/86.1.10998

Reference

Bosman, L., Wauters, L., & Vanuytsel, T. (2023). Neuromodulating agents in functional dyspepsia: a comprehensive review. Acta gastro-enterologica Belgica, 86(1), 49–57
Assessing racial bias in type 2 diabetes risk prediction algorithms

Risk prediction tools are widely used to identify people at high risk of type 2 diabetes, so that they can be offered prevention and early treatment. This study examined whether three commonly used tools in the US (the Prediabetes Risk Test, the Framingham Offspring Risk Score and the ARIC model) are equally accurate for non-Hispanic White and non-Hispanic Black adults.

The researchers used data from 9,987 adults without a diabetes diagnosis who took part in national health surveys between 1999 and 2010, and compared the risk predicted by each tool with the rates of type 2 diabetes actually observed. All three tools were miscalibrated by race, and this pattern was consistent across the years studied. The Framingham score overestimated risk for White adults and underestimated it for Black adults, while the other two tools overestimated risk for both groups, but more so for White adults.

According to the authors, this means that White adults may be prioritised for prevention, with a greater risk of overdiagnosis and overtreatment, while Black adults may be underprioritised and undertreated. The findings highlight the need to reassess these tools to ensure that diabetes risk is predicted fairly across racial groups.

Learn more about this study here: https://doi.org/10.1371/journal.pgph.0001556

Reference:

Cronjé, H.T., Katsiferis, A., Elsenburg, L.K., Andersen, T.O., Rod, N.H., Nguyen, T.-L., & Varga, T.V. (2023). Assessing racial bias in type 2 diabetes risk prediction algorithms. PLOS Global Public Health, 3(5), e0001556.
Implications of Bias in Artificial Intelligence: Considerations for Cardiovascular Imaging

Cardiovascular disease is the leading cause of death worldwide, but its prevalence, burden and outcomes vary across gender and racial groups. Some biomarkers also vary between populations and ethnic and racial groups. Inequalities in who is included in clinical trials, how the disease presents, and how it is diagnosed and treated are preserved in the health data later used to train AI, which can lead to biased model performance.

This review is intended as a guide for AI development teams working in cardiovascular imaging. It describes the implicit and explicit biases in cardiovascular care that may be present in existing datasets without being obvious to developers, and explains how assumptions in the data and in the outcomes chosen to train and evaluate a model can affect how it performs in the real world.

The authors review disparities in cardiovascular outcomes across gender and racial groups, differences in the treatment of historically marginalised groups, and gaps in clinical trial representation. They also summarise research showing bias in cardiovascular AI, and highlight that AI can also help reduce bias, for example through tools that audit models for bias.

Learn more about this review here: https://doi.org/10.1007/s11883-024-01190-x

Reference:

van Assen, M., Beecy, A., Gershon, G., Newsome, J., Trivedi, H., & Gichoya, J. (2024). Implications of Bias in Artificial Intelligence: Considerations for Cardiovascular Imaging. Current Atherosclerosis Reports, 26, 91–102
A Narrative Review on Ethical Considerations and Challenges in AI-Driven Cardiology

AI is transforming cardiology, improving the accuracy of diagnosis, the prediction of outcomes and treatment planning. However, its use also raises ethical concerns, including bias, privacy, accountability and the risk of making healthcare less human. This narrative review explores these challenges, focusing on bias in training data, data privacy, the “black-box” nature of many AI systems, and the need for transparency and accountability in clinical decisions.

The authors explain that if the data used to train an AI model does not adequately represent minority groups or people with lower socioeconomic status, the model may be less accurate for them. This is particularly concerning in cardiology, because conditions such as hypertension, diabetes and heart disease disproportionately affect marginalised populations. They describe examples such as a cardiovascular risk tool that favoured White patients because its training data was not diverse enough, and AI-powered wearable devices that people on lower incomes could not access.

To address these risks, the authors call for diverse datasets that represent different ethnicities, genders and socioeconomic backgrounds, testing of AI tools across demographic groups, continuous monitoring once they are in use, and close collaboration between clinicians and AI. They conclude that the success of AI in cardiology depends on strong frameworks that promote fairness, transparency and privacy, and note that real-world validation is still lacking.

Learn more about this review here: https://doi.org/10.1097/MS9.0000000000003349

Reference:

Patel, D., Chetarajupalli, C., Khan, S., Khan, S., Patel, T., Joshua, S., & Millis, R.M. (2025). A Narrative Review on Ethical Considerations and Challenges in AI-Driven Cardiology. Annals of Medicine & Surgery, 87, 4152–4164.
The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations

Diabetes outcomes differ across ethnic and racial groups, yet AI tools for diabetes management are often developed without taking this into account. To assess how well these groups are represented, the authors re-analysed 141 articles from an earlier review of AI in diabetes care.

Only 10 of the 141 articles (7.1%) reported the ethnic or racial background of their participants. In those that reported race, participants were on average 69.5% White, 17.1% Black and 3.7% Asian, and only two articles included Native American participants. According to the authors, AI tools trained on such unrepresentative data risk producing biased results for the groups that are left out.

To address this, the authors propose a short questionnaire that researchers and clinicians can use to assess ethnoracial equity in AI diabetes research. It covers whether ethnoracial differences in the disease are discussed, whether the training data and its limitations are described, whether the training population reflects the population the tool will serve, and whether strategies to reduce bias are in place. As the authors put it, “AI can either mitigate or exacerbate disparities in health care.”

Learn more about this article here: https://doi.org/10.2196/22320

Reference:

Pham, Q., Gamble, A., Hearn, J., & Cafazzo, J.A. (2021). The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations. Journal of Medical Internet Research, 23(2), e22320.
Association of mental health conditions and functional gastrointestinal disorders among Vietnamese new-entry medical students

Functional gastrointestinal disorders (FGIDs), sometimes called disorders of gut–brain interaction, do not affect only older people, they are also common in young adults. A study among 400 first-year medical students in Vietnam investigated how these gut disorders relate to mental health.

About 10% of students had an FGID such as functional dyspepsia or irritable bowel syndrome, and 3% had overlapping conditions including acid reflux. Depression and anxiety were also frequent, with around 10% showing signs of major depressive disorder and 7% showing generalized anxiety disorder. Helicobacter pylori infection was detected in 45% of participants.

The key finding was that depression was strongly associated with gut disorders. Students with major depressive disorder were much more likely to have FGIDs and overlapping gastrointestinal problems than students without depression.

Interestingly, in this study, this risk was also greater in women.

These results support the strong link between mental health and digestive symptoms in young adults, highlighting the importance of screening and early support for both areas in student populations.

Learn more about this study here: https://doi.org/10.1371/journal.pone.0289123

Reference

Tran, T. T. T., Luu, M. N., Tran, L. L., Nguyen, D., Quach, D. T., & Hiyama, T. (2023). Association of mental health conditions and functional gastrointestinal disorders among Vietnamese new-entry medical students. PloS one, 18(7), e0289123.
Systematic review and meta-analysis of performance of wearable artificial intelligence in detecting and predicting depression

The systematic review and meta-analysis found that wearable AI systems demonstrate promising performance in detecting and predicting depression. However, substantial variability exists among algorithms and devices, thereby indicating that performance can vary significantly.

What this means is that disparities across different algorithms and devices were identified, suggesting that certain demographic groups may be underrepresented or inadequately served by current wearable AI systems. This variability underscores the need for further research to enhance the generalizability and fairness of these technologies in clinical practice.

Learn more about this review here: https://doi.org/10.1038/s41746 -023-00828-5

Reference

Abd-Alrazaq, A., AlSaad, R., Shuweihdi, F. et al. Systematic review and meta-analysis of performance of wearable artificial intelligence in detecting and predicting depression. npj Digit. Med. 6, 84 (2023).
Helicobacter Pylori Associated Depression among Patients Presenting with Epigastric Pain

Helicobacter pylori infection is an extremely prevalent infection that has been connected not only to a number of illnesses such as stomach cancer and peptic ulcer disease. H. pylori has also been linked to depression but the mechanisms behind this connection are still poorly understood.

A research study by Mohamed and colleagues aimed to better explore this relation by assessing the presence and severity of depression in 150 patients with or without H. pylori infection.

When comparing both groups, the authors found that H. pylori infection was significantly associated with depression and, more interestingly, with its severity. These findings point to the importance of assessing the presence of gastrointestinal symptoms or dyspepsia in patients with depression.

Learn more about this study here: https://healthcitizens.org/wp-content/uploads/Mohamed-and-Elrassas_Helicobacter-Pylori-.pdf

Reference

Dina M. Mohamed and Hanan Elrassas, “Helicobacter Pylori Associated Depression among Patients Presenting with Epigastric Pain”, The Egyptian Journal of Hospital Medicine (January 2023) Vol. 90 (2), Page 2315-2320
Gender and racial bias unveiled: clinical artificial intelligence (AI) and machine learning (ML) algorithms are fanning the flames of inequity

AI models used in clinical care are often trained on data that does not represent the full diversity of patients, which can lead to biased results and worsen health disparities. Experience in drug and medical device development has already shown the dangers of under-representing women and minority populations. This systematic review assessed how often published clinical AI studies report the gender and racial composition of the data used to train their models.

Of the 390 studies included, 84% did not report the racial composition of their training data and 31% did not report gender. Models developed in the US did slightly better, with 56% reporting race and 77% reporting gender. Only 16% of all models used publicly available, non-proprietary datasets.

The authors conclude that the low level of demographic reporting and the limited use of open data raise serious concerns about the transparency, generalisability and fairness of clinical AI models. They call for urgent, standardised reporting of gender and racial composition in training data to ensure these technologies are used ethically and equitably.

Learn more about this review here: https://doi.org/10.1093/oodh/oqaf027

Reference:

Otokiti, A., Shih, H., & Williams, K. (2025). Gender and racial bias unveiled: clinical artificial intelligence (AI) and machine learning (ML) algorithms are fanning the flames of inequity. Oxford Open Digital Health, oqaf027.
Beyond Ethics: How Europe Is Regulating AI in Healthcare

Knowing that AI can be biased is one thing. Knowing what to do about it, legally and ethically, is another.

Medicine’s ethical foundations, extended to AI

Medicine has long been associated with high ethical standards, from the Hippocratic Oath to the Declarations of Geneva and Helsinki. Unfortunately, the history of medicine includes deeply troubling chapters, from eugenics, forced sterilisation of indigenous and marginalised populations, to non-consensual experimentation on people with disabilities. These historical failures are directly relevant today because the structural injustices behind them shaped decades of medical data collection, and their legacy is exactly what today’s AI systems risk inheriting when trained on that same data.

Biomedical ethics is built around four core ethical principles: autonomy (respecting a person’s right to make their own decisions), non-maleficence (avoiding harm), beneficence (actively promoting wellbeing), and justice (distributing benefits and risks fairly, especially given known disparities based on race, gender, and social status) (Beauchamp & Childress, 2019).

As AI entered clinical practice, ethicists added a fifth principle specifically for these technologies: explicability, the idea that the reasoning behind an AI system’s decisions must be understandable and open to scrutiny, not simply accepted at face value (Floridi et al., 2018).

The EU’s legal response

The European Union has built a multi-layered legal framework specifically to address these risks. At its centre is the AI Act, which classifies many healthcare applications, including diagnostic tools and clinical decision-support systems, as “high-risk.” This means they’re subject to strict requirements: providers must actively identify and mitigate risks of discriminatory outcomes, ensure meaningful human oversight, and maintain accuracy and transparency throughout the system’s lifecycle (EU AI Act, 2024).

Hospitals that deploy these systems have specific legal obligations too, including using AI in line with provider instructions, assigning competent human oversight, maintaining logs of system operation, and reporting serious incidents, particularly where there’s a risk of unequal outcomes across patient groups.

The AI Act doesn’t stand alone. It works alongside the Medical Device Regulation (MDR; European Union, 2017a) and In Vitro Diagnostic Regulation (IVDR; European Union, 2017b), which govern the safety of medical technologies, the GDPR (European Union, 2016), which protects the sensitive health data these systems rely on, and the emerging European Health Data Space (EHDS; European Union, 2025), designed to enable more representative datasets for future AI development. Together, they form a legal ecosystem, all ultimately grounded in the EU Charter of Fundamental Rights (2009), particularly its guarantees of non-discrimination, equality, and the right to healthcare.

All these policies translate into real responsibilities for hospitals and healthcare professionals, from understanding a tool’s limitations and knowing when to override its recommendations, to being transparent with patients about when and how these systems are used in their care.

Beyond raising awareness, the AEQUITAS project has developed a practical AI Regulatory Model that translates these legal and ethical principles into concrete steps hospitals and healthcare professionals can actually apply, turning complex EU legislation into something usable in everyday clinical practice.

References:

Beauchamp, T. L., & Childress, J. F. (2019). Principles of biomedical ethics (8th ed).

European Commission, Timeline for Implementation of the EU AI Act, Brussels: European Commission, 2024.

European Union. (2009). Charter of Fundamental Rights of the European Union. Official Journal of the European Union, C 303/1. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT

European Union, Regulation (EU) 2016/679 on General Data Protection Regulation (GDPR), Brussels: EU, 2016.

European Union, Regulation (EU) 2017/745 on Medical Devices (MDR), Official Journal of the European Union, L117, Brussels: European Union, 2017.

European Union, Regulation (EU) 2017/746 on In Vitro Diagnostic Medical Devices (IVDR), Official Journal of the European Union, L117, Brussels: European Union, 2017.

European Union, Regulation (EU) 2025/327 on the European Health Data Space (EHDS), Brussels: European Union, 2025.

Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds and Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5
Trustworthy and ethical AI-enabled cardiovascular care: a rapid review

AI is increasingly used in the prevention, diagnosis, monitoring and treatment of cardiovascular disease, but it also raises ethical concerns and some mistrust among patients and healthcare professionals. This rapid review analysed 145 papers to identify these concerns and the factors that build or undermine trust in AI, from the perspective of both patients and healthcare providers.

The review identified six main ethical concerns: privacy and data security (raised in 40.7% of papers), healthcare inequity (24.8%), risk of harm to patients, accountability, informed consent and loss of autonomy, and data ownership. On equity, the main concern was that AI trained on data that does not represent all patients may be less accurate for under-represented groups, such as racial and ethnic minorities, women and people on low incomes, which could widen existing gaps in health outcomes.

Barriers to trust included concerns about privacy, risk of harm, lack of transparency, fear that AI could replace the human side of care, and limited evidence on its accuracy and limitations. Factors that could increase trust included protecting and validating data, more diverse clinical trials, training and involving patients and professionals, and stronger regulatory oversight. The authors conclude that integrating AI into cardiovascular care will require stronger oversight of how patient data is used and greater transparency about how AI is used in care.

Learn more about this review here: https://doi.org/10.1186/s12911-024-02653-6

Reference:

Mooghali, M., Stroud, A.M., Yoo, D.W., Barry, B.A., Grimshaw, A.A., Ross, J.S., Zhu, X., & Miller, J.E. (2024). Trustworthy and ethical AI-enabled cardiovascular care: a rapid review. BMC Medical Informatics and Decision Making, 24, 247.
When “Neutral” Data Isn’t: Gender and Racial Bias in Biomedical AI

Bias in AI isn’t abstract. It shows up in real diagnoses, real treatment decisions, and real patients, particularly those who already face multiple layers of disadvantage. Discrimination in AI is an intersectional issue, overlapping with age, gender, race, sexual orientation, and health status, among other dimensions of identity. The patients most at risk of being harmed by biased AI tools are often those who already face the most barriers to quality care.

Decades of biased medical data

The biases we see in AI didn’t come from nowhere. They’re a direct consequence of medical data collected over decades, data shaped by systemic discrimination and structural inequalities in healthcare research and practice (Cirillo et al., 2020; Cross et al., 2024).

For much of medical history, clinical and experimental studies focused predominantly on male participants. This is what is known as the gender health gap. In 2020, only 5% of global health research funding went to women’s health research. This was split into 4% for women’s cancers and 1% for all other women-specific health conditions, with 25% of that further limited to fertility research (Nature Reviews Bioengineering, 2024).

It’s worth noting this isn’t a one-way problem: in some areas, such as depression, men are underrepresented in clinical data, largely because they’re less likely to seek care, report symptoms, or receive a diagnosis (Smith et al., 2018).

Racial bias is equally well documented

Racial bias in medicine is also extensively documented, particularly in the United States where research shows that, for instance, Black patients and other minority groups receive fewer medical procedures, lower rates of surgical intervention, and fewer referrals to specialists than white patients, regardless of clinical need (Bowser, 2001; Williams & Wyatt, 2015). When AI systems are trained on data that reflects decades of existing inequalities in healthcare access and treatment, they risk encoding and perpetuating those inequalities at a much larger scale. While much of the evidence comes from the US, the structural conditions producing racial bias, including socioeconomic inequalities, barriers to access and underrepresentation in research, are present across Europe too.

LGBTQIA+ patients face their own layer of risk

It is also important to recognise the specific situation of LGBTQIA+ individuals, who experience discrimination in healthcare and are subject to stereotypes that affect the care they receive. These social and cultural factors perpetuate discrimination and have a measurable impact on health and healthcare. Research has shown that 16% of LGBTQIA+ individuals report discrimination in healthcare encounters, and 18% avoid seeking care altogether due to fear of mistreatment (Chang et al., 2025). Another large-scale study found that in emergency department scenarios, AI recommendations for LGBTQIA+ patients included mental health interventions six to seven times more often than was clinically appropriate (Chang et al., 2025).

How this plays out in three key areas

In cardiovascular care, women’s symptoms often differ from the “classic” presentation described in medical textbooks, itself largely based on male patient data (Fatunde et al., 2025). As a result, women are offered fewer diagnostic tests, less medication, and fewer specialist referrals (Al Hamid et al., 2024). Racial bias compounds the problem: pulse oximeters, devices routinely used to measure blood oxygen saturation, have been shown to produce less accurate readings for patients with darker skin tones (Sjoding et al., 2020), a bias that then feeds directly into AI-based triage systems.

In diabetes care, racial bias is extensively documented. Studies have shown that African American patients present systematically higher A1c values than white patients with the same average blood glucose (Karter et al., 2023). If an AI system uses A1c as a proxy for glycaemic control without accounting for this, it risks producing incorrect diagnoses for Black patients.

In depression, both gender and racial bias carry significant weight, especially in tools using natural language processing for screening. Men and women tend to express psychological distress differently (Pennebaker et al., 2003), so a system trained predominantly on one gender’s language patterns may screen inaccurately for the other. Most mental health AI tools also still operate on binary gender assumptions, excluding non-binary, transgender, and gender non-conforming individuals from both the data and the populations these tools are meant to serve (Hafner et al., 2024).

Why this matters

These biases translate into delayed diagnoses, inappropriate treatments, and unequal care, every day, for real patients. Understanding these patterns is exactly what equips healthcare professionals to ask better questions, challenge assumptions, and advocate for the patients most at risk, which is precisely what the AEQUITAS training is designed to do.

References:

Chang, C.T., Srivathsa N., Bou-Khalil, C., Swaminathan, A., Lunn, M.R., Mishra, K., Koyejo, S,. Daneshjou, R. (2025). Evaluating anti-LGBTQIA+ medical bias in large language models. PLOS Digit Health 4(9): e0001001. https://doi.org/10.1371/journal.pdig.0001001

Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A.A., Valencia, A., Rementeria, M.J., Chadha, A.S., & Mavridis, N. (2020). Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. npj Digit. Med. 3(81). https://doi.org/10.1038/s41746-020-0288-5

Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS Digital Health, 3(11), e0000651. https://doi.org/10.1371/journal.pdig.0000651

Funding research on women’s health. (2024). Nature Reviews Bioengineering, 2, 797–798. https://doi.org/10.1038/s44222-024-00253-7

Hafner, F.S., Valdivia, A., Rocher, L. 2025. Gender Trouble in Language Models: An Empirical Audit Guided by Gender Performativity Theory. FAccT ’25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 1677–1695. https://doi.org/10.1145/3715275.3732112

Karter, A. J., Parker, M. M., Moffet, H. H., & Gilliam, L. K. (2023). Racial and Ethnic Differences in the Association Between Mean Glucose and Hemoglobin A1c. Diabetes Technology & Therapeutics, 25(10), 697–704. https://doi.org/10.1089/dia.2023.0153

Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological Aspects of Natural Language Use: Our Words, Our Selves. Annual Review of Psychology, 54(1), 547–577. https://doi.org/10.1146/annurev.psych.54.101601.145041

Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020). Racial Bias in Pulse Oximetry Measurement. New England Journal of Medicine, 383(25), 2477–2478. https://doi.org/10.1056/NEJMc2029240

Smith, D.T., Mouzon, D.M., & Elliott, M. (2018). Reviewing the assumptions about men’s mental health: An exploration of the gender binary. American Journal of Men’s Health, 12(1), 78–89. https://doi.org/10.1177/1557988316630953

Bowser, R. (2001). Racial bias in medical treatment. Dick. L. Rev., 105(3), 365.

Williams, D. R., & Wyatt, R. (2015). Racial Bias in Health Care and Health: Challenges and Opportunities. JAMA, 314(6), 555. https://doi.org/10.1001/jama.2015.9260
The Hidden Ways AI Can Discriminate

When we talk about bias in AI, we are referring to “computer systems that systematically and unfairly discriminate against certain individuals or groups in favour of others” (Friedman & Nissenbaum, 1996). This is not about occasional errors, but about consistent, predictable patterns of unfair outcomes. Furthermore, biases in AI systems are complex, as they can enter the system at almost any stage, from the data it’s trained on to the way it’s ultimately used in a hospital setting. Recognising where bias originates, and how different types can reinforce each other, is the first step towards addressing it.

Three broad roots of bias

Researchers have identified three broad categories of bias that apply to all computer systems. The first is pre-existing bias, which comes from existing inequalities in society that get absorbed into the system, sometimes without anyone realising it (Friedman & Nissenbaum, 1996). The second is technical bias, which arises from the practical compromises made when translating complex human realities into clean computational form. And the third is emergent bias, which appears only after a system is deployed, as the world around it changes in ways the original design never anticipated.

Bias in the AI pipeline

These three categories give us a useful starting point. But for AI and machine learning specifically, researchers have identified seven more precise bias types that can affect these systems throughout their development and use (Suresh & Guttag, 2021):

Historical bias reflects prejudices and stereotypes already present in training data, even if the data is technically accurate. A 1990 study found that after coronary bypass surgery, male patients received pain medication significantly more often than female patients, who were instead given sedatives more frequently (Calderone, 1990). An AI trained on data like this would learn to replicate that same pattern, perpetuating the very inequality it inherited.

Representation bias happens when certain groups are underrepresented in training data. An AI trained mostly on data from one demographic will simply perform worse for everyone else. A well-known example is skin cancer detection tools that are significantly less accurate for patients with darker skin tones because the training datasets contained predominantly images from fair-skinned individuals (Guo et al., 2021).

Measurement bias creeps in when the way something is measured isn’t equally accurate or fair across groups. One striking case involved an algorithm that used healthcare costs as a proxy for how sick someone actually was, and that ended up disadvantaging Black patients, who, facing disproportionate levels of poverty, tended to spend less on healthcare than equally sick white patients (Obermeyer et al., 2019).

Aggregation bias occurs when a single, one-size-fits-all model is applied to genuinely diverse populations, ignoring the fact that the same data point can mean very different things depending on a person’s background.

Learning bias emerges from technical decisions made while building the model itself, choices that can amplify disparities already present in the data, sometimes without the developers being aware. Research has shown, for instance, that differential privacy, a technique meant to protect patient confidentiality, can end up reducing a model’s accuracy for underrepresented groups even further (Bagdasaryan & Shmatikov, 2019).

Evaluation bias happens when the datasets used to test and benchmark the model don’t reflect the real population it will actually serve, meaning a system can look accurate on paper while quietly failing specific groups of patients in practice.

Finally, deployment bias arises when a tool is used in a context very different from the one it was designed and tested for, undermining its reliability in ways that are easy to miss.

Why this matters for patients

Understanding where bias comes from is the first step to catching it, and to making sure the AI tools shaping modern medicine work fairly for everyone, not just the patients who happen to be well represented in the data.

This is exactly the kind of practical, structured understanding the AEQUITAS training equips healthcare professionals with, helping them recognise these patterns before they translate into real harm for real patients.

References:

Bagdasaryan, E., & Shmatikov, V. (2019). Differential Privacy Has Disparate Impact on Model Accuracy (arXiv:1905.12101). arXiv. https://doi.org/10.48550/arXiv.1905.12101

Calderone, K. L. (1990). The influence of gender on the frequency of pain and sedative medication administered to postoperative patients. Sex Roles, 23(11), 713–725. https://doi.org/10.1007/BF00289259

Friedman, B., & Nissenbaum, H. (1996). Bias in computer systems. ACM Transactions on Information Systems, 14(3), 330–347. https://doi.org/10.1145/230538.230561

Guo, L. N., Lee, M. S., Kassamali, B., Mita, C., & Nambudiri, V. E. (2021). Bias in, bias out: Underreporting and underrepresentation of diverse skin types in machine learning research for skin cancer detection-A scoping review. Journal of the American Academy of Dermatology, S0190-9622(21)02086-7. https://doi.org/10.1016/j.jaad.2021.06.884

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342

Suresh, H., & Guttag, J. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO ’21, 1–9. https://doi.org/10.1145/3465416.3483305
Artificial intelligence bias in the prediction and detection of cardiovascular disease

AI can help identify people at risk of cardiovascular disease, detect it early and allow timely intervention. However, bias can arise at any step in the development, validation and use of these algorithms, causing them to perform poorly for historically marginalised groups. This article discusses the sources and consequences of AI bias in predicting and detecting cardiovascular disease, and presents strategies to reduce it.

Bias can come from training data that does not represent the population a tool is meant for, from measurements or devices that work less well for some groups, and from leaving out important factors such as social determinants of health. It can also reflect inequalities in care: because women and ethnic minority patients are more likely to have their cardiovascular disease missed or diagnosed late, algorithms trained on this data may be less accurate at detecting it in these groups. Unequal access to digital technologies can further limit how well some groups are represented in the data. The authors give several examples of AI models that performed worse for women, Black patients or older adults.

To reduce bias, the authors propose an AI health equity framework covering the whole lifecycle of an algorithm: diverse research teams, early involvement of patients, representative and balanced data, the inclusion of social determinants of health, input from clinicians, validation across different groups, transparent reporting of results and monitoring after implementation. They conclude that, without these measures, AI bias could worsen existing healthcare inequities and miss opportunities to predict and detect cardiovascular disease.

Learn more about this article here: https://doi.org/10.1038/s44325-024-00031-9

Reference:

Mihan, A., Pandey, A., & Van Spall, H.G.C. (2024). Artificial intelligence bias in the prediction and detection of cardiovascular disease. npj Cardiovascular Health, 1, 31.
AI Meets Healthcare: Understanding the Promise and the Risks

Artificial intelligence is no longer a distant, futuristic concept in medicine. It’s already here, and is becoming an everyday part of clinical practice, supporting diagnosis, clinical decision-making, treatment planning, and patient risk prediction. In fact, according to the World Health Organization, nearly three quarters of EU countries are now using AI-assisted diagnostics in some form, and that number is only expected to grow (Alowais et al., 2023; World Health Organization, 2026).

As the use of AI promises to further revolutionize the sector, what should we know before trusting it with our health?

How AI works

At its core, biomedical AI relies on three main approaches. Machine learning allows systems to learn patterns from data and make predictions without being explicitly programmed for every scenario. Deep learning, a more advanced form of machine learning, uses layered neural networks (loosely inspired by the human brain) to process complex information like medical images or clinical notes. And natural language processing (NLP) enables computers to understand and interpret human language, such as a doctor’s written notes or a patient’s description of their symptoms.

Real examples already in use

In cardiovascular care, AI tools are helping detect arrhythmias from ECG readings and estimating a patient’s individual risk of heart disease. In diabetes management, AI-powered retinal scans can catch signs of diabetic retinopathy earlier than ever before. And in mental health, natural language processing tools are being explored to help identify signs of depression in the way people write or speak, and to support patients between therapy sessions.

So what’s the catch?

Research has shown that AI applications can lead to misinterpretation and the promotion of biases and discrimination (Varona & Suarez, 2022). In essence, AI systems are only as good as the data they learn from. If that data reflects existing inequalities, for example, if it comes mostly from male patients, or predominantly from one ethnic group, the AI will inherit those biases. It will perform well for the people it “knows,” and poorly for everyone else.

Several documented cases have shown AI tools trained on biased datasets producing less accurate results for underrepresented patients, sometimes with real consequences for diagnosis and treatment.

What AEQUITAS is doing about it

This is exactly the challenge our AEQUITAS project, funded by the European Union, was designed to address. By developing training resources, a database documenting real cases of bias in biomedical AI, and a practical regulatory model for hospitals and CSOs, AEQUITAS is helping healthcare professionals and institutions across Europe use these powerful tools more safely, and more fairly, for every patient.

Understanding how AI works, and where it can go wrong, is the first step toward making sure it works for everyone.

References:

Alowais, S.A., Alghamdi, S.S., Alsuhebany, N. et al. Revolutionizing healthcare: the role of artificial intelligence in clinical practice. BMC Med Educ 23, 689 (2023). https://doi.org/10.1186/s12909-023-04698-z

Varona, D., & Suárez, J. L. (2022). Discrimination, Bias, Fairness, and Trustworthy AI. Applied Sciences, 12(12), 5826. https://doi.org/10.3390/app12125826

World Health Organization. 2026. “Artificial intelligence is reshaping health systems: state of readiness across the European Union.” World Health Organization.
The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

AI health algorithms are increasingly used to predict risk and support treatment decisions, often outperforming humans in diagnostic accuracy. However, they can also underestimate risk for marginalised and underserved patients, delaying the treatment they need. This scoping review analysed 23 sources published between 2020 and 2024 to examine racial and ethnic bias in these algorithms, the role of oversight and stakeholders, and how this bias can be reduced.

A common explanation for this bias is that algorithms are trained on data in which majority populations are overrepresented, making them less reliable for others. Black, Native American, Asian and Latinx populations are often underrepresented in health datasets. Bias can also come from people: racial stereotypes absorbed during medical training can affect how clinicians assess patients, and these assessments end up in the health records used to train algorithms. The authors also describe a feedback loop: when patients from marginalised groups distrust the health system and use it less, data about them becomes even less representative, and the algorithms less accurate for them.

To address this, the review highlights the need for continuous monitoring of algorithms once they are in use, in a similar way to how medicines are monitored after approval, and for retraining or withdrawing those that create inequities. It also stresses that simply including or removing race from an algorithm does not solve the problem, and that responsibility should be shared by developers, health systems, clinicians, regulators and patients. The authors conclude that, without this oversight, AI in healthcare risks entrenching health inequities.

Learn more about this review here: https://doi.org/10.1080/13557858.2024.2422848

Reference:

Hussain, S.A., Bresnahan, M., & Zhuang, J. (2025). The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review. Ethnicity & Health, 30(2), 197–214.
Gender Bias in Diagnosis, Prevention, and Treatment of Cardiovascular Diseases: A Systematic Review

Cardiovascular disease is a leading cause of death for both women and men, but there is growing evidence that women are not always diagnosed and treated in the same way. This systematic review analysed 19 studies to examine gender bias in the diagnosis, prevention and treatment of cardiovascular diseases.

The review found that women’s cardiovascular disease was less often recorded when their symptoms were milder than men’s, or when they were misdiagnosed as digestive or anxiety-related problems. Women were offered fewer diagnostic tests, such as electrocardiograms and coronary angiography, were less often referred to cardiologists or admitted to hospital, and, once in hospital, were less likely to receive coronary interventions. They were also prescribed cardiovascular medicines less often than men, except for medicines for high blood pressure and angina, and saw themselves as being at lower risk of cardiovascular disease.

The authors conclude that women receive fewer diagnostic tests and medicines than men, and link these differences to limited knowledge of how cardiovascular disease presents differently in women and men.

Learn more about this review here: https://doi.org/10.7759/cureus.54264

Reference:

Al Hamid, A., Beckett, R., Wilson, M., Jalal, Z., Cheema, E., Al-Jumeily, D., Coombs, T., Ralebitso-Senior, K., & Assi, S. (2024). Gender Bias in Diagnosis, Prevention, and Treatment of Cardiovascular Diseases: A Systematic Review. Cureus, 16(2), e54264.
Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases

AI models are increasingly used in cardiovascular medicine, but there are concerns that they may not work equally well for people of different racial and ethnic backgrounds. This systematic review examined how accurate and clinically applicable AI models for diagnosing heart disease and predicting outcomes are across racial and ethnic groups.

Of 1,704 references screened, 11 studies were included, covering diagnosis, prognosis and the analysis of heart images. Nine of them (82%) found racial or ethnic bias, while two found no differences between groups. For example, an AI model predicting heart failure from electrocardiograms was less accurate for young Black patients than for young Asian or White patients, and AI tools analysing cardiac MRI scans were more accurate for White people than for people from minority ethnic groups.

The authors conclude that AI models can perform well for some groups and less accurately for others, and that bias should be addressed at every stage, from data collection to clinical use. They recommend methods to reduce bias, standardised ways of measuring fairness, considering social determinants of health rather than race alone, and greater transparency through the sharing of code and data.

Learn more about this review here: https://doi.org/10.1016/j.ejrad.2024.111867

Reference:

Cau, R., Pisu, F., Suri, J.S., & Saba, L. (2024). Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases. European Journal of Radiology, 111867.
Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity

AI has been seen as a tool that could help detect and reduce the racial and ethnic inequities that persist in cardiovascular care. However, AI can also carry and spread biases of its own, which means it must be approached with care. This review looks at the role of AI in cardiology in identifying and addressing these inequities, and in promoting diversity in research, as well as the concerns its use raises and strategies for using it fairly.

The authors describe how AI has already helped reduce some inequities. In one study, an AI tool identified patients with severe heart valve disease and alerted their doctors, improving follow-up care for patients of all races. Even so, Black patients still faced greater delays in receiving appropriate care than White patients, showing that AI is not a “silver bullet” for eliminating inequities. AI can also perpetuate existing biases, due to limited diversity in training data, biases within the datasets themselves, and inadequate mechanisms for detecting and monitoring bias. For example, an AI model trained mostly on data from White patients to predict heart failure from electrocardiograms performed worse for Black patients, and particularly for Black women.

To use AI in cardiovascular care in a way that addresses racial and ethnic inequities, the authors stress the need for careful design, starting with meticulous data collection and a thorough review of training data. They also call for rigorous testing and continuous monitoring of algorithms to ensure they perform equally well for all groups, as well as greater diversity in the AI workforce and the involvement of stakeholders.

Learn more about this review here: https://doi.org/10.1007/s12170-024-00745-6

Reference:

Amponsah, D., Thamman, R., Brandt, E., James, C., Spector-Bagdady, K., & Yong, C.M. (2024). Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity. Current Cardiovascular Risk Reports, 18(11), 153–162.
Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias

AI has transformed diabetes management, particularly through tools that predict glucose levels and support clinical decisions. However, there are concerns that these technologies may not work equally well for different racial, ethnic and socioeconomic groups. This systematic review assessed how fair AI-based glucose prediction models are, focusing on differences in performance between groups, how fairness is reported, and biases linked to physiological measures.

Of 1,243 articles initially identified, only 10 met the criteria for inclusion. Just one of them explicitly evaluated whether AI model performance differed by race. Reporting on fairness was rare: only 7% of AI diabetes studies documented ethnic or racial data, and almost none carried out fairness audits. The review also highlights known physiological differences: for the same average glucose level, African American patients tend to have higher HbA1c values than White patients. If HbA1c is used as a reference to train AI models, this difference can be built into them. Yet none of the AI studies reviewed took this into account.

The authors also note that traditional diabetes risk scores overestimate risk for White individuals and underestimate it for Black individuals, a bias that AI systems trained on these scores could inherit. They conclude that, without systematic efforts to reduce bias, AI-based diabetes technologies risk worsening existing health disparities, and call for future research to prioritise transparency, representativeness and fairness.

Learn more about this review here: https://doi.org/10.61186/ist.202502.05.04

Reference:

Alipour, M., & Alipour, A. (2025). Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias. InfoScience Trends, 2(5), 33–46.
Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions

Machine learning models can help predict which patients with diabetes are likely to be readmitted to hospital, but they need to work fairly for all patients. This study compared several of these models, including deep learning, generalised linear models, gradient boosting machines and naive Bayes, assessing not only their accuracy but also how fair their predictions were across age, gender and race.

The gradient boosting model performed best overall, with an accuracy of 82.2% and consistently low error rates for most racial and gender groups. However, across several of the models, African American and Hispanic patients had higher rates of false positives, meaning they were more likely to be wrongly predicted to be readmitted. No single model performed best for every group: some worked better for certain groups of patients and others for other groups.

The authors conclude that there is no one-size-fits-all solution, and that models need to be chosen and adjusted carefully to ensure they are both accurate and fair for all patients. According to them, fair algorithms can help reduce disparities and improve outcomes for patients with diabetes from all backgrounds.

Learn more about this study here: https://doi.org/10.1109/ICHI61247.2024.00105

Reference:

Al-Zanbouri, Z., Sharma, G., & Raza, S. (2024). Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions. In 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI) (pp. 660–669). IEEE.
Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study

Large language models such as GPT-4 are seen as promising tools in health care, from automating administrative tasks to supporting clinical decisions. However, they also risk perpetuating biases and giving incorrect diagnoses, with a direct impact on patient care. This study assessed whether GPT-4 shows racial and gender biases in four possible uses: medical education, diagnostic reasoning, treatment planning and assessments of patients.

The researchers used clinical cases from medical education resources and from published research on implicit bias in health care. When asked to create clinical cases, GPT-4 did not reflect the real demographic diversity of medical conditions, and consistently produced cases that stereotyped how conditions present in different groups. For the same clinical cases, its lists of possible diagnoses were more likely to include diagnoses that stereotype certain races, ethnicities and genders.

The treatment plans and assessments it produced also varied with patients’ demographic characteristics, including recommendations for more expensive procedures and differences in how patients were perceived. The authors conclude that tools like GPT-4 urgently need thorough and transparent bias assessments for each intended use before they are introduced into clinical care.

Learn more about this study here: https://doi.org/10.1016/S2589-7500(23)00225-X

Reference:

Zack, T., Lehman, E., Suzgun, M., Rodriguez, J.A., Celi, L.A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D.W., Abdulnour, R.-E.E., Butte, A.J., & Alsentzer, E. (2024). Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health, 6(1), e12–e22.
Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes

Continuous glucose monitors, small sensors worn on the body that track blood sugar levels throughout the day, are increasingly combined with AI models that predict where glucose levels are heading. Because White people are overrepresented in medical research, these models may perform worse for other groups if they are trained mainly on White patients’ data. This study tested this using data from 205 adults and children with type 1 diabetes in the US, 101 White and 104 Black.

The researchers trained AI models to predict glucose levels 60 minutes ahead, using datasets with different proportions of White and Black participants, from 0% to 100% White. As the proportion of White participants in the training data increased, prediction errors grew slightly more for Black participants than for White participants. The difference was small but statistically significant.

When the models were then personalised to each individual, using a technique called transfer learning, this difference became smaller and was no longer statistically significant. The authors note that the differences may not be large enough to matter clinically on their own, but conclude that diverse training data is important for fair prediction models, and that personalisation may help build fairer tools.

Learn more about this study here: https://doi.org/10.1371/journal.pdig.0000918

Reference:

Thomsen, H.B., Li, L.Y., Isaksen, A.A., Lebiecka-Johansen, B., Bour, C., Fagherazzi, G., van Doorn, W.P.T.M., Varga, T.V., & Hulman, A. (2025). Racial disparities in continuous glucose monitoring-based 60-min glucose predictions among people with type 1 diabetes. PLOS Digital Health, 4(6), e0000918.
Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia

Diabetic retinopathy, damage to the retina caused by diabetes, can lead to vision loss if it is not detected early. This study tested an AI system for diabetic retinopathy screening in real-world conditions, in two endocrinology clinics in Melbourne and three Aboriginal medical services in Western Australia, and also gathered the views of patients and health professionals.

Of the 236 people screened, 203 were included in the main analysis. Overall, the system identified diabetic retinopathy requiring referral with a sensitivity of 96.9% and a specificity of 87.7%, and over 93% of participants were satisfied with the service. However, specificity was substantially lower among participants recruited from Aboriginal medical services, meaning that more people without the condition were wrongly flagged for referral. The study used clinic location as an approximation of Indigenous status.

The authors note that the AI system had been trained exclusively on images from Chinese adults, and that differences in retinal pigmentation may affect its accuracy. They also point to other possible factors, such as poorer image quality, older age and other eye conditions. These explanations were not tested in the study.

Learn more about this study here: https://doi.org/10.1038/s41598-021-94178-5

Reference:

Scheetz, J., Koca, D., McGuinness, M., Holloway, E., Tan, Z., Zhu, Z., O’Day, R., Sandhu, S., MacIsaac, R.J., Gilfillan, C., Turner, A., Keel, S., & He, M. (2021). Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia. Scientific Reports, 11, 15808.
Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis

AI models developed using data from very similar populations may not work as well for other groups, and could reflect or even worsen existing racial and ethnic disparities in health care. This study tested whether this was the case for an AI model that uses electrocardiograms (ECGs) to detect a weakened heart pumping function, known as low left ventricular ejection fraction.

The model had been developed with data from nearly 45,000 patients, 96.2% of whom were non-Hispanic White. When tested on a separate group of almost 53,000 patients, it performed consistently well across all the racial and ethnic groups analysed: non-Hispanic White, Asian, Black/African American, Hispanic/Latino and American Indian/Native Alaskan. A separate AI model was able to identify patients’ racial group from the ECG to some extent, showing that ECG characteristics do vary by race, but this did not affect the first model’s performance.

The authors conclude that, in this case, racial differences in the ECG did not affect the model’s ability to detect the condition. Even so, they recommend that the performance of all new AI tools in medicine be reported across different ethnic, racial, age and sex groups, to ensure they are used responsibly.

Learn more about this study here: https://doi.org/10.1161/CIRCEP.119.007988

Reference:

Noseworthy, P.A., Attia, Z.I., Brewer, L.C., Hayes, S.N., Yao, X., Kapa, S., Friedman, P.A., & Lopez-Jimenez, F. (2020). Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis. Circulation: Arrhythmia and Electrophysiology, 13(3), e007988.
Fairness And Bias Mitigation in AI Models for Diabetes Diagnosis: A Comparative Evaluation of Algorithmic Approaches

AI models that predict diabetes can produce biased results for certain groups of patients, and several methods have been developed to reduce this. This study compared three of these methods on the same data, a widely used dataset of 768 women aged 21 and over, assessing fairness across age groups.

Each method had different strengths. The first relied on simple, transparent decision rules, making its predictions easy to interpret, but it had no specific mechanism to ensure fairness. The second was designed to prevent the model from basing its predictions on age, and achieved perfect results on the fairness measures used while keeping accuracy at an acceptable level. The third corrected the imbalance between patients with and without diabetes in the data, which gave the highest overall accuracy but did not ensure fairness.

The authors conclude that none of the methods achieves the right balance between accuracy, fairness and interpretability on its own, and propose combining several strategies to reduce bias in AI-based diabetes diagnosis.

Learn more about this study here: https://doi.org/10.54554/jtec.2025.17.03.004

Reference:

Mohd Hamdan, M.D.H., Ab Jabal, M.F., Abdul Rahman, S., & Kapi, A.Y. (2025). Fairness And Bias Mitigation in AI Models for Diabetes Diagnosis: A Comparative Evaluation of Algorithmic Approaches. Journal of Telecommunication, Electronic and Computer Engineering, 17(3), 27–34.
Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images

AI can accurately diagnose coronary artery disease from myocardial perfusion imaging, a type of scan that shows blood flow to the heart muscle. However, these models are often trained on high-risk patients who were referred for invasive tests, which can lead them to overestimate the probability of disease in other patients. This study tested ways of training AI so that the probabilities it estimates match the real probability of disease more closely.

Deep learning models were trained on 828 patients from three sites and then tested on a separate group of 511 patients, including 179 with a low likelihood of disease. The best results came from a model trained with additional examples of patients without coronary artery disease and with little or no sign of reduced blood flow on the scan. It produced more accurate probability estimates than the model trained without these additional cases, and the improvement was most pronounced in women. It was also better at identifying disease than standard measures taken from the scans.

The authors conclude that adding more low-risk patients during training can improve how accurately AI estimates the probability of disease. This matters especially for lower-risk patients and for women, for whom overestimated probabilities could affect decisions about further tests and treatment.

Learn more about this study here: https://doi.org/10.1007/s00259-022-05972-w

Reference:

Miller, R.J.H., Singh, A., Otaki, Y., Tamarappoo, B.K., Kavanagh, P., Parekh, T., Hu, L.-H., Gransar, H., Sharir, T., Einstein, A.J., Fish, M.B., Ruddy, T.D., Kaufmann, P.A., Sinusas, A.J., Miller, E.J., Bateman, T.M., Dorbala, S., Di Carli, M.F., Liang, J.X., … Slomka, P.J. (2023). Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images. European Journal of Nuclear Medicine and Molecular Imaging, 50(2), 387–397.
Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction

Machine learning models are increasingly used to estimate a person’s risk of cardiovascular disease, but bias can be introduced when health data is collected and prepared, affecting how well these models work for certain groups. This study used electronic health records from 109,490 people treated at Vanderbilt University Medical Center, in the US, to test whether several machine learning models performed equally well across race and gender. It also compared them with a widely used risk equation from the American Heart Association that does not use machine learning.

Most of the machine learning models were less biased than the American Heart Association equation. Differences between racial groups were small, but differences between women and men were considerably larger. According to the authors, both the risk equation and the machine learning models were biased against women.

The researchers also tested three ways of reducing this bias. Removing race and gender from the data did not significantly reduce bias in most models, and balancing the number of people from each group in the training data did not consistently help either. Balancing the proportion of people with cardiovascular disease in each group reduced the gender bias, but not the racial bias, and slightly reduced accuracy in many cases. The authors conclude that gender disparities in cardiovascular risk prediction need to be investigated and corrected.

Learn more about this study here: https://doi.org/10.1016/j.jbi.2023.104294

Reference:

Li, F., Wu, P., Ong, H.H., Peterson, J.F., Wei, W.-Q., & Zhao, J. (2023). Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction. Journal of Biomedical Informatics, 138, 104294.
Depression among people with dyspepsia and H. pylori infection: A community based cross-sectional study in Ethiopia

A 2022 community-based study in southwest Ethiopia explored potential links between Helicobacter pylori infection, dyspepsia, and depression in residents aged 13 years and older.

Among the 871 participants – most of whom were married, had no formal education, and lived in rural areas – around 11% showed signs of probable depression. The prevalence was slightly higher (13%) among those experiencing at least one symptom of dyspepsia. Interestingly, in this study, H. pylori infection alone was not linked to higher depression rates, nor were age or gender.

The findings suggest digestive symptoms may increase the risk of depression, highlighting the importance of addressing both mental and gut health when managing gastrointestinal symptoms.

Learn more about this study here: https://doi.org/10.1371/journal.pone.0275424

Reference

Soboka, M., Gudina, E. K., Gashaw, M., Amare, H., Berhane, M., Desalegn, H., Tewolde, D., Jebena, M. G., Ali, S., Wieser, A., Froeschl, G., & Tesfaye, M. (2022). Depression among people with dyspepsia and H. pylori infection: A community based cross-sectional study in Ethiopia. PloS one, 17(10), e0275424.
Digital health tools for the passive monitoring of depression: a systematic review of methods

This systematic review examines studies linking passive data from smartphones and wearables to depression, identifying key methodological flaws and threats to reproducibility. It highlights biases such as representation, measurement, and evaluation bias, stemming from small, homogenous samples and inconsistent feature construction.

Although gender and race are not explicitly discussed, the lack of diversity in study populations suggests potential demographic bias. The review calls for improved reporting standards and broader sample inclusion to enhance generalizability and clinical relevance. These improvements are essential for ensuring that digital mental health tools are equitable and reliable across diverse populations.

Learn more about this review here: https://doi.org/10.1038/s41746-021-00548-8

Reference

De Angel, V., Lewis, S., White, K., Oetzmann, C., Leightley, D., Oprea, E., Lavelle, G., Matcham, F., Pace, A., Mohr, D. C., Dobson, R., & Hotopf, M. (2022). Digital health tools for the passive monitoring of depression: a systematic review of methods. NPJ digital medicine, 5(1), 3.
Rapamycin Attenuates Anxiety and Depressive Behavior Induced by Helicobacter pylori in Association with Reduced Circulating Levels of Ghrelin

In this experimental study, researchers investigated how H. pylori infection influences depression-like behavior and certain biological markers in mice, focusing especially on the hormone ghrelin.

Mice infected with the bacteria showed more anxiety- and depression-like behaviors in standard laboratory tests compared with healthy mice. They moved less, showed more signs of stress, and had lower preference for sweet solutions, a common indicator of loss of pleasure.
Biologically, H. pylori infection led to reduced circulating ghrelin levels and activation of the mTOR signaling pathway in the stomach. It was also associated with increased markers of brain inflammation and cell injury in the hippocampus, a brain region important for mood regulation. When mice were treated with rapamycin, an mTOR inhibitor, ghrelin levels increased and brain-inflammation markers were reduced.

Overall, the study suggests that H. pylori infection may contribute to anxiety- and depression-like behaviors by lowering ghrelin levels and increasing neuroinflammation, highlighting a potential biological link between gut infection and mental health.

Learn more about this study here: https://doi.org/10.1155/2022/2847672

Reference

Tian, Jiageng, Wang, Zeyu, Ren, Yadi, Jiang, Yong, Zhao, Ying, Li, Man, Zhang, Zhiguang, Rapamycin Attenuates Anxiety and Depressive Behavior Induced by Helicobacter pylori in Association with Reduced Circulating Levels of Ghrelin, Neural Plasticity, 2022, 2847672, 8 pages, 2022.
Bias Discovery in Machine Learning Models for Mental Health

This article examined how AI can unintentionally reproduce social and demographic biases when applied to mental health prediction. Using benzodiazepine prescriptions as a proxy for conditions such as depression and anxiety, a study analyzed machine learning models trained on patient data to identify systematic disparities.

It found that women are more frequently predicted to receive such treatments, reflecting gender bias, while the models perform less accurately for minority ethnic groups, indicating representation and evaluation bias. The AI models here are not used to prescribe drugs but rather to predict treatment likelihoods, revealing how bias in healthcare data can lead to inequitable AI performance in the context of depression-related care.

Learn more about the article here: https://doi.org/10.3390/info13050237

Reference

Mosteiro, P.J., Kuiper, J., Masthoff, J., Scheepers, F., & Spruit, M. (2022). Bias Discovery in Machine Learning Models for Mental Health. Inf., 13, 237.
Increased risk of short-term depressive disorder after Helicobacter pylori eradication: A population-based nested cohort study

A study using Taiwan’s National Health Insurance data found that antibiotic therapy for H. pylori in patients with peptic ulcer disease was linked to a short-term increase in psychiatrist-diagnosed depression within 30 days. Women and patients treated with clarithromycin were particularly at higher risk.

The researchers reported that the increased risk of depression after H. pylori eradication therapy may involve alterations in the brain-gut-microbiome axis induced by antibiotic treatment, as is it well known that antibiotics can change the gut microbial composition, metabolism, and function, thereby affecting human health and possibly contributing to the pathophysiology of depression.

Based on these findings, the authors recommend that clinicians should monitor mental health shortly after H. pylori eradication, as short-term depressive symptoms may occur and be easily overlooked.

Learn more about this study here: https://doi.org/10.1111/hel.12824

Reference

Tsai C-F, Chen M-H, Wang Y-P, et al. Increased risk of short-term depressive disorder after Helicobacter pylori eradication: A population-based nested cohort study. Helicobacter. 2021; 26:e12824.
GWAS of peptic ulcer disease implicates Helicobacter pylori infection, other gastrointestinal disorders and depression

A study of over 450,000 people in the UK Biobank identified 8 independent genes that affect stomach acid, gut movement, and the body’s response to infection, including susceptibility to Helicobacter pylori infection.

The study also explored connections between these gut conditions and depression, which often occurs alongside digestive problems, providing new insights into the complex interplay between gut health and mental well-being.

Learn more about this study here: https://doi.org/10.1038/s41467-021-21280-7

Reference

Wu, Y., Murray, G.K., Byrne, E.M. et al. GWAS of peptic ulcer disease implicates Helicobacter pylori infection, other gastrointestinal disorders and depression. Nat Commun 12, 1146 (2021)
Helicobacter pylori Infection Is Associated with Long-Term Cognitive Decline in Older Adults: A Two-Year Follow-Up Study

Helicobacter pylori infection is usually known for causing stomach problems, but it may also affect brain health. This research study published in 2023 followed 268 older adults with memory complaints for two years to see whether H. pylori infection was linked to cognitive decline.

While at the beginning of the study, people with and without H. pylori performed similarly on memory tests, over the two-year follow-up, those with a history of infection showed greater declines in their Mini-Mental State Examination (MMSE) scores. After taking into account age, sex, education, genetic risk factors and common medical conditions, H. pylori infection was still associated with a significantly higher risk of cognitive decline, with infected participants more likely to lose three or more MMSE points and showing a faster rate of decline over time.

These findings suggest that H. pylori infection may contribute to progressive cognitive deterioration in older adults with memory complaints and may be relevant in understanding pathways linking infection and dementia.

Learn more about this study here: https://doi.org/10.3233/JAD-221112

Reference

Wang J, Yu N-W, Wang D-Z, et al. Helicobacter pylori Infection Is Associated with Long-Term Cognitive Decline in Older Adults: A Two-Year Follow-Up Study. Journal of Alzheimer’s Disease. 2023;91(4):1351-1358.
Race, Sex, and Age Disparities in the Performance of ECG Deep Learning Models Predicting Heart Failure

AI models that analyse electrocardiograms (ECGs) could help identify people at high risk of heart failure early, and potentially reduce racial disparities in heart failure outcomes. But whether these models perform equally well for different groups of patients has been little studied. This study used ECGs from 326,518 patient encounters at Stanford Hospital to evaluate a model trained to predict heart failure within five years, comparing its performance by race, ethnicity, age and sex.

Overall, there were no significant differences in performance between racial groups, and performance declined with age. However, among patients aged 0 to 40, the model performed significantly worse for Black patients than for any other racial group, with the largest differences among young Black women.

The researchers tested several ways of reducing these disparities: including race, ethnicity, sex and age in the model, training separate models for each racial group, and training it on data with equal racial representation. None of these improved the disparities. Adjusting the model’s decision thresholds individually by race, age and sex did lead to substantial improvements. The authors warn that machine learning tools for heart failure risk perpetuating existing disparities, and suggest these individualised thresholds as one way to reduce them.

Learn more about this study here: https://doi.org/10.1161/CIRCHEARTFAILURE.123.010879

Reference:

Kaur, D., Hughes, J.W., Rogers, A.J., Kang, G., Narayan, S.M., Ashley, E.A., & Perez, M.V. (2024). Race, Sex, and Age Disparities in the Performance of ECG Deep Learning Models Predicting Heart Failure. Circulation: Heart Failure, 17(1), e010879.
Predictive Accuracy of Stroke Risk Prediction Models Across Black and White Race, Sex, and Age Groups

Stroke is a leading cause of death and serious long-term disability in the US, with a particularly high risk among Black individuals. Risk prediction tools help identify who would benefit most from prevention, but they need to work equally well for everyone. This study compared several of these tools, including two stroke-specific models, a general cardiovascular risk model and two new machine learning models, using data from 62,482 Black and White adults aged 45 and over from four large US studies.

Overall, the models performed similarly, and the new machine learning models did not improve their accuracy. However, all of them were worse at distinguishing between people who would and would not have a stroke when applied to Black individuals. Among women, the models’ accuracy score was 0.76 for White women and 0.69 for Black women. Among men, it ranged from 0.71 to 0.72 for White men and from 0.64 to 0.66 for Black men. The models also performed better in adults under 60 than in older adults.

The authors conclude that current approaches are not enough to close this gap, and that the range of risk factors considered and the modelling techniques used need to be improved to address these racial disparities.

Learn more about this study here: https://doi.org/10.1001/jama.2022.24683

Reference:

Hong, C., Pencina, M.J., Wojdyla, D.M., Hall, J.L., Judd, S.E., Cary, M., Engelhard, M.M., Berchuck, S., Xian, Y., D’Agostino, R., Howard, G., Kissela, B., & Henao, R. (2023). Predictive Accuracy of Stroke Risk Prediction Models Across Black and White Race, Sex, and Age Groups. JAMA, 329(4), 306–317.
Confounders mediate AI prediction of demographics in medical imaging

Previous research has suggested that AI can identify a patient’s age, sex and even race from medical images. This study tested whether this holds for echocardiograms, ultrasound videos of the heart, using more than 530,000 videos from two US medical centres, Cedars-Sinai and Stanford.

The models predicted sex well and age with an average error of about 9 years, and these results held across both centres. Race, however, could not be reliably predicted. When the researchers changed the balance of age or sex in the data used to train the race model, its performance varied widely, while changing the racial composition of the data had little effect on the predictions of sex and age.

The authors conclude that much of the model’s apparent ability to predict race comes from “shortcuts” through other features that differ between racial groups, rather than from race itself. They warn that treating AI predictions of race as evidence of biological differences risks reinforcing a false idea that has contributed to medical errors, since differences in health between racial groups often reflect social factors and structural racism. They call for further research into which imaging features carry demographic information, and into the risks of bias this creates in medical AI.

Learn more about this study here: https://doi.org/10.1038/s41746-022-00720-8

Reference:

Duffy, G., Clarke, S.L., Christensen, M., He, B., Yuan, N., Cheng, S., & Ouyang, D. (2022). Confounders mediate AI prediction of demographics in medical imaging. npj Digital Medicine, 5, 188.
Addressing Artificial Intelligence Bias in Retinal Diagnostics

Diabetic retinopathy, damage to the retina caused by diabetes, is one of the conditions in which AI is used to support diagnosis by analysing images of the back of the eye. This study looked at what happens when the data used to train these systems is unbalanced, and tested a way of reducing the resulting bias.

The researchers used a public dataset of retinal images to which clinicians had added skin-tone labels. To recreate a situation of imbalance, they removed from the training data the images of people with darker skin who had diabetic retinopathy requiring referral to a specialist, while keeping these images in the test data. The standard system was 73.0% accurate for people with lighter skin and 60.5% for people with darker skin, a gap of 12.5 percentage points.

They then trained the system with additional synthetic retinal images created by generative AI models. Accuracy became 72.0% for people with lighter skin and 71.5% for people with darker skin, reducing the gap to 0.5 points. The authors conclude that unbalanced training data can lead to unequal accuracy across groups, and that synthetic images may help reduce this bias in the diagnosis of diabetic retinopathy and possibly in other AI systems used in eye care.

Learn more about this study here: https://doi.org/10.1167/tvst.10.2.13

Reference:

Burlina, P., Joshi, N., Paul, W., Pacheco, K.D., & Bressler, N.M. (2021). Addressing Artificial Intelligence Bias in Retinal Diagnostics. Translational Vision Science & Technology, 10(2), 13.
Age, sex and race bias in automated arrhythmia detectors

Algorithms that read electrocardiograms (ECGs) automatically are increasingly used to help identify heart conditions, but few studies have examined whether they perform equally well for different groups of patients. This study compared 56 open-source algorithms, developed by participants in the 2021 PhysioNet Challenge, on more than 130,000 ECGs, grouping the results by sex, age and race.

Several performance measures were similar across groups, but others showed clear differences. Accuracy was significantly lower for Black patients and higher for Asian patients, with a gap of 39% on the test set. Accuracy also fell with age, by around 12% per decade, and results for male and female patients differed depending on the measure used.

The researchers then retrained one of the best-performing models with an additional constraint designed to reduce performance differences across sex, race and age. This led to a small loss in overall performance but a significant reduction in bias. The authors conclude that bias depends on the model’s design, the population it is trained on and how its performance is measured, and that all of these should be closely examined.

Learn more about this study here: https://doi.org/10.1016/j.jelectrocard.2022.07.007

Reference:

Perez Alday, E.A., Rad, A.B., Reyna, M.A., Sadr, N., Gu, A., Li, Q., Dumitru, M., Xue, J., Albert, D., Sameni, R., & Clifford, G.D. (2022). Age, sex and race bias in automated arrhythmia detectors. Journal of Electrocardiology, 74, 5–9.
Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases

Electrocardiography (ECG) enhanced by artificial intelligence is emerging as a promising tool for diagnosing and managing cardiovascular disease. This review explains how AI can detect subtle patterns in the ECG that human interpreters may miss, helping to identify problems such as arrhythmias and heart failure more quickly and consistently.

However, the authors also highlight important limitations, including systematic bias. When training datasets do not adequately represent different ages, sexes and racial groups, models can perform worse for those who are underrepresented, since ECG patterns vary with age, physiological differences between the sexes and genetic factors. To make these tools safe and fair in clinical practice, the review calls for more diverse datasets, bias mitigation strategies and continuous validation across populations.

Learn more about this review here: https://doi.org/10.1016/j.jelectrocard.2024.01.006

Reference:

Muzammil, M.A., Javid, S., Afridi, A.K., Siddineni, R., Shahabi, M., Haseeb, M., Fariha, F., Kumar, S., Zaveri, S., & Nashwan, A.J. (2024). Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases. Journal of Electrocardiology, 83, 30–4
Cardiovascular Disease Screening in Women: Leveraging Artificial Intelligence and Digital Tools

Cardiovascular disease remains the leading cause of death in women, and there is growing evidence of sex- and gender-based differences in how it develops and progresses. This review provides an overview of the main opportunities for cardiovascular screening in women and of the potential role of artificial intelligence and digital tools.

The authors note that healthcare providers, guideline committees and policymakers are not yet well equipped to collect data on women, interpret it, or use it to design targeted interventions. The review includes a section on bias in AI and looks at different stages of a woman’s life, from youth and pregnancy to menopause and older age, as well as the implications of AI for clinical guidelines and public health.

Learn more about this review here: https://doi.org/10.1161/CIRCRESAHA.121.319876

References:

Adedinsewo, D.A., Pollak, A.W., Phillips, S.D., Smith, T.L., Svatikova, A., Hayes, S.N., Mulvagh, S.L., Norris, C., Roger, V.L., Noseworthy, P.A., Yao, X., & Carter, R.E. (2022). Cardiovascular Disease Screening in Women: Leveraging Artificial Intelligence and Digital Tools. Circulation Research, 130(4), 673–690.
Implicit gender bias in the diagnosis and treatment of type 2 diabetes: A randomized online study

This study examined whether implicit gender biases, unconscious associations about people based on their gender, influence how general practitioners (GPs) diagnose and treat type 2 diabetes. Ninety-nine GPs completed tests measuring two biases, one related to lifestyle (women having a healthier lifestyle than men) and one related to communication (men being less communicative than women), and then assessed clinical cases featuring a male or female patient.

Female GPs showed a significant lifestyle-related bias, while GPs of both genders showed a communication-related bias. The patient’s gender also affected clinical decisions: for example, GPs were less certain about the diabetes diagnosis when the patient was a woman.

These findings show how unconscious bias can shape clinical decision-making.

Learn more about this study here: https://doi.org/10.1111/dme.15087

Reference:

Skvortsova, A., Meeuwis, S.H., Vos, R.C., Vos, H.M.M., van Middendorp, H., Veldhuijzen, D.S., & Evers, A.W.M. (2023). Implicit gender bias in the diagnosis and treatment of type 2 diabetes: A randomized online study. Diabetic Medicine, 40(8), e15087
The association between psychological status and the development of early gastric cancer from atrophic gastritis

A recent hospital-based, cross-sectional observational study, was conducted in the Chinese population to explore the potential relationship between psychological state and the progression of atrophic gastritis (AG; caused by H. pylori or not) to early stage gastric cancer (EGC).

The study included 258 individuals receiving care in the Department of Gastroenterology at The Affiliated Hospital of Xuzhou Medical University between March 2020 and November 2024, and included 173 patients diagnosed with AG and 85 with EGC. Clinical profiles, demographic characteristics, and laboratory indices were initially recorded, and then comparative analyses were conducted between groups.

Results showed that, compared to the AG group, patients with EGC exhibited significantly higher psychological distress and depressive tendencies. These findings, while not conclusive, imply a possible association between psychological state and the presence of early gastric cancer (EGC) in patients with atrophic gastritis, thus suggesting that they could function as independent indicators and contribute to the malignant progression of EGC in patients.

Incorporating mental health screening tools could, therefore, offer supplementary value in the broader context of evaluating patients who might carry an increased likelihood of disease progression.

Learn more about this study here: http://dx.doi.org/10.1097/MD.0000000000045653

Reference

Liang, Mengmeng BD; Wang, Juan MD; Li, Rui MD; Yang, Jun MD; Liu, Yuping BD; Zhao, Lian BD. The association between psychological status and the development of early gastric cancer from atrophic gastritis. Medicine 104(45):p e45653, November 07, 2025
The association between H. pylori infection and cognitive deterioration: a systematic review and meta-analysis

The association between cognitive decline and H. pylori infection remains controversial, with some evidence suggesting that H. pylori eradication may slow the progression of the disease.

A new meta-analysis reviewed 16 studies to explore whether H. pylori affects cognitive function and whether cognitive decline is linked to higher rates of infection.

The analysis found that people with H. pylori infection had a higher risk of cognitive decline, especially when cognitive dysfunction and dementia were combined. However, the infection was not clearly linked to Alzheimer’s disease. Conversely, people with Alzheimer’s disease were more likely to have H. pylori infection than those without, though the association was weaker for other forms of dementia.

These findings suggest a bidirectional relationship in which H. pylori may contribute to cognitive decline, and certain cognitive conditions may increase susceptibility to infection. The study also highlights the need for more well-designed research to fully understand this complex interaction.

Learn more about this study here: https://doi.org/10.1186/s40001-025-03160-8

Reference

Elhady, M.M., Zidan, A., Rabea, E.M. et al. The association between H. pylori infection and cognitive deterioration: a systematic review and meta-analysis. Eur J Med Res 30, 846 (2025)
Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches

A study found that large language models (LLMs) outperform traditional deep neural network (DNN) embeddings in automated depression detection and show reduced gender bias, though racial disparities remain. Among DNN fairness-mitigation techniques, the worst-group loss provided the best balance between overall accuracy and demographic fairness, while fairness-regularized loss underperformed.

The identified biases affect the fairness and diagnostic reliability of AI systems for mental health assessment, particularly by disadvantaging underrepresented racial and gender groups, mainly Hispanic participants in the case of this research. Such disparities risk perpetuating inequities in automated mental health screening and could undermine trust and validity in clinical or public health applications.

Learn more about the study here: https://doi.org/10.48550/arXiv.2509.25795

Reference

Junias, O., Kini, P., & Chaspari, T. (2025). Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches. 2025 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), 1-7.
Domain Adversarial Training for Mitigating Gender Bias in Speech-based Mental Health Detection

A domain adversarial training (DAT) was developed in a study as a method to reduce gender bias in AI models for depression and PTSD detection using speech data (E-DAIC dataset).

DAT improved F1-scores up to +13% and reduced gender gaps in detection accuracy, improving generalization across male and female participants, specially addressing the effects of the latter’s underrepresentation.

Learn more about this study here: https://doi.org/10.48550/arXiv.2505.03359

Reference

Kim, J., Yoon, H., Oh, W., Jung, D., Yoon, S., Kim, D., Lee, D., Lee, S., & Yang, C. (2025). Domain Adversarial Training for Mitigating Gender Bias in Speech-based Mental Health Detection. 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 1-7.
Developing personalized algorithms for sensing mental health symptoms in daily life

This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data.

It finds that these tools underperform in larger, more diverse populations because the behavioral patterns used to predict depression are inconsistent across demographic and socioeconomic subgroups.

Specifically, the AI models often misclassify individuals from certain groups—such as older adults or those from different racial or gender backgrounds—as being at lower risk than they actually are. The authors emphasize the need for tailored, subgroup-aware approaches to improve reliability and fairness in mental health prediction tools. This work highlights the importance of addressing demographic bias to ensure equitable AI deployment in mental healthcare.

Learn more about this study here: https://doi.org/10.1038/s44184-025-00147-5

Reference

Timmons, A.C., Tutul, A.A., Avramidis, K. et al. Developing personalized algorithms for sensing mental health symptoms in daily life. npj Mental Health Res 4, 34 (2025).
Regulatory Effects of Probiotics on Anxiety and Depression‑Like Behaviors in H. pylori‑Infected Rats

In a recent experimental study, researchers used rats to explore whether the use of probiotics such as Lactobacillus can mitigate anxiety- and depression-like behaviors, counteracting the psychological and biological effects of H. pylori infection. Infected rats were treated with each probiotic alone or with the combination, and were then evaluated using standard behavioral tests for anxiety and depression.

Both probiotics, especially when co-administered, reversed the depressive and anxiogenic effects induced by H. pylori. Probiotic supplementation also corrected several brain changes linked to H. pylori, including oxidative stress, inflammation, reduced BDNF/serotonin, and elevated corticosterone.

The findings suggest that multi-strain probiotics may help manage psychiatric symptoms associated with H. pylori infection, and that they merit further clinical evaluation in patients with psychiatric comorbidities.

Learn more about this study here: https://doi.org/10.1007/s12602-025-10674-4

Reference

Ahmadi-Soleimani, S.M., Masoudi, M., Tabrizi, A.M.A. et al. Regulatory Effects of Probiotics on Anxiety and Depression-Like Behaviors in H. pylori-Infected Rats. Probiotics & Antimicro. Prot. (2025)
Minding the Gaps: Neuroethics, AI, and Depression

In this article, the author highlights the benefits and potential issues regarding the use of AI in depression diagnosis/treatment, focusing on the prevalent gender, racial and ethnicity biases.

It is mentioned that, given the historical, inherent biases in society generally and healthcare specifically, AI-driven advancements are not going to serve minority groups as a matter of course. Unless they are tailored to represent and serve all communities equally, they will exacerbate existing biases and disparities.

Learn more about this article here: https://nonprofitquarterly.org/minding-the-gaps-neuroethics-ai-and-depression/

Reference

Boothroyd, Gemma (2024), “Minding the Gaps: Neuroethics, AI, and Depression”, in Nonprofit Quarterly Magazine, winter 2024, “Health Justice in the Digital Age: Can We Harness AI for Good?”
Bias and Fairness in AI-Based Mental Health Models

The paper examines bias and fairness issues in AI-based mental health applications, including diagnostic tools, chatbots, and suicide risk prediction models. It reports how unrepresentative datasets lead to misdiagnosis and unequal outcomes across different socioeconomic, gender and racial groups – namely concerning women, local ethnic minorities or non-Western societies -, and presents mitigation strategies such as diverse datasets, fairness metrics, and human-in-the-loop approaches.

Learn more about this paper here: https://www.researchgate.net/publication/389214235_Bias_and_Fairness_in_AI-Based_Mental_Health_Models

Reference

Barnty, Barnabas & Joseph, Oloyede & Ok, Emmanuel. (2025). Bias and Fairness in AI-Based Mental Health Models.
AI and Mental Healthcare – ethical and regulatory considerations

This governmental report discusses the ethical and regulatory considerations of using artificial intelligence in mental healthcare in the UK.

Bias in AI tools (algorithmic bias) can stem from various places, including tools being trained on biased datasets and outputting discriminatory outcomes or developers making biased decisions in the design or training of such tools. For example, mental health Electronic health record (EHR) data is susceptible to cohort and label bias. This can occur because culture-bound presentations of mental disorders, combined with a lack of transcultural literacy among clinicians, often lead to both over- and under-diagnosis. People can also exhibit bias when using AI tools, such as over-relying on, or mistrusting AI outputs. All these biases can be conscious or unconscious.

Learn more about the report here: https://doi.org/10.58248/PN738

Reference

Gardiner, Hannah and Natasha Mutebi (2025), AI and Mental Healthcare – ethical and regulatory considerations, UK Parliament – POST, POSTnote 738, 31 January 2025
A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection

This study examines classification parity across sex and finds that female adolescents have systematically under-diagnosed mental health disorders: their model’s accuracy was ~4 % lower and false negative rate ~9 % higher compared to male patients. The source of the bias resides in the textual data, namely notes corresponding to male patients tended to be on average 500 words longer and had distinct word usage. To mitigate this, the authors introduce a de-biasing method, based on neutralizing biased terms (gendered words and pronouns) and reducing sentences to essential clinical information. After correcting, diagnostic bias is reduced by up to 27%.

This emphasizes how linguistically transmitted bias—ensuing from word choice and gendered language—consistently leads to the under-diagnosis of mental health disorders among female adolescents, which critically undermines the impartiality of medical diagnosis and treatment.

Learn more about this study here: https://doi.org/10.48550/arXiv.2501.00129

Reference

Ive, J., Bondaronek, P., Yadav, V., Santel, D., Glauser, T., Cheng, T., Strawn, J.R., Agasthya, G., Tschida, J., Choo, S., Chandrashekar, M., Kapadia, A.J., & Pestian, J.P. (2024). A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection.
The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media

The study found that the BDI-Sen dataset used for depression symptom detection on social media exhibits gender bias, with machine learning models such as mentalBERT showing predictive disparities that generally favour male users. Although bias mitigation techniques like data augmentation reduced the bias, they did not eliminate it completely.

The existence of this bias affects the fairness and reliability of AI systems in detecting depression symptoms, leading to unequal predictive performance across genders. This can result in under- or over-identification of depression symptoms in certain groups, thereby compromising the validity of such systems for clinical or mental health monitoring.

Learn more about this study here: https://studenttheses.uu.nl/handle/20.500.12932/47734

Reference

Gierschmann, Lara (2024), The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media, Utrecht University, unpublished Master Thesis
Gender Bias in AI’s Perception of Cardiovascular Risk

The study investigated gender bias in GPT-4’s assessment of coronary artery disease risk and showed that there was a substantial shift in the perception of risk between men and women when a psychiatric comorbidity was added to the vignette, even when they presented identical complaints.

This resulted in women being assessed as having lower risk of CAD when concurrently having a psychiatric condition.

Learn more about this study here: https://www.jmir.org/2024/1/e54242

Reference

Achtari M, Salihu A, Muller O, Abbé E, Clair C, Schwarz J, Fournier S Gender Bias in AI’s Perception of Cardiovascular Risk J Med Internet Res 2024;26:e54242 DOI: 10.2196/54242
Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

Considering how there is limited research on fairness in automated decision making systems in the clinical domain, particularly in the mental health domain, this study explores clinicians’ perceptions of AI fairness through two distinct scenarios: violence risk assessment and depression phenotype recognition using textual clinical notes.

Clinicians were engaged with through semi-structured interviews to understand their fairness perceptions and to identify appropriate quantitative fairness objectives for these scenarios. Then, a set of bias mitigation strategies were compared, developed to improve at least one of the four selected fairness objectives. The findings underscore the importance of carefully selecting fairness measures, as prioritizing less relevant measures can have a detrimental rather than a beneficial effect on model behavior in real-world clinical use.

Learn more about the article here: https://doi.org/10.1609/aies.v7i1.31732

Reference

Sogancioglu, G., Mosteiro, P., Salah, A. A., Scheepers, F., & Kaya, H. (2024). Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1390-1400.
Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

Depression and anxiety are common, often co-occurring mental health disorders that complicate diagnosis due to overlapping symptoms and reliance on subjective assessments.

Standard diagnostic tools are widely used but can introduce bias, as they depend on self-reported symptoms and clinician interpretation, which vary across individuals. These methods also fail to account for neurobiological factors such as neurotransmitter imbalances and altered brain connectivity.

Similarly, clinical AI/ML models used in healthcare often lack demographic diversity in their training data, with most studies failing to report race and gender, leading to biased outputs and reduced fairness. EEG offers a promising, objective approach to monitoring brain activity, potentially improving diagnostic accuracy and helping address biases in mental health assessment, as this study found.

Learn more about it here: https://doi.org/10.3390/brainsci14101018

Reference

Yousufi, M., Damaševičius, R., & Maskeliūnas, R. (2024). Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121. Brain sciences, 14(10), 1018.
Deconstructing demographic bias in speech-based machine learning models for digital health

This study explores gender and race bias in machine learning models that use speech to detect mental health outcomes, namely anxiety and depression. The authors examined where bias can come from, both in the acoustic features extracted from speech and in the labels used to train the models, as well as in the models’ decisions.

They also tested two ways of reducing bias: using only the speech features that carry the least demographic information, and transforming the data to remove traces of demographic information while keeping what matters for detecting the mental health condition. Performance varied across demographic groups, and the results of these approaches were mixed, highlighting the need for careful design so that speech-based tools work well for everyone.

Learn more about this study here: https://doi.org/10.3389/fdgth.2024.1351637

Reference

Yang M, El-Attar AA and Chaspari T (2024) Deconstructing demographic bias in speech-based machine learning models for digital health. Front. Digit. Health 6: 1351637.
Racial bias in AI-mediated psychiatric diagnosis and treatment: a qualitative comparison of four large language models

The article investigates racial bias in psychiatric diagnosis and treatment recommendations across four large language models (LLMs): Claude, ChatGPT, Gemini, and NewMes-15. The study evaluates the models’ responses to ten psychiatric cases representing five diagnoses (depression, anxiety, schizophrenia, eating disorders, and ADHD) under three conditions: race-neutral, race-implied, and race-explicitly stated (African American).

Key findings include:

1) Bias in Treatment Recommendations: LLMs often proposed inferior or divergent treatments when racial characteristics were explicitly or implicitly indicated, particularly for schizophrenia and anxiety cases. Diagnostic decisions showed minimal bias overall.

2) Model Performance: NewMes-15 exhibited the highest degree of racial bias, while Gemini demonstrated the least bias across conditions.

3) Statistical Analysis: A Kruskal–Wallis H-test revealed significant differences in bias among the LLMs, with Gemini being significantly less biased than ChatGPT and NewMes-15.

4) Challenges in AI Development: The study highlights that LLMs trained on biased datasets may perpetuate racial disparities in psychiatric care, even when specialized medical training data is used. Local LLMs, despite their cost and privacy advantages, showed higher susceptibility to bias compared to larger, online models.

Learn more about this study here: https://doi.org/10.1038/s41746-025-01746-4

Reference

Bouguettaya, A., Stuart, E.M. & Aboujaoude, E. Racial bias in AI-mediated psychiatric diagnosis and treatment: a qualitative comparison of four large language models. npj Digit. Med. 8, 332 (2025).
Key language markers of depression on social media depend on race

A recent U.S. study published in PNAS found that artificial intelligence models analyzing social media posts can detect signs of depression in white Americans but are far less accurate for Black Americans, underscoring the dangers of using AI trained on non-diverse data in healthcare.

According to co-author Sharath Chandra Guntuku from Penn Medicine, these differences suggest that prior AI models and language-based assessments have largely overlooked racial diversity. While the researchers noted that social media analysis should not be used for diagnosis, it may still help assess risk or monitor mental health trends in communities

Learn more about the study here: https://www.pnas.org/doi/10.1073/pnas.2319837121

Reference

S. Rai et al (2024), Key language markers of depression on social media depend on race, Proc. Natl. Acad. Sci. U.S.A. 121 (14)
Artificial Intelligence in mental health and the biases of language based models

In this literature review of the uses of Natural Language Processing (NLP) models in psychiatry, an approach that “systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective” was employed to find existing patterns.

The result was that significant biases were found, with respect to religion, race, gender, nationality, sexuality and age.

Learn more about this review here: https://doi.org/10.1371/journal.pone.0240376

Reference

Straw, I., & Callison-Burch, C. (2020). Artificial Intelligence in mental health and the biases of language based models. PloS one, 15(12), e0240376.
Genetic correlation, pleiotropic loci and shared risk genes between major depressive disorder and gastrointestinal tract disorders

Although depression is often linked to digestive disorders, the biological connection behind this phenomenon has remained unclear.

A recent genome-wide association study analyzed genetic data from hundreds of thousands of people to explore potential links between major depressive disorder (MDD) and gastrointestinal conditions such as peptic ulcers (mainly caused by H. pylori infection), acid reflux, irritable bowel disease, and inflammatory bowel disease.

The researchers found that depression shares genetic risk factors with most digestive disorders, meaning that some of the same genes and genetic regions influence both mental and gut health. The study also suggested that genetic susceptibility to certain gut conditions may increase the risk of depression, once again highlighting the strong gut-brain connection.

These findings could help scientists better understand the gut-brain connection and may point to new ways to treat gastrointestinal symptoms in patients with depression.

Learn more about this study here: https://doi.org/10.1016/j.jad.2025.01.048

Reference

Zhou, S., Zi, J., Hu, Y., Wang, X., Cheng, G., & Xiong, J. (2025). Genetic correlation, pleiotropic loci and shared risk genes between major depressive disorder and gastrointestinal tract disorders. Journal of affective disorders, 374, 84–90.
Association of Helicobacter pylori Infection with Depression and Anxiety: A Systematic Review and Meta-Analysis

The association between Helicobacter pylori infection and depression and anxiety has been reported in the literature.

A meta-analysis was developed in 2024 with the aim of investigating the association between H. pylori infection with these mental health conditions. The systematic search was conducted not only in international sources such as PubMed, Web of Science, and Embase, but also in Chinese databases, and looked for observational studies that reported the incidence or prevalence of depression and anxiety in patients with H. pylori infection.

Surprisingly, while the findings of this analysis showed a significant positive association between the bacteria and anxiety disorders, the association with depression appeared to be insignificant. Nevertheless, this finding seems to imply that clinicians treating H. pylori patients should also address their psychological well-being.

Learn more about this review here: https://doi.org/10.1155/2024/9247586

Reference

Li, Lu, Ren, Yadi, Wang, Zeyu, Niu, Yanqing, Zhao, Ying, Aihaiti, Xiaherezhati, Ji, Yinglan, Li, Man, Association of Helicobacter pylori Infection with Depression and Anxiety: A Systematic Review and Meta-Analysis, International Journal of Clinical Practice, 2024, 9247586, 9 pages, 2024.
Impact of Helicobacter pylori eradication on age‑specific risk of incident dementia in patients with peptic ulcer disease: a nationwide population‑based cohort study

A large South Korean cohort study from 2024 examined whether peptic ulcer disease (PUD) and Helicobacter pylori eradication therapy influence dementia risk in adults aged 55–79.

Using national health insurance data from 2002–2015 and propensity score matching, researchers assessed overall dementia and Alzheimer’s disease (AD) over 5–10 years. While the researchers did not directly verify the presence of the bacteria, their findings were based on treatment history.

The results showed that PUD was associated with a higher risk of developing dementia, with a stronger link for overall dementia than for AD. Eradication therapy itself did not markedly change overall risk, but later treatment was associated with greater dementia risk, highlighting the importance of timely management. Age-stratified analyses also indicated elevated AD risk, particularly in individuals in their 60s and 70s.

Overall, the findings suggest that PUD is a risk factor for dementia in older adults, and that early treatment of H. pylori infection may play a role in prevention strategies for neurodegenerative diseases.

Learn more about this study here: https://doi.org/10.1007/s11357-024-01284-z

Reference

Kang, D.W., Lee, JW., Park, M.Y. et al. Impact of Helicobacter pylori eradication on age-specific risk of incident dementia in patients with peptic ulcer disease: a nationwide population-based cohort study. GeroScience 47, 1161–1174 (2025).
Bio-Med AI Summer School debates the role of AI in healthcare

In September 2024, the National Library of Portugal in Lisbon hosted the Bio-Med AI Summer School, a three-day international event on the role of artificial intelligence in healthcare. Organised by whatnext.law, a research centre of NOVA School of Law and Vieira de Almeida, in partnership with the Biomedical Research Foundation of the Academy of Athens, it brought together academics, professionals and students from health, law, technology and ethics. The goal was to bridge the gap between AI and the fields of biomedicine and health.

Speakers shared examples of how AI is already being used in Portugal and elsewhere. These included summarising medical consultations so doctors can focus on their patients, chatbots that help patients book appointments, detecting signs of prostate cancer in MRI scans, drafting reports from medical images, and a large language model designed to interpret mental health questions.

Participants also stressed that these tools must remain tools, with the final decision always resting with doctors. They highlighted the importance of knowing where data comes from, ensuring its quality and complying with European regulation, such as the AI Act. In the closing session, former Minister of Health Adalberto Campos Fernandes highlighted the role AI could play in increasing access to healthcare worldwide and accelerating medical research.

Learn more here: https://eco.sapo.pt/2024/09/18/inteligencia-artificial-uma-revolucao-ao-servico-da-saude/

Reference:

ECO. (2024, September 18). Inteligência artificial: Uma revolução ao serviço da saúde. ECO.
Investigating the synergistic effects of amitriptyline and H. pylori eradication on depressive-like behaviors and inflammatory cytokines in mice

Recent research has highlighted the potential role of Helicobacter pylori in the pathogenesis of psychiatric disorders. In this experimental study, researchers used mice to explore whether combining amitriptyline with H. pylori eradication therapy produced greater benefits than either treatment alone.

Male mice were firstly allocated into four groups: healthy controls, H. pylori-infected mice, mice receiving antidepressant treatment only, and mice receiving both antidepressant and eradication therapy.

The initial findings confirmed that H. pylori infection induced depression-like behaviors in mice. In addition, while the use of antidepressants alone slightly improved these depression-like behaviors, it was the combination of antidepressants and bacterial eradication therapy that induced a more significant improvement in these symptoms. The combined treatment also appears to have led to an improvement in the psychomotor function of the animals.

Inflammatory cytokines (e.g., interleukin-1β (IL-1β), interleukin-6 (IL-6), and tumor necrosis factor-alpha (TNF-α)) are usually released in response to Helicobacter colonization and tissue damage, leading to a chronic state of inflammation, but are also associated with depression-like behaviors in humans and animal models, behaviors that have been shown to be attenuated by the use of antidepressants. In this study, researchers found that not only was H. pylori infection associated with changes in inflammatory cytokine levels, but that these alterations were partially reversed when both therapies were used together.

Overall, the findings suggest that targeting both infection and mood symptoms simultaneously may offer enhanced behavioral and anti-inflammatory benefits compared to isolated treatment approaches.

Learn more about this study here: https://doi.org/10.1016/j.physbeh.2024.114552

Reference

Miao Xu and Hua Fan, “Investigating the synergistic effects of amitriptyline and H. pylori eradication on depressive-like behaviors and inflammatory cytokines in mice”, Physiology & Behavior, Volume 281, 1 July 2024, 114552
The Psychotic Impact of Helicobacter pylori Gastritis and Functional Dyspepsia on Depression: A Systematic Review

The clinical practice of adding antidepressant drugs to the therapy for the eradication of Helicobacter pylori, in addition to the standard drug regimen, has been widely considered in recent years but its specific role in this treatment is still unclear.

In this systematic review researchers tried to establish if there is an association between H. pylori gastritis and depression and to further analyze the therapeutic effect of antidepressants on symptomatic relief in gastritis. For that, they analyzed randomized clinical trials, cross-sectional and prospective studies, and review articles that examined H. pylori infection, depression, functional dyspepsia, and antidepressant treatment. They focused especially on patients whose digestive symptoms did not improve even after successful H. pylori eradication.

Across the studies, a clear pattern emerged: patients with ongoing functional dyspepsia after H. pylori treatment often improved when they were given antidepressants, even when standard eradication therapy alone had not worked. However, the authors highlight that more research is needed before this approach becomes routine medical practice.

Learn more about this review here: https://doi.org/10.7759/cureus.5956

Reference

Al Quraan A M, Beriwal N, Sangay P, et al. (October 21, 2019) The Psychotic Impact of Helicobacter pylori Gastritis and Functional Dyspepsia on Depression: A Systematic Review. Cureus 11(10)
Psychological effects of Helicobacter pylori-associated atrophic gastritis in patients under 50 years: A cross-sectional study

A cross-sectional, observational study involving 975 Japanese individuals who underwent a health checkup, has found that people with atrophic gastritis had a significantly higher risk of experiencing psychological distress or depressed mood.

Interestingly, the risk was higher in females under 50 years old, regardless of H. pylori infection status.

Although the mechanism remains to be elucidated, the researchers suggest there is a possibility that nutritional status, neuroendocrinologic factors, and/or socioeconomic factors are involved. However, further studies are necessary to reveal the precise underlying mechanisms.

Learn more about this study here: https://doi.org/10.1111/hel.12445

Reference

Takeoka A, Tayama J, Kobayashi M, et al. Psychological effects of Helicobacter pylori-associated atrophic gastritis in patients under 50 years: A cross-sectional study. Helicobacter. 2017; 22:e12445
Correlation between social factors and anxiety-depression in function dyspepsia: do relationships exist?

A research study conducted on the Chinese population in 2014 aimed at evaluating the prevalence and the social factors linked to anxiety and depression in patients with functional dyspepsia (FD). This study included 907 patients with FD who attended a gastroenterology service.

Despite being a hospital-based study, results showed that patients with functional dyspepsia had higher anxiety and depression scores when compared to data from the general population. They also found that a higher prevalence of A/D was observed in women, older individuals, those with lower socioeconomic status (lower wages, lower education levels), and those with more stressful jobs, making these aspects risk factors for the development of A/D. Interestingly, they found no differences in relation to family history.

Learn more about this study here: https://doi.org/10.5114/pg.2014.47897

Reference

Huang, Z., Yang, X., Lan, L., Liu, T., Liu, C., & Li, J. et al. (2014). Correlation between social factors and anxiety-depression in function dyspepsia: do relationships exist?. Gastroenterology Review/Przegląd Gastroenterologiczny, 9(6), 348-353.