Category: Research

Bias and Fairness in AI-Based Mental Health Models

The paper examines bias and fairness issues in AI-based mental health applications, including diagnostic tools, chatbots, and suicide risk prediction models. It reports how unrepresentative datasets lead to misdiagnosis and unequal outcomes across different socioeconomic, gender and racial groups – namely concerning women, local ethnic minorities or non-Western societies -, and presents mitigation strategies such as diverse datasets, fairness metrics, and human-in-the-loop approaches.

Learn more about this paper here: https://www.researchgate.net/publication/389214235_Bias_and_Fairness_in_AI-Based_Mental_Health_Models

Reference

Barnty, Barnabas & Joseph, Oloyede & Ok, Emmanuel. (2025). Bias and Fairness in AI-Based Mental Health Models.
AI and Mental Healthcare – ethical and regulatory considerations

This governmental report discusses the ethical and regulatory considerations of using artificial intelligence in mental healthcare in the UK.

Bias in AI tools (algorithmic bias) can stem from various places, including tools being trained on biased datasets and outputting discriminatory outcomes or developers making biased decisions in the design or training of such tools. For example, mental health Electronic health record (EHR) data is susceptible to cohort and label bias. This can occur because culture-bound presentations of mental disorders, combined with a lack of transcultural literacy among clinicians, often lead to both over- and under-diagnosis. People can also exhibit bias when using AI tools, such as over-relying on, or mistrusting AI outputs. All these biases can be conscious or unconscious.

Learn more about the report here: https://doi.org/10.58248/PN738

Reference

Gardiner, Hannah and Natasha Mutebi (2025), AI and Mental Healthcare – ethical and regulatory considerations, UK Parliament – POST, POSTnote 738, 31 January 2025
A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection

This study examines classification parity across sex and finds that female adolescents have systematically under-diagnosed mental health disorders: their model’s accuracy was ~4 % lower and false negative rate ~9 % higher compared to male patients. The source of the bias resides in the textual data, namely notes corresponding to male patients tended to be on average 500 words longer and had distinct word usage. To mitigate this, the authors introduce a de-biasing method, based on neutralizing biased terms (gendered words and pronouns) and reducing sentences to essential clinical information. After correcting, diagnostic bias is reduced by up to 27%.

This emphasizes how linguistically transmitted bias—ensuing from word choice and gendered language—consistently leads to the under-diagnosis of mental health disorders among female adolescents, which critically undermines the impartiality of medical diagnosis and treatment.

Learn more about this study here: https://doi.org/10.48550/arXiv.2501.00129

Reference

Ive, J., Bondaronek, P., Yadav, V., Santel, D., Glauser, T., Cheng, T., Strawn, J.R., Agasthya, G., Tschida, J., Choo, S., Chandrashekar, M., Kapadia, A.J., & Pestian, J.P. (2024). A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection.
The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media

The study found that the BDI-Sen dataset used for depression symptom detection on social media exhibits gender bias, with machine learning models such as mentalBERT showing predictive disparities that generally favour male users. Although bias mitigation techniques like data augmentation reduced the bias, they did not eliminate it completely.

The existence of this bias affects the fairness and reliability of AI systems in detecting depression symptoms, leading to unequal predictive performance across genders. This can result in under- or over-identification of depression symptoms in certain groups, thereby compromising the validity of such systems for clinical or mental health monitoring.

Learn more about this study here: https://studenttheses.uu.nl/handle/20.500.12932/47734

Reference

Gierschmann, Lara (2024), The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media, Utrecht University, unpublished Master Thesis
Association of Helicobacter pylori Infection with Depression and Anxiety: A Systematic Review and Meta-Analysis

The association between Helicobacter pylori infection and depression and anxiety has been reported in the literature.

A meta-analysis was developed in 2024 with the aim of investigating the association between H. pylori infection with these mental health conditions. The systematic search was conducted not only in international sources such as PubMed, Web of Science, and Embase, but also in Chinese databases, and looked for observational studies that reported the incidence or prevalence of depression and anxiety in patients with H. pylori infection.

Surprisingly, while the findings of this analysis showed a significant positive association between the bacteria and anxiety disorders, the association with depression appeared to be insignificant. Nevertheless, this finding seems to imply that clinicians treating H. pylori patients should also address their psychological well-being.

Learn more about this review here: https://doi.org/10.1155/2024/9247586

Reference

Li, Lu, Ren, Yadi, Wang, Zeyu, Niu, Yanqing, Zhao, Ying, Aihaiti, Xiaherezhati, Ji, Yinglan, Li, Man, Association of Helicobacter pylori Infection with Depression and Anxiety: A Systematic Review and Meta-Analysis, International Journal of Clinical Practice, 2024, 9247586, 9 pages, 2024.
Impact of Helicobacter pylori eradication on age‑specific risk of incident dementia in patients with peptic ulcer disease: a nationwide population‑based cohort study

A large South Korean cohort study from 2024 examined whether peptic ulcer disease (PUD) and Helicobacter pylori eradication therapy influence dementia risk in adults aged 55–79.

Using national health insurance data from 2002–2015 and propensity score matching, researchers assessed overall dementia and Alzheimer’s disease (AD) over 5–10 years. While the researchers did not directly verify the presence of the bacteria, their findings were based on treatment history.

The results showed that PUD was associated with a higher risk of developing dementia, with a stronger link for overall dementia than for AD. Eradication therapy itself did not markedly change overall risk, but later treatment was associated with greater dementia risk, highlighting the importance of timely management. Age-stratified analyses also indicated elevated AD risk, particularly in individuals in their 60s and 70s.

Overall, the findings suggest that PUD is a risk factor for dementia in older adults, and that early treatment of H. pylori infection may play a role in prevention strategies for neurodegenerative diseases.

Learn more about this study here: https://doi.org/10.1007/s11357-024-01284-z

Reference

Kang, D.W., Lee, JW., Park, M.Y. et al. Impact of Helicobacter pylori eradication on age-specific risk of incident dementia in patients with peptic ulcer disease: a nationwide population-based cohort study. GeroScience 47, 1161–1174 (2025).
Gender Bias in AI’s Perception of Cardiovascular Risk

The study investigated gender bias in GPT-4’s assessment of coronary artery disease risk and showed that there was a substantial shift in the perception of risk between men and women when a psychiatric comorbidity was added to the vignette, even when they presented identical complaints.

This resulted in women being assessed as having as lower risk of CAD when concurrently having a psychiatric condition.

Learn more about this study here: https://www.jmir.org/2024/1/e54242

Reference

Achtari M, Salihu A, Muller O, Abbé E, Clair C, Schwarz J, Fournier S
Gender Bias in AI’s Perception of Cardiovascular Risk
J Med Internet Res 2024;26:e54242
DOI: 10.2196/54242
Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

Considering how there is limited research on fairness in automated decision making systems in the clinical domain, particularly in the mental health domain, this study explores clinicians’ perceptions of AI fairness through two distinct scenarios: violence risk assessment and depression phenotype recognition using textual clinical notes.

Clinicians were engaged with through semi-structured interviews to understand their fairness perceptions and to identify appropriate quantitative fairness objectives for these scenarios. Then, a set of bias mitigation strategies were compared, developed to improve at least one of the four selected fairness objectives. The findings underscore the importance of carefully selecting fairness measures, as prioritizing less relevant measures can have a detrimental rather than a beneficial effect on model behavior in real-world clinical use.

Learn more about the article here: https://doi.org/10.1609/aies.v7i1.31732

Reference

Sogancioglu, G., Mosteiro, P., Salah, A. A., Scheepers, F., & Kaya, H. (2024). Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1390-1400.
Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

Depression and anxiety are common, often co-occurring mental health disorders that complicate diagnosis due to overlapping symptoms and reliance on subjective assessments.

Standard diagnostic tools are widely used but can introduce bias, as they depend on self-reported symptoms and clinician interpretation, which vary across individuals. These methods also fail to account for neurobiological factors such as neurotransmitter imbalances and altered brain connectivity.

Similarly, clinical AI/ML models used in healthcare often lack demographic diversity in their training data, with most studies failing to report race and gender, leading to biased outputs and reduced fairness. EEG offers a promising, objective approach to monitoring brain activity, potentially improving diagnostic accuracy and helping address biases in mental health assessment, as this study found.

Learn more about it here: https://doi.org/10.3390/brainsci14101018

Reference

Yousufi, M., Damaševičius, R., & Maskeliūnas, R. (2024). Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121. Brain sciences, 14(10), 1018.
Deconstructing demographic bias in speech-based machine learning models for digital health

This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data.

It finds that the model underperforms across several demographic subgroups, including gender, race, age, and socioeconomic status, often misclassifying individuals with depression as low-risk. For example, older adults and Black or low-income individuals were frequently ranked lower in risk than healthier younger or White individuals.

These biases stem from inconsistent relationships between sensed behaviors and depression across groups. The authors emphasized the need for subgroup-specific modeling to improve fairness and reliability in mental health AI tools.

Learn more about this study here: https://doi.org/10.3389/fdgth.2024.1351637

Reference

Yang M, El-Attar AA and Chaspari T (2024) Deconstructing demographic bias in speech-based machine learning models for digital health. Front. Digit. Health 6: 1351637.