Health Citizens – European Research Institute

Tag: AEQUITAS

  • A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection

    A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection

    This study examines classification parity across sex and finds that female adolescents have systematically under-diagnosed mental health disorders: their model’s accuracy was ~4 % lower and false negative rate ~9 % higher compared to male patients. The source of the bias resides in the textual data, namely notes corresponding to male patients tended to be on average 500 words longer and had distinct word usage. To mitigate this, the authors introduce a de-biasing method, based on neutralizing biased terms (gendered words and pronouns) and reducing sentences to essential clinical information. After correcting, diagnostic bias is reduced by up to 27%.

    This emphasizes how linguistically transmitted bias—ensuing from word choice and gendered language—consistently leads to the under-diagnosis of mental health disorders among female adolescents, which critically undermines the impartiality of medical diagnosis and treatment.

    Learn more about this study here: https://doi.org/10.48550/arXiv.2501.00129

    Reference

    Ive, J., Bondaronek, P., Yadav, V., Santel, D., Glauser, T., Cheng, T., Strawn, J.R., Agasthya, G., Tschida, J., Choo, S., Chandrashekar, M., Kapadia, A.J., & Pestian, J.P. (2024). A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection. 

  • The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media

    The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media

    The study found that the BDI-Sen dataset used for depression symptom detection on social media exhibits gender bias, with machine learning models such as mentalBERT showing predictive disparities that generally favour male users. Although bias mitigation techniques like data augmentation reduced the bias, they did not eliminate it completely.

    The existence of this bias affects the fairness and reliability of AI systems in detecting depression symptoms, leading to unequal predictive performance across genders. This can result in under- or over-identification of depression symptoms in certain groups, thereby compromising the validity of such systems for clinical or mental health monitoring.

    Learn more about this study here: https://studenttheses.uu.nl/handle/20.500.12932/47734

    Reference

    Gierschmann, Lara (2024), The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media, Utrecht University, unpublished Master Thesis

  • AEQUITAS

    AEQUITAS

    Proposal: Preparation of the CSOs and public healthcare sector to address gender and racial biases that might arise from the wide usage of AI in order to protect and promote fundamental rights

    Implementation: 2025 to 2027

    Call:  CERV-2024-CHAR-LITI – Promote civil society organizations’ awareness of, capacity building and implementation of the EU Charter of Fundamental Rights

    Proposed Budget:   1 061 761,00€

    Keywords: gender and racial biases, biomedical AI, EU Charter of Fundamental Rights

    Objective: The topic of the AEQUITAS project is the gender and racial biases that have been reported in biomedical AI and which can lead to misdiagnosis and mistreatment and how they pose a threat to the fundamental rights protected by the EU Charter. The aim of the project is 3-fold:

    – to increase the capacity of the CSOs and human rights organizations in educating the public, monitoring the biases and advocating for the protection of the fundamental rights especially regarding the biomedical AI;

    – to increase the knowledge of healthcare staff from public hospitals on the biases that biomedical AI can present due to the biased data from which they were fed and to help them approach and consult these AI systems with a critical mindset;

    – to develop an AI Regulatory Model that will be used by CSOs and public hospitals in their practices;

    – to develop a European network of CSOs and public hospitals that will collaborate and support each other in the effort to raise public’s awareness on the gender and racial biases of biomedical AI and on the applications of the EU Charter;

    – to raise awareness of the EU Charter of fundamental rights and its application in the AI era;

    – to develop and distribute policy recommendations in order to advocate for the need to regulate biomedical AI.

    Partners:

    • Innovation Hive – Kypseli Kainotomias
    • Kentro Ginaikeion Meleton Kai Ereyvnon Astiki Mi K
    • Universitat Zu Koln
    • Center for the Study of Democracy
    • C.I.P. Citizens in Power
    • Moterų Informacijos Centras Asociacija Mic
    • Lobby Europeo de Mujeres en Espana LEM España
    • TIA Formazione Internazionale Associazione APS
    • Health Citizens – European Institute
    • Technologiko Panepistimio Kyprou
    • Cyens Centre of Excellence
    • Edex – Educational Excellence Corporation Limited
    • Rite Research Institute for Technological Evolution 
    • Vucable
    • TotalEU Production

    Project Website: under development

  • Gender Bias in AI’s Perception of Cardiovascular Risk

    Gender Bias in AI’s Perception of Cardiovascular Risk

    The study investigated gender bias in GPT-4’s assessment of coronary artery disease risk and showed that there was a substantial shift in the perception of risk between men and women when a psychiatric comorbidity was added to the vignette, even when they presented identical complaints.

    This resulted in women being assessed as having as lower risk of CAD when concurrently having a psychiatric condition.

    Learn more about this study here: https://www.jmir.org/2024/1/e54242

    Reference

    Achtari M, Salihu A, Muller O, Abbé E, Clair C, Schwarz J, Fournier S
    Gender Bias in AI’s Perception of Cardiovascular Risk
    J Med Internet Res 2024;26:e54242
    DOI: 10.2196/54242

  • Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

    Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

    Considering how there is limited research on fairness in automated decision making systems in the clinical domain, particularly in the mental health domain, this study explores clinicians’ perceptions of AI fairness through two distinct scenarios: violence risk assessment and depression phenotype recognition using textual clinical notes.

    Clinicians were engaged with through semi-structured interviews to understand their fairness perceptions and to identify appropriate quantitative fairness objectives for these scenarios. Then, a set of bias mitigation strategies were compared, developed to improve at least one of the four selected fairness objectives. The findings underscore the importance of carefully selecting fairness measures, as prioritizing less relevant measures can have a detrimental rather than a beneficial effect on model behavior in real-world clinical use.

    Learn more about the article here: https://doi.org/10.1609/aies.v7i1.31732

    Reference

    Sogancioglu, G., Mosteiro, P., Salah, A. A., Scheepers, F., & Kaya, H. (2024). Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1390-1400.

  • Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

    Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

    Depression and anxiety are common, often co-occurring mental health disorders that complicate diagnosis due to overlapping symptoms and reliance on subjective assessments.

    Standard diagnostic tools are widely used but can introduce bias, as they depend on self-reported symptoms and clinician interpretation, which vary across individuals. These methods also fail to account for neurobiological factors such as neurotransmitter imbalances and altered brain connectivity.

    Similarly, clinical AI/ML models used in healthcare often lack demographic diversity in their training data, with most studies failing to report race and gender, leading to biased outputs and reduced fairness. EEG offers a promising, objective approach to monitoring brain activity, potentially improving diagnostic accuracy and helping address biases in mental health assessment, as this study found.

    Learn more about it here: https://doi.org/10.3390/brainsci14101018

    Reference

    Yousufi, M., Damaševičius, R., & Maskeliūnas, R. (2024). Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121. Brain sciences, 14(10), 1018.

  • Deconstructing demographic bias in speech-based machine learning models for digital health

    Deconstructing demographic bias in speech-based machine learning models for digital health

    This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data.

    It finds that the model underperforms across several demographic subgroups, including gender, race, age, and socioeconomic status, often misclassifying individuals with depression as low-risk. For example, older adults and Black or low-income individuals were frequently ranked lower in risk than healthier younger or White individuals.

    These biases stem from inconsistent relationships between sensed behaviors and depression across groups. The authors emphasized the need for subgroup-specific modeling to improve fairness and reliability in mental health AI tools.

    Learn more about this study here: https://doi.org/10.3389/fdgth.2024.1351637

    Reference

    Yang M, El-Attar AA and Chaspari T (2024) Deconstructing demographic bias in speech-based machine learning models for digital health. Front. Digit. Health 6: 1351637. 

  • Key language markers of depression on social media depend on race

    Key language markers of depression on social media depend on race

    A recent U.S. study published in PNAS found that artificial intelligence models analyzing social media posts can detect signs of depression in white Americans but are far less accurate for Black Americans, underscoring the dangers of using AI trained on non-diverse data in healthcare.

    According to co-author Sharath Chandra Guntuku from Penn Medicine, these differences suggest that prior AI models and language-based assessments have largely overlooked racial diversity. While the researchers noted that social media analysis should not be used for diagnosis, it may still help assess risk or monitor mental health trends in communities

    Learn more about the study here: https://www.pnas.org/doi/10.1073/pnas.2319837121

    Reference

    S. Rai et al (2024), Key language markers of depression on social media depend on race, Proc. Natl. Acad. Sci. U.S.A. 121 (14)

  • Artificial Intelligence in mental health and the biases of language based models

    Artificial Intelligence in mental health and the biases of language based models

    In this literature review of the uses of Natural Language Processing (NLP) models in psychiatry, an approach that “systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective” was employed to find existing patterns.

    The result was that significant biases were found, with respect to religion, race, gender, nationality, sexuality and age.

    Learn more about this review here: https://doi.org/10.1371/journal.pone.0240376

    Reference

    Straw, I., & Callison-Burch, C. (2020). Artificial Intelligence in mental health and the biases of language based models. PloS one, 15(12), e0240376.