Health Citizens – European Research Institute

Tag: Representation Bias

  • Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

    Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation

    Considering how there is limited research on fairness in automated decision making systems in the clinical domain, particularly in the mental health domain, this study explores clinicians’ perceptions of AI fairness through two distinct scenarios: violence risk assessment and depression phenotype recognition using textual clinical notes.

    Clinicians were engaged with through semi-structured interviews to understand their fairness perceptions and to identify appropriate quantitative fairness objectives for these scenarios. Then, a set of bias mitigation strategies were compared, developed to improve at least one of the four selected fairness objectives. The findings underscore the importance of carefully selecting fairness measures, as prioritizing less relevant measures can have a detrimental rather than a beneficial effect on model behavior in real-world clinical use.

    Learn more about the article here: https://doi.org/10.1609/aies.v7i1.31732

    Reference

    Sogancioglu, G., Mosteiro, P., Salah, A. A., Scheepers, F., & Kaya, H. (2024). Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1390-1400.

  • Deconstructing demographic bias in speech-based machine learning models for digital health

    Deconstructing demographic bias in speech-based machine learning models for digital health

    This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data.

    It finds that the model underperforms across several demographic subgroups, including gender, race, age, and socioeconomic status, often misclassifying individuals with depression as low-risk. For example, older adults and Black or low-income individuals were frequently ranked lower in risk than healthier younger or White individuals.

    These biases stem from inconsistent relationships between sensed behaviors and depression across groups. The authors emphasized the need for subgroup-specific modeling to improve fairness and reliability in mental health AI tools.

    Learn more about this study here: https://doi.org/10.3389/fdgth.2024.1351637

    Reference

    Yang M, El-Attar AA and Chaspari T (2024) Deconstructing demographic bias in speech-based machine learning models for digital health. Front. Digit. Health 6: 1351637. 

  • Key language markers of depression on social media depend on race

    Key language markers of depression on social media depend on race

    A recent U.S. study published in PNAS found that artificial intelligence models analyzing social media posts can detect signs of depression in white Americans but are far less accurate for Black Americans, underscoring the dangers of using AI trained on non-diverse data in healthcare.

    According to co-author Sharath Chandra Guntuku from Penn Medicine, these differences suggest that prior AI models and language-based assessments have largely overlooked racial diversity. While the researchers noted that social media analysis should not be used for diagnosis, it may still help assess risk or monitor mental health trends in communities

    Learn more about the study here: https://www.pnas.org/doi/10.1073/pnas.2319837121

    Reference

    S. Rai et al (2024), Key language markers of depression on social media depend on race, Proc. Natl. Acad. Sci. U.S.A. 121 (14)

  • Artificial Intelligence in mental health and the biases of language based models

    Artificial Intelligence in mental health and the biases of language based models

    In this literature review of the uses of Natural Language Processing (NLP) models in psychiatry, an approach that “systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective” was employed to find existing patterns.

    The result was that significant biases were found, with respect to religion, race, gender, nationality, sexuality and age.

    Learn more about this review here: https://doi.org/10.1371/journal.pone.0240376

    Reference

    Straw, I., & Callison-Burch, C. (2020). Artificial Intelligence in mental health and the biases of language based models. PloS one, 15(12), e0240376.