Knowing that AI can be biased is one thing. Knowing what to do about it, legally and ethically, is another.
Medicine’s ethical foundations, extended to AI
Medicine has long been associated with high ethical standards, from the Hippocratic Oath to the Declarations of Geneva and Helsinki. Unfortunately, the history of medicine includes deeply troubling chapters, from eugenics, forced sterilisation of indigenous and marginalised populations, to non-consensual experimentation on people with disabilities. These historical failures are directly relevant today because the structural injustices behind them shaped decades of medical data collection, and their legacy is exactly what today’s AI systems risk inheriting when trained on that same data.
Biomedical ethics is built around four core ethical principles: autonomy (respecting a person’s right to make their own decisions), non-maleficence (avoiding harm), beneficence (actively promoting wellbeing), and justice (distributing benefits and risks fairly, especially given known disparities based on race, gender, and social status) (Beauchamp & Childress, 2019).
As AI entered clinical practice, ethicists added a fifth principle specifically for these technologies: explicability, the idea that the reasoning behind an AI system’s decisions must be understandable and open to scrutiny, not simply accepted at face value (Floridi et al., 2018).
The EU’s legal response
The European Union has built a multi-layered legal framework specifically to address these risks. At its centre is the AI Act, which classifies many healthcare applications, including diagnostic tools and clinical decision-support systems, as “high-risk.” This means they’re subject to strict requirements: providers must actively identify and mitigate risks of discriminatory outcomes, ensure meaningful human oversight, and maintain accuracy and transparency throughout the system’s lifecycle (EU AI Act, 2024).
Hospitals that deploy these systems have specific legal obligations too, including using AI in line with provider instructions, assigning competent human oversight, maintaining logs of system operation, and reporting serious incidents, particularly where there’s a risk of unequal outcomes across patient groups.
The AI Act doesn’t stand alone. It works alongside the Medical Device Regulation (MDR; European Union, 2017a) and In Vitro Diagnostic Regulation (IVDR; European Union, 2017b), which govern the safety of medical technologies, the GDPR (European Union, 2016), which protects the sensitive health data these systems rely on, and the emerging European Health Data Space (EHDS; European Union, 2025), designed to enable more representative datasets for future AI development. Together, they form a legal ecosystem, all ultimately grounded in the EU Charter of Fundamental Rights (2009), particularly its guarantees of non-discrimination, equality, and the right to healthcare.
All these policies translate into real responsibilities for hospitals and healthcare professionals, from understanding a tool’s limitations and knowing when to override its recommendations, to being transparent with patients about when and how these systems are used in their care.
Beyond raising awareness, the AEQUITAS project has developed a practical AI Regulatory Model that translates these legal and ethical principles into concrete steps hospitals and healthcare professionals can actually apply, turning complex EU legislation into something usable in everyday clinical practice.
References:
Beauchamp, T. L., & Childress, J. F. (2019). Principles of biomedical ethics (8th ed).
European Commission, Timeline for Implementation of the EU AI Act, Brussels: European Commission, 2024.
European Union. (2009). Charter of Fundamental Rights of the European Union. Official Journal of the European Union, C 303/1. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:12012P/TXT
European Union, Regulation (EU) 2016/679 on General Data Protection Regulation (GDPR), Brussels: EU, 2016.
European Union, Regulation (EU) 2017/745 on Medical Devices (MDR), Official Journal of the European Union, L117, Brussels: European Union, 2017.
European Union, Regulation (EU) 2017/746 on In Vitro Diagnostic Medical Devices (IVDR), Official Journal of the European Union, L117, Brussels: European Union, 2017.
European Union, Regulation (EU) 2025/327 on the European Health Data Space (EHDS), Brussels: European Union, 2025.
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds and Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5