Algorithms that read electrocardiograms (ECGs) automatically are increasingly used to help identify heart conditions, but few studies have examined whether they perform equally well for different groups of patients. This study compared 56 open-source algorithms, developed by participants in the 2021 PhysioNet Challenge, on more than 130,000 ECGs, grouping the results by sex, age and race.
Several performance measures were similar across groups, but others showed clear differences. Accuracy was significantly lower for Black patients and higher for Asian patients, with a gap of 39% on the test set. Accuracy also fell with age, by around 12% per decade, and results for male and female patients differed depending on the measure used.
The researchers then retrained one of the best-performing models with an additional constraint designed to reduce performance differences across sex, race and age. This led to a small loss in overall performance but a significant reduction in bias. The authors conclude that bias depends on the model’s design, the population it is trained on and how its performance is measured, and that all of these should be closely examined.
Learn more about this study here: https://doi.org/10.1016/j.jelectrocard.2022.07.007
Reference:
Perez Alday, E.A., Rad, A.B., Reyna, M.A., Sadr, N., Gu, A., Li, Q., Dumitru, M., Xue, J., Albert, D., Sameni, R., & Clifford, G.D. (2022). Age, sex and race bias in automated arrhythmia detectors. Journal of Electrocardiology, 74, 5–9.