AI Models for Predicting Heart Failure: Far from Perfect

Lilu Anderson
Photo: Finoracle.me

Heart Failure Incident Rates Vary Widely by Age and Race, Study Shows

A recent study on heart failure incident rates has revealed significant variations based on age and race. The study, conducted by Kaur et al., analyzed nearly 60,000 heart failure incidents over five years. The data showed that 9% of patients aged 40 or younger developed heart failure, which rose dramatically to 36.6% for patients over 80. Additionally, heart failure within five years was more common among Black patients (23.5%) compared to white, Asian, and Hispanic patients.

Age and Race Impact the Accuracy of AI Model for Heart Failure Diagnosis

The study also examined the performance of an artificial intelligence (AI) model in diagnosing heart failure. The researchers found no notable differences between racial groups concerning the model’s performance. However, the accuracy of the AI model decreased significantly with age. Its performance, measured by the area under the ROC curve (AUC), decreased from 0.80 for patients aged 40 and younger to 0.66 for patients over 80.

AI Model Less Effective for Young Black Patients in Diagnosing Heart Failure

Of particular concern was the AI model’s performance when evaluating 12-lead ECGs from younger Black patients. The measured AUC was only 0.69 for this subgroup, significantly lower than the AUC of 0.80 for non-Hispanic white patients. The calibration curves showed that the AI model was best calibrated for Asian and non-Hispanic white patients, indicating a potential underdiagnosis of heart failure among Black patients, especially Black women.

Calibration Issues: AI Model Best Suited for Asian and Non-Hispanic White Patients

The study further highlighted calibration issues with the AI model. Researchers discovered that the model’s performance was best suited for Asian and non-Hispanic white patients. The observed fraction of heart failure cases among Black patients exceeded the model’s predicted probability, indicating a higher underdiagnosis within this racial group. This finding suggests that the AI model may perpetuate existing disparities in patient outcomes.

More Research Needed to Address Disparities in AI Model’s Accuracy for Heart Failure Diagnosis

The study’s findings emphasize the need for further research to ensure that advanced AI models do not perpetuate disparities in patient outcomes. The researchers caution against using the ECG deep learning model for heart failure prognosis among certain demographic subgroups. It is clear that addressing these disparities and improving the accuracy and calibration of AI models is crucial to achieving equitable healthcare outcomes for all patients.

Analyst comment

Positive: The study provides valuable insights on heart failure incident rates based on age and race, helping to identify demographic groups at higher risk.

Short: The market for AI models for heart failure diagnosis may experience scrutiny and demand for improvement to address disparities in accuracy and calibration across different demographic subgroups. Additional research is needed to ensure equitable healthcare outcomes.

Share This Article
Lilu Anderson is a technology writer and analyst with over 12 years of experience in the tech industry. A graduate of Stanford University with a degree in Computer Science, Lilu specializes in emerging technologies, software development, and cybersecurity. Her work has been published in renowned tech publications such as Wired, TechCrunch, and Ars Technica. Lilu’s articles are known for their detailed research, clear articulation, and insightful analysis, making them valuable to readers seeking reliable and up-to-date information on technology trends. She actively stays abreast of the latest advancements and regularly participates in industry conferences and tech meetups. With a strong reputation for expertise, authoritativeness, and trustworthiness, Lilu Anderson continues to deliver high-quality content that helps readers understand and navigate the fast-paced world of technology.