Transformer-based Models for Cardiovascular Disease Predictions from Electronic Health Records: A Systematic Review

Abstract

This systematic literature review (SLR) analyses 16 studies published between 2020 and 2025 that applied transformer-based or other machine learning models to predict cardiovascular disease (CVD) using electronic health records (EHRs). Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, the review ensures transparency in the identification, screening, and quality appraisal of eligible studies. The key findings reveal a rapid shift from traditional machine learning models, such as Random Forest, toward transformer architectures like the Bidirectional Encoder Representation from Transformers for Electronic Health Record (BEHRT) and its variants. These models demonstrate a superior discrimination (Area Under Curve: 0.84 to 0.93) due to their capacity to model long-term temporal dependencies. Explainable AI (XAI) tools, such as attention visualisation, were frequently employed, yet clinical interpretability and integration into decision support remain underexplored. The review also highlights opportunities in federated and privacy-preserving learning, multimodal data fusion, and hybrid architectures that integrate transformers with traditional machine learning methods. This review addresses a gap in the past literature by being the first SLR to compare transformer variants for the prediction of CVDs. Other SLRs examined general CVD risk models, but the present SLR analyses interpretability, external validation and methodological limitations to transformer models. The findings of the recent SLR reported challenges that include data-shift limitations, model-poor population generalisation and their limitations to clinical adoption, which highlights the need for more evaluation protocols and clinicians’ interpretability frameworks.