2017 HSR&D/QUERI National Conference
4054 — Rethinking Risk Prediction Using Electronic Health Record data: The VA Risk Score for Cardiovascular Disease (VARS-CVD)
Lead/Presenter: Wyndy Wiitala, COIN - Ann Arbor
All Authors: Wiitala WL (VA HSR&D Center for Clinical Management Research)
Hayward RA (VA HSR&D Center for Clinical Management Research; Dept of Internal Medicine, University of Michigan)
Hofer TP (VA HSR&D Center for Clinical Management Research; Dept of Internal Medicine, University of Michigan)
Bentley DB (VA HSR&D Center for Clinical Management Research)
Sussman JB (VA HSR&D Center for Clinical Management Research; Dept of Internal Medicine, University of Michigan)
Accurate risk prediction is central to clinical decision-making and personalizing treatment choices. The availability of EHR data makes it practical to use complex risk prediction models developed in the population of interest. We examined if the VA electronic health record could be used to better predict atherosclerotic cardiovascular disease (ASCVD) than the current standard.
The sample included VA ambulatory care patients aged 45-80 in 2006 with no history of ASCVD or heart failure (N = 1,512,092). Our outcome was fatal or nonfatal myocardial infarction or stroke over 5 years using VHA, Medicare, and National Death Index data. We compared the standard ASCVD score with new scores: 1) ASCVD risk predictors calibrated on our VA sample (VARS-CVD); 2) An expanded set of 45 predictors that included multiple measures of blood pressure, nonHDL, and pulse (VARS-CVD+). We also tested the impact of a flexible machine learning model (Boosting) using the VARS-CVD+ predictors. We used training and testing datasets and fit all models separately for men and women. We compared the models using traditional measures of risk assessment.
The VARS-CVD+ models had better discrimination (AUROC = 0.702 in men, 0.753 in women) than the ASCVD and VARS-CVD models (AUROCs = 0.657-0.699). There was little difference between logistic regression (AUROC = 0.702-0.753) and boosting (AUROC = 0.705-0.737). Calibration was poor for ASCVD, but good for the other scores. Reclassification measures revealed significant differences in average risk and improvements in classification for both VARS-CVD+ scores over VARS-CVD, but no differences between logistic and boosting methods. For medium-risk patients according to VARS-CVD, about 20% had at least a 2.5% change in absolute risk when using VARS-CVD+ scores.
An expanded set of predictors improved risk prediction over models created from traditional risk factors. More flexible modeling methods did not result in significant advantages over traditional logistic regression methods.
A central goal of personalized medicine is to tailor treatment to individual patient risks, benefits, and preferences. This requires accurate information and an effective way to measure risk. An ASCVD risk score that takes advantage of EHR data and that is developed and validated on Veterans may help the VHA to provide better care and reduce over- and under-treatment.