Talk to the Veterans Crisis Line now
An official website of the United States government
IIR 11-088
Developing and Validating a Veterans Affairs Cardiac Risk Score
Rodney A. Hayward, MD VA Ann Arbor Healthcare System, Ann Arbor, MI Ann Arbor, MI Funding Period: September 2012 - August 2016 Portfolio Assignment: Healthcare Informatics |
|
BACKGROUND/RATIONALE:
Evidence-based medicine (EBM) has hugely advanced medical care by answering the question: "Which treatments work on average in a population?" Personalized medicine seeks to utilize EBM and other tools to answer the patient's question: "Am I likely to benefit and how much?" Thus, a central goal of patient-centered medical care is to tailor treatment agreements toward individual patient risks, benefits, and preferences. This benefit is now recognized in major guidelines for cholesterol and aspirin. Effective tailoring, however, cannot be possible without accurate predictions of individual patients' risks of developing major medical illness. While multivariate risk/benefit prediction to individually tailor treatments could greatly increase treatment precision, tools to facilitate such improvements in VHA are lacking. For cardio-and cerebrovascular disease (CCV), which is the leading cause of morbidity and mortality in the U.S., current risk prediction tools have substantial shortcomings, including requiring manual entry of risk factor information, being developed and calibrated on patient populations quite different from VHA's (i.e., poor generalization to other populations), and failing to use new data-mining techniques or the robust continuum of clinical data available in VA's electronic medical record (VA EMR). To address these CCV treatment concerns, this project developed a VA-based risk prediction score to address the major concern of external validity, in addition to developing novel clinical algorithms to tailor clinical CCV decision-making and risk/benefit communication to individual Veterans. OBJECTIVE(S): The main goal of this project was to create a maximally effective risk prediction tool that is conceptually and clinically distinct from other commonly used risk-prediction tools. To reach this goal, this study had two overarching specific aims: 1) To develop and assess the merits of two competing approaches to developing VA EMR-derived CCV risk prediction tools, one using standard regression models (REG models) and one using machine learning models (ML Models); and, 2) To compare the accuracy and clinical impact of these VA EMR-derived CCV risk prediction tools to each other and to commonly used risk prediction models developed outside of VA, such as the Framingham and Euro SCORE risk tools using traditional risk assessment techniques and newly-developed, clinically-relevant algorithms. METHODS: Our study population consisted of established national VHA patients age 45 and older. All primary analyses were conducted on those without known CCV, while those with a history of CCV disease or those receiving a procedure for CCV conditions during the past two years were assumed to have known CCV disease and therefore excluded. We created a national, longitudinal cohort using laboratory and pharmacy data from the DSS National Data Extracts and CDW outpatient pharmacy; outpatient and inpatient ICD-9 and CPT codes from the SAS Medical Datasets and VA-CMS data; clinical measures from the Corporate Data Warehouse; cause of death from the National Death Index. Our analytic approaches included regression-based predictive models (REG models) utilizing logistic regression methods, in addition to using machine learning models (ML models) using Random Forests and Boosting analytic techniques. We conducted an internal validation of our major CCV and fatal CCV models using REG and ML by examining the performance of these models using two separate cohorts of randomly selected subjects consisting of two-thirds of the population for the development dataset and one-third for the validation dataset. We compared our developed risk score (VARS-CVD) to other commonly used risk prediction models developed outside of VA, including the American College of Cardiology/American Heart Association's (ASCVD) Score, Framingham Risk Score, UKPDS Risk Score, and EuroSCORE. FINDINGS/RESULTS: Using an outcome of CVD events during a five year follow-up period, our study population consisted of 1,512,092 patients (1,435,937 men and 76,155 women). For Aim 1, the developed VARS-CVD score had good calibration and passable discrimination. Results include a C-statistic of 0.66 in men and 0.73 in women. Machine learning models did not offer any statistical advantages over traditional logistic regression. For Aim 2, we compared the VARS-CVD score to the ASCVD score, which has recently become the field's standard based on ACC/AHA cholesterol guidelines. The VARS score had similar discrimination to the ASCVD score, but much better calibration. The ASCVD score predicted 63% more events than what we observed. These findings suggest that large managed care organizations can use their own electronic health records and administrative outcome data to create a cardiovascular disease (CVD) risk prediction score with major advantages over those developed in traditional cohort studies. The ASCVD score had an AUROC (which is a measure of discrimination) that was similar to our internal, health-system specific VARS-CVD score; however, ASCVD systematically over-estimated the observed risk in our cohort by almost 60%. Moreover, these findings demonstrate that our internally developed score is much better calibrated than the externally-developed score. Furthermore, the ASCVD score classified 46,508 patients as moderate or high-risk, a level at which statin therapy is recommended, while these same patients were classified as low-risk using the VARS-CVD risk tool. IMPACT: We have developed a model that can be used as an automated VA EMR-based CCV risk prediction tool that can be used to help tailor clinical decision-making to individual Veteran circumstances, thereby increasing efficiency of care management and aiding patient-centered decision-making for Veterans. This project's findings should have dramatic policy and clinical implications. Our work demonstrates that clinical risk scores can be made efficiently within an individual healthcare system's EHR. This work could be followed at other managed care organizations, in addition to being adjusted to many other clinical conditions. As EHRs improve and "Big Data" becomes more central to patient self-management, our findings offer a blueprint for how these tools can be used in clinical practice, preventing CVD events, and guiding statin use for patients who are more likely to benefit. Furthermore, our VA-specific Cardiac Risk Score could have two major impacts on healthcare in VHA: 1) The VARS-CVD score could be easily integrated into practice via the VA EMR or a web-based interface; and, 2) By improving how we use the basic medications for CCV prevention, saving lives and expense. External Links for this ProjectNIH ReporterGrant Number: I01HX000681-01A2Link: https://reporter.nih.gov/project-details/8397772 Dimensions for VA Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects. Learn more about Dimensions for VA. VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project PUBLICATIONS:Journal Articles
DRA:
Aging, Older Veterans' Health and Care, Health Systems Science, Cardiovascular Disease
DRE: Epidemiology, Treatment - Observational Keywords: none MeSH Terms: none |
