skip to page content
Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Systems Research

Go to the VA ORD website
Go to the QUERI website

IIR 09-370 – HSR Study

IIR 09-370
Facility Mortality and Quality of Care among VA Outpatients with Diabetes
Alfredo J. Selim, MD MPH
VA Bedford HealthCare System, Bedford, MA
Bedford, MA
Funding Period: April 2010 - March 2012
Diabetes mellitus is accompanied by serious complications and reduced life expectancy. It affects approximately 25% of patients receiving care in the Veterans Health Administration (VA).

In approximately 80% of adult patients with diabetes death results from cardiovascular disease, most often coronary artery disease or cerebrovascular disease. These are conditions that must be satisfactorily managed in order to improve survival.

Over recent decades a number of ambulatory care interventions have been shown to result in a substantial reduction in mortality due to cardiovascular disease. Thus, it is reasonable to expect that high quality outpatient care can achieve a meaningful improvement in the overall survival of patients with diabetes. It is known, however, that quality of care for patients with diabetes varies across VA facilities and it is likely that there is variation in patient mortality well. In preliminary work, we have found VISN-level effects on survival that support this probability. This variation in VISN-level mortality raises concern that there will be still larger variation across facilities, because facilities are more heterogeneous than VISNs in the quality of care that they provide.

We have developed and validated a risk adjustment method for mortality rates of ambulatory care patients over the course of two previously funded VA HSR&D IIR proposals. We now have optimized it for patients with diabetes in order to examine the relationship between the quality of ambulatory care and the mortality of patients with diabetes across VA facilities. Our overall objective was to obtain actionable information that would make it possible to reduce the mortality of VA patients with diabetes.

1. To modify our risk-adjustment method so as to optimize its performance in predicting the mortality of ambulatory care patients with diabetes.

2. To determine the degree to which facility mortality rates correlate with life-prolonging goals of treatment (lipid lowering medication, aspirin and smoking cessation).

This observational study used data collected between Fiscal Years 2002 and 2008 from the Survey of Health Experiences of Patients, a prospective monitoring system of outcomes of patients receiving ambulatory care in the Veterans Health Administration (VA).

Our first objective was to refine our original risk adjustment method by accounting for the particular complications of patients with diabetes that increase mortality. We adapted the weights given to the conditions in the Charlson Index to make them "diabetes specific". To do this, we re-calculated the Cox proportional hazards model that Deyo used for predicting mortality. Model performance was tested in a development sample of 226,502 patient records (2/3 randomly sampled) and then the coefficient values were applied to a validation sample of 113,249 patient records (the other 1/3). The performance analysis included a c-statistic, which is a measure of the model's predictive ability to discriminate among patients by ordering them according to rates of the outcome event. We used the Hosmer-Lemeshow test to evaluate the calibration of the model. We determined whether case-mix adjusted mortality rates alter assessment of facility performance.

Our second objective was to determine the degree to which facility mortality rates correlate with life-prolonging treatment effects (lipid lowering medication use, smoking cessation, and use of aspirin to prevent recurrent cardiovascular events). We correlated facility mortality rates and life-prolonging treatment effects using patient measures aggregated to the facility level, using Pearson correlation coefficients. Our study was powered to detect correlations as small as 0.23. We considered correlations greater than or equal to this to be important and meaningful for the use of outpatient mortality as a quality measure. To study these variables' effect on mortality at the patient level and to compare across facilities we used a hierarchical logistic model with random effects for facilities.

Of the 262,937 diabetic patients, 7,312 (2.78%) died during the 12-months of follow-up. Observed mortality rates across the 127 VA facilities varied significantly from 1.0% to 3.6% (p <0.0001). Age, gender, modified Charlson Index, physical health, and mental health were significant predictors of dying. The resulting risk-adjusted mortality model performed well in cross-validated tests of discrimination (c-statistic = 0.802) and fit (likelihood ratio test= 3216.75). Analysis of variance confirmed that the 127 facilities differed in their average level of expected risk (p <0.001). The facilities' risk-adjusted rates and ranks differed considerably from unadjusted ratings. Thus, risk adjustment is essential to accurate measurement and understanding of variations in performance.

We examined the association between the variations across facilities of life-prolonging treatments (lipid lowering medications, aspirin and smoking cessation) with mortality. On average, 67% of the patients were on a lipid lowering medication, 37% of the patients were on aspirin and 39% of the patients were "currently" active smokers. The correlation coefficient between the proportion of lipid lowering medication use, aspirin and smoking cessation and the observed/expected mortality ratio were -0.11, 0.16 and 0.19, respectively. Hierarchical regression models were run separately for each life prolonging treatment. When we compared the random effects from the risk-adjustment only hierarchical model to those from the model that includes the life prolonging treatment, by calculating the square root of the facility random effect estimate (you can think of this as 1 standard deviation in the rates across facilities on the logit scale), these models also showed a weak association.

We should highlight several important findings. First, we developed a risk-adjustment model for mortality in patients with diabetes receiving outpatient care in the VA that contains clinically credible variables and that has good statistical performance. Second, the risk adjustment model revealed that case-mix differences do exist across VA facilities. Third, the risk-adjustment model application changes the assessment of facility performance and identifies outliers with good or bad performance. Fourth, it shows promise for evaluating quality of care, making cross-facility comparisons, and identifying best practices and model procedures. Fifth, when examining life-prolonging treatments, we have found that the VA has a remarkable consistency of care across facilities nationwide. Such small degree of variation in life prolonging treatment was not sufficient to explain the variation in mortality across facilities. New studies should focus on other aspect of quality of care such as the organizational structure and function of VA facilities that might affect the degree to which patient with diabetes are provided with treatment that are known to reduce the morbility and mortality of cardiovascular.

External Links for this Project

NIH Reporter

Grant Number: I01HX000358-01

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


None at this time.

DRA: Diabetes and Other Endocrine Disorders
DRE: Treatment - Comparative Effectiveness
Keywords: none
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.