Performance measures coupled with incentives have been viewed as critical components driving the VA quality transformation in the late 1990s. Measures related to control of intermediate outcomes among patients with diabetes have shown striking improvements over time which is laudable, but there are signs that this success comes at a cost. There are concerns that rates of overtreatment and other unintended consequences are increasing as a result of the pressure to meet increasingly stringent goals and benchmarks. To date, measures have been dichotomous; e.g., a patient's lab values meet the goal or not. The Diabetes QUERI has helped develop more meaningful "clinical action" measures that move away from dichotomous intermediate outcome measures that may drive some of the overtreatment. Other unintended consequences may result not only from overly stringent, dichotomous intermediate outcome performance measures but also from the way those measures are implemented or used (or misused) at the local facility.
The overarching goal of this project was to assess gaps and unintended consequences of implementing existing and the new diabetes-related performance measures in VHA in order to provide feedback to Office of Analytics and Business Intelligence (OABI) regarding ways to improve measurement construction and implementation of new performance measures. The specific study aims were: 1) To understand consequences of diabetes performance measurement on providers and patients, particularly unintended consequences; 2) To explore how different local measurement implementation strategies affect consequences of diabetes performance measures; and 3) To explore new candidate performance measures to minimize unintended consequences such as over-treatment among patients with diabetes.
This study was a Phase 3 pre-implementation study to identify and understand unintended consequences and gaps in implementation of diabetes-related performance measures. We used a mixed methods design. We will purposively select a sample of medical centers based on their performance (highest, lowest) with respect to three diabetes measures (BP, LDL, HbA1c) and prevalence of over-treating BP to maximize variation among study sites. To explore and categorize unintended consequences of performance measurement and the process of implementation, we will conduct semi-structured interviews with VISN leadership, facility leadership, primary care providers, care managers and other staff identified by these providers, who are involved in providing care to patients with diabetes. In addition, guided by information from our qualitative interviews and our expert clinical work group, we will identify potential measures of inappropriate overtreatment and specify the construction of these measures.
We interviewed 59 leaders and staff from four VAMCs in four different VISNs and completed site visits at each facility.
Aim One & Aim Two: To understand consequences of diabetes performance measurement on providers and patients, particularly unintended consequences, and to explore how different implementation strategies play a role in the manifestation of those consequences.
We learned about several positive consequences of performance measures: 1) measures are often translated to clinical reminders, which help providers remember to address routine, preventive care with their patients; 2) high priority goals trigger quality improvement initiatives, such as introducing more Shared Medical Appointments, using clinical pharmacists to help manage complicated diabetes patients, and establishing integrated diabetes clinics.
However, we also identified the following unintended consequences of performance measures. These issues must be interpreted within the larger trend of marked improvement within the VHA, which has in part been driven by performance measurement:
1. Providers are disenfranchised from the development and implementation of performance measurement.
2. Performance measures are not well defined and are largely dichotomous. They are often translated directly to clinical reminders and conflated with treatment guidelines.
3. Executive Performance Plans (EPPs) are changed every year, but are not published until well into the fiscal year. Related performance measures and goals are not adequately communicated to providers; very few providers or leaders know about the newest VA/DoD diabetes treatment guidelines. This problem is exacerbated by high executive-level turn-over.
4. Primary Care providers shoulder a disproportionate share of the burden to meet performance goals.
5. Executive leadership and providers struggle with the vast number of measures they need to monitor. Many providers feel that the large number of clinical reminders degrades the provider-patient relationship.
6. The linkage between performance measures and performance-based pay is tenuous and misunderstood, and does not serve to motivate providers to meet measures.
7. EPRP data is increasingly inadequate and problematic as more facilities engage in "provider profiling."
8. Providers only receive negative feedback on performance goals via lists of patients who "fell out" on a measure. This causes angst and cognitive dissonance; providers believe they provide quality care to complex patients, yet they only receive feedback on their failure to meet performance measures.
Aim Three: Our analyses have revealed that increased age, greater healthcare use, and co-morbidity, as well as external changes in policies affecting treatment guidelines, contribute to overtreatment. Several events related to LDL-related treatment guidelines have provided the opportunity to assess treatment in response to these key events. June 2011 saw the highest percentage of patients (72%) with LDL<100, the "traditional" performance measure goal. However, in that same month, the FDA issued a recommendation limiting the use of the highest approved dose of the cholesterol-lowering medication simvastatin (80mg) because of increased risk of muscle damage. One year later, the Clinical Analysis Reporting (CAR), Office of Informatics and Analytics (OIA) announced a forthcoming revised measure where lipid control would be deemed appropriate if "the patient is receiving at least a moderate dose of a statin drug or LDL-cholesterol (LDL-C) value is 100 or less." Four months later in October 2012, a National Clinical Reminder was released. By November 2012, the percentage of patients with LDL<100 dropped to 69%. In this same time period, the percentage of patients on a high dose statin decreased by approximately 5%.
Findings from this study will be translated into specific recommendations for our partners in OABI: 1) modifications to existing measures as well as suggestions for new measures, so that intended consequences are maximized and unintended consequences are minimized; 2) development of measures that track over-treatment to balance measures that address under-treatment; and 3) multi-level implementation strategies that can be used in the field. Our findings will provide needed information to guide development of a bundled set of strategies to enhance implementation of new measures, setting the stage for a future SDP to evaluate how improvements in both implementation strategies and performance measurement affect quality of care for patients with diabetes. Performance measurement plays a prominent role in VHA. These types of improvements in measuring and implementing performance will ultimately improve patient safety and quality of care.
- Damschroder LJ, Robinson CH, Francis J, Bentley DR, Krein SL, Rosland AM, Hofer TP, Kerr EA. Effects of performance measure implementation on clinical manager and provider motivation. Journal of general internal medicine. 2014 Dec 1; 29 Suppl 4:877-84.
- Damschroder LJ, Robinson C, Francis J, Hofer T, Bentley D, Krein SL, Rosland A, Kerr EA. Bringing Performance Measurement into the Age of Patient-centered Care. Paper presented at: AcademyHealth Annual Research Meeting; 2014 Jun 8; San Diego, CA.