Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
FORUM - Translating research into quality health care for Veterans

» Back to Table of Contents


Providing Optimal Care Is Complex — And So Is Measuring It

Performance measurement is a powerful policy tool for promoting efficient high-quality care. However, it is often underestimated just how harmful performance measures can be when they are poorly constructed.1,2,3 Perhaps the hardest lesson is that developing performance measures that promote optimal care usually requires clinically detailed data and complex measures, and that simplistic or naive "good care" measures can have strong perverse incentives.

This push for simple measures is strongest in the community where the resources and will to invest in electronic medical records and chart-based review have been lacking.3 However, since there is considerable political pressure for the VA to adopt those measures used in the community (so as to allow benchmarking), the recent push for all-or-none "good care" measures is of considerable concern (e.g., A1c < 7 percent, blood pressure < 130/80, LDL < 100mg/dl, etc). Such measures are likely to be very imprecise measures of efficient high-quality care, and are also prone to perverse incentives, such as promoting treatment irrespective of how small the potential benefit and how great the patient burden or risks.1,2,3

Take annual diabetes eye exams for example. Although well-timed photo coagulation for early diabetic retinopathy is one of the most beneficial treatments in all of medicine, research conducted by VA HSR&D suggests that almost all visual impairment that is preventable by early detection will be captured by: 1) screening those with no known eye disease every two to three years, and 2) close individualized surveillance (every 4 to 12 months) after early retinopathy has been detected.4 Therefore, annual exams are not of high importance for the vast majority of low-risk patients, and are too infrequent for the highest risk patients who will account for most preventable complications. As a result, the conventional performance measure (annual examinations for diabetic patients), which provides a strong incentive to focus resources on getting everyone into clinic every 13 months, provides no incentive to develop an effective system to optimize care.

In fact, a health system that schedules exams every 10 to 11 months and devotes its scarce administrative resources to trying to get anyone who misses this appointment into the clinic as soon as possible, is likely to improve its performance rating while doing almost nothing to improve outcomes. In contrast, a health system that uses its administrative resources to aggressively reschedule those needing close follow up (because of known retinopathy) and to have low-risk patients seen at least every two years (those whose last retinal exam was normal on their last visit), may make its performance rating much worse while substantially improving patient outcomes.

It should not be surprising therefore that when we tried to develop an effective system to improve eye screening and follow up for diabetic patients, that the prevailing annual eye exam performance measure was one of the biggest barriers to implementing the more effective system. Although many may criticize clinic leaders for not doing the "right" thing, these clinics and providers have huge demands on their time and attention. As has happened with so many important quality improvement initiatives that we've consulted on in the past 10 years, the clinicians and administrators eventually said, "If the problem you want us to address is really that important, then get the performance measure changed. We are struggling to meet dozens of demands and we just do not have the time and personnel to electively take on more things."

It is rare that "good care" can be measured simply. Although the new NCQA diabetes measures of A1c < 7 percent and BP < 130/80 may seem straightforward enough, they actually provide strong incentives for speculative, costly, and potentially dangerous polypharmacy. In addition, these are unadjusted outcome measures that are likely to be inaccurate. For example, the A1c measure more strongly rewards adding or increasing the dose of a glitazone, which has high costs and limited data on long-term safety, in someone with an A1c of 7.5 percent than it does for doing so in someone with an A1c of 8.5 percent, even though a risk of an A1c of 7.5 percent is trivial compared to the risk of an A1c of 8.5 percent.

Similarly, although adding up to three to four medications at moderate doses in pursuit of good blood pressure control in a high CV-risk patient has been shown to be highly beneficial, the benefits of pursuing the 130/80 targets using more than three to four medications is pure speculation. Furthermore, there is ample reason to be concerned about harmful effects from polypharmacy or excessively lowering diastolic blood pressure.

Trying to measure a complex clinical scenario using simplistic performance measures and wishful thinking is increasingly promoted by disease advocates and industry-sponsored experts in the community. NCQA's new "good control" diabetes measures and its resistance to revising its eye care measure despite the strong advice of evidence-based medicine experts is just one example of this trend. This is not to suggest that we should limit ourselves to only measuring bad care, such as A1c > 9 percent or retinal screening exams > two years. However, if we wish to measure "good care," we will need more nuanced performance measures that consider the benefit of reaching these "optimal" treatment goals, as well as the risks, costs, and patient burden associated with treatments needed to reach these optimal goals.

  1. Hayward RA. Performance Measurement in Search of a Path. New England Journal of Medicine 2007; 356(9):951-3.
  2. McMahon LF, Hofer TP, Hayward RA. Physician-level P4P-DOA? Can Quality-based Payment Be Resuscitated? American Journal of Managed Care 2007; 13(5):233-6.
  3. Hayward RA. All-or-nothing Treatment Targets Make Bad Performance Measures. American Journal of Managed Care 2007; 13(3):126-8.
  4. Hayward RA, et al. Causes of Preventable Visual Loss in Type 2 Diabetes Mellitus: An Evaluation of Suboptimally-timed Retinal Photocoagulation. Journal of General Internal Medicine 2005; 20:467-9.

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.