Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website
FORUM - Translating research into quality health care for Veterans

» Back to Table of Contents

Building a Better VA Performance Measurement System

Why measure quality of care? The answer to this question might seem obvious, but in thinking about how we can build a better quality measurement system, we must first clarify the purpose of this sometimes labor intensive activity. The most obvious answer is that measuring performance is an essential step in guiding quality improvement. But no measurement set is comprehensive, and there is always a danger that tracking performance for selected indicators will lead to improvement in those limited areas at the expense of equally important areas not targeted by the performance measurement system. So, a second purpose to any quality measurement system must also be to represent the broader and underlying quality provided in areas beyond the system. With luck, a quality measurement system will lead to improvement in those areas as well.

The VHA performance measurement system is among the most widely implemented in the world. Managers receive regular reports as to how their regions or facilities are doing on selected measures of essential processes of care and intermediate outcomes. They pass these incentives down to line providers. What is the evidence that performance measurement has improved care? While it is difficult to disentangle the effects of performance measurement from other components of VHA reorganization that began in the 1990s, it is clear that VHA performance on the tracked items has surpassed competing systems. For example, Jha et al. compared the VHA to Medicare and found better performance for 12 of 13 measures. But was this teaching to the test? How much of the improvement was confined to the tracked areas?

A few years ago, Eve Kerr, Beth McGlynn, myself, and others investigated this question using a very broad range of quality indicators included in RAND's QA Tools measurement set. We confirmed that the VHA outperformed a community sample overall on basic process measures. In the tracked areas, the difference was marked. VHA patients received 66 percent of recommended care as compared to 43 percent, a 23 point advantage. In unrelated measures of quality, the VHA also did slightly better, but the difference was much smaller, on the order of 5 percent. To paraphrase Kevin Costner in Field of Dreams, this is evidence that if you build it (the performance measurement system, that is), they will come.

So far, it seems that performance measurement is bearing fruit in its primary purpose of guiding quality improvement to measured areas, but not for the secondary purpose of engineering more widespread improvement. Interestingly, however, we found that the VHA performed better on measures that were related, but not the same as, the targeted areas. A related measure might be an aspect of diabetes that was not in the performance set, or an immunization that was not directly tracked like influenza vaccination. The VHA advantage here was real, 12 percent. We hypothesize that this is due to a chain reaction effect in the minds of the providers, who now think about diabetes care more because of performance measurement, but do so more holistically than the performance measurement set would require. Likewise, clinic managers have adjusted their thinking by building systems to make it easier to vaccinate for influenza and thus ease other vaccinations as well. So, if you build it, they will come, and they might stay at your neighbor's house, too.

What are the implications for building a better performance measurement set? First, we cannot afford to ignore important broad areas of care, because if we do, we may fail to spark quality improvement in related areas. For example, the VHA performance measurement system had ignored acute care in the past, and this is now being remedied. Second, we should empirically test how well leading indicator systems represent broader concepts of quality of care to help us choose leading indicators more wisely. The VHA system is widely admired, and considerations such as these will keep it at the forefront.