» Back to Table of Contents
Why measure quality of care? The answer
to this question might seem obvious, but in
thinking about how we can build a better
quality measurement system, we must first
clarify the purpose of this sometimes labor
intensive activity. The most obvious answer
is that measuring performance is an essential
step in guiding quality improvement.
But no measurement set is comprehensive,
and there is always a danger that tracking
performance for selected indicators will lead
to improvement in those limited areas at
the expense of equally important areas not
targeted by the performance measurement
system. So, a second purpose to any
quality measurement system must also be to
represent the broader and underlying quality
provided in areas beyond the system. With
luck, a quality measurement system will lead
to improvement in those areas as well.
The VHA performance measurement system
is among the most widely implemented
in the world. Managers receive regular
reports as to how their regions or facilities
are doing on selected measures of essential
processes of care and intermediate outcomes.
They pass these incentives down
to line providers. What is the evidence that
performance measurement has improved
care? While it is difficult to disentangle the
effects of performance measurement from
other components of VHA reorganization
that began in the 1990s, it is clear that
VHA performance on the tracked items has
surpassed competing systems. For example,
Jha et al. compared the VHA to Medicare
and found better performance for 12 of 13
measures. But was this teaching to the test?
How much of the improvement was confined
to the tracked areas?
A few years ago, Eve Kerr, Beth McGlynn,
myself, and others investigated this question
using a very broad range of quality indicators
included in RAND's QA Tools measurement
set. We confirmed that the VHA outperformed
a community sample overall on
basic process measures. In the tracked areas,
the difference was marked. VHA patients
received 66 percent of recommended care as
compared to 43 percent, a 23 point advantage.
In unrelated measures of quality, the
VHA also did slightly better, but the difference
was much smaller, on the order of
5 percent. To paraphrase Kevin Costner in
Field of Dreams, this is evidence that if you
build it (the performance measurement system,
that is), they will come.
So far, it seems that performance measurement
is bearing fruit in its primary purpose
of guiding quality improvement to measured
areas, but not for the secondary purpose
of engineering more widespread improvement.
Interestingly, however, we found that
the VHA performed better on measures
that were related, but not the same as, the
targeted areas. A related measure might be
an aspect of diabetes that was not in the
performance set, or an immunization that
was not directly tracked like influenza vaccination.
The VHA advantage here was real,
12 percent. We hypothesize that this is due
to a chain reaction effect in the minds of
the providers, who now think about
diabetes care more because of performance
measurement, but do so more holistically
than the performance measurement set
would require. Likewise, clinic managers
have adjusted their thinking by building
systems to make it easier to vaccinate for
influenza and thus ease other vaccinations
as well. So, if you build it, they will come,
and they might stay at your neighbor's
house, too.
What are the implications for building a
better performance measurement set? First,
we cannot afford to ignore important broad
areas of care, because if we do, we may fail
to spark quality improvement in related
areas. For example, the VHA performance
measurement system had ignored acute care
in the past, and this is now being remedied.
Second, we should empirically test how
well leading indicator systems represent
broader concepts of quality of care to help
us choose leading indicators more wisely.
The VHA system is widely admired, and
considerations such as these will keep it at
the forefront.
|