|
|
Health services researchers in VA have
the benefit of working within a health care
organization that is engaged in one of the
boldest, large scale initiatives regarding
performance measurement. Dr. Kussman's
article explains what the next generation
of that performance measurement system
will be, and how it relates to the Executive
Career Field plan. This new initiative offers
many exciting opportunities for health
services researchers interested in quality
measurement and improvement.
First, there is the traditional role of health
services research in understanding and
refining the individual measures themselves.
How reliable is measurement? How
accurately does the measure assess what is
considered "quality" for this condition? Is
it useful for distinguishing between providers?
We know, for example, from work
done by Eve Kerr and colleagues that existing
External Peer Review Process (EPRP)
measures correlate significantly but modestly
with two other ways of assessing quality—
using a larger and more comprehensive set
of process measures, and the implicit review
of medical records by peers.1
The finding by Kerr et al. suggests that
each of these measurement systems provides
a valid assessment of quality but each
is also only a part of the picture. What are
the implications of this finding? How can
the measurement be improved? Health services
research has a long history of doing
this kind of "basic science" on the properties
of quality measures. As new measures
get proposed for inclusion in VA's measurement
set, and as we learn more about
what processes influence patient outcomes,
there is—literally—an endless stream of
this kind of work that needs doing.
I would like to suggest that health services
researchers get more involved in another kind
of research, which looks at performance measurement
as an intervention. An intervention
that is complex, which includes many components,
such as the process that sets what the
measurement criteria are, activities to achieve
buy-in by people in the field, mechanisms for
providing feedback, benchmarking, the use
of a modest financial incentive to stimulate
performance, etc. As an intervention, VA's
performance measurement is likely influenced
by many factors—at the VISN level, at the
facility level, even perhaps at the individual
level. And with 21 VISNs and hundreds of
facilities, these factors are going to have a
natural degree of variation throughout VA.
What kinds of components in this multi-component
intervention, and what organizational
characteristics, are associated with varying
"success"? Can we identify specific organizational
features and characteristics of individual
intervention components that lead to greater
"success"? Does performance measurement
produce unintended consequences, and how
can these be minimized? Do some measures
or implementation methods work better than
others to promote improvement and minimize
unintended consequences? Research to
date has only begun to generate the needed
insights regarding how best to implement
large policy interventions such as performance
measurement and feedback.
The needed research is much messier than
many researchers like—it is rarely feasible to
study these policies experimentally, meaning
that inference is going to have to come from
observational studies. In most circumstances,
it is not even possible to study policy implementation
in a controlled fashion. I think it
unlikely that policymakers would accept on
a regular basis the implementation of performance
measurement components on some
facilities and not others. But just because it is
more difficult does not mean we should shy
away from the task. Indeed, VA health services
researchers should strive to be at the leading
edge of developing the methods by which
generalizable knowledge can result from the
study of real world policy initiatives.
There are already resources within VA to
help us. The VA's QUERI program has been
generating useful lessons about the difficulties
in studying the implementation of complex
interventions. Implementation Science, co-edited
by Brian Mittman of the HSR&D Center
at Sepulveda, publishes research in this field
(visit www.implementationscience.com). We
have an exciting task in front of us. It's going
to be a challenge. But it is also going to offer
great rewards, in terms of understanding how
better to improve the delivery of care by VA,
and by extension, other health systems.
- Kerr E, et al. Quality by Any Other Name: A
Comparison of Three Profiling Systems for Assessing
Health Care Quality, Health Services Research, in press.
top
|
