Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
FORUM - Translating research into quality health care for Veterans

» Back to Table of Contents


The Role of Health Services Research in Performance Measurement

Health services researchers in VA have the benefit of working within a health care organization that is engaged in one of the boldest, large scale initiatives regarding performance measurement. Dr. Kussman's article explains what the next generation of that performance measurement system will be, and how it relates to the Executive Career Field plan. This new initiative offers many exciting opportunities for health services researchers interested in quality measurement and improvement.

First, there is the traditional role of health services research in understanding and refining the individual measures themselves. How reliable is measurement? How accurately does the measure assess what is considered "quality" for this condition? Is it useful for distinguishing between providers? We know, for example, from work done by Eve Kerr and colleagues that existing External Peer Review Process (EPRP) measures correlate significantly but modestly with two other ways of assessing quality— using a larger and more comprehensive set of process measures, and the implicit review of medical records by peers.1

The finding by Kerr et al. suggests that each of these measurement systems provides a valid assessment of quality but each is also only a part of the picture. What are the implications of this finding? How can the measurement be improved? Health services research has a long history of doing this kind of "basic science" on the properties of quality measures. As new measures get proposed for inclusion in VA's measurement set, and as we learn more about what processes influence patient outcomes, there is—literally—an endless stream of this kind of work that needs doing.

I would like to suggest that health services researchers get more involved in another kind of research, which looks at performance measurement as an intervention. An intervention that is complex, which includes many components, such as the process that sets what the measurement criteria are, activities to achieve buy-in by people in the field, mechanisms for providing feedback, benchmarking, the use of a modest financial incentive to stimulate performance, etc. As an intervention, VA's performance measurement is likely influenced by many factors—at the VISN level, at the facility level, even perhaps at the individual level. And with 21 VISNs and hundreds of facilities, these factors are going to have a natural degree of variation throughout VA.

What kinds of components in this multi-component intervention, and what organizational characteristics, are associated with varying "success"? Can we identify specific organizational features and characteristics of individual intervention components that lead to greater "success"? Does performance measurement produce unintended consequences, and how can these be minimized? Do some measures or implementation methods work better than others to promote improvement and minimize unintended consequences? Research to date has only begun to generate the needed insights regarding how best to implement large policy interventions such as performance measurement and feedback.

The needed research is much messier than many researchers like—it is rarely feasible to study these policies experimentally, meaning that inference is going to have to come from observational studies. In most circumstances, it is not even possible to study policy implementation in a controlled fashion. I think it unlikely that policymakers would accept on a regular basis the implementation of performance measurement components on some facilities and not others. But just because it is more difficult does not mean we should shy away from the task. Indeed, VA health services researchers should strive to be at the leading edge of developing the methods by which generalizable knowledge can result from the study of real world policy initiatives.

There are already resources within VA to help us. The VA's QUERI program has been generating useful lessons about the difficulties in studying the implementation of complex interventions. Implementation Science, co-edited by Brian Mittman of the HSR&D Center at Sepulveda, publishes research in this field (visit www.implementationscience.com). We have an exciting task in front of us. It's going to be a challenge. But it is also going to offer great rewards, in terms of understanding how better to improve the delivery of care by VA, and by extension, other health systems.

  1. Kerr E, et al. Quality by Any Other Name: A Comparison of Three Profiling Systems for Assessing Health Care Quality, Health Services Research, in press.

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.