The report is a product of the VA/HSR Evidence Synthesis Program.

Evidence Brief: Use of Performance Measures as Criteria for Selecting Community Surgical Providers

The Veterans Choice Program was established in 2014 to expand Veterans' access to community providers when VA medical facilities have long wait times or when geographic accessibility is excessively burdensome for Veterans (<than 40 miles). VA's Office of Community Care is considering adopting performance measures to help identify eligible community orthopedic and cardiac surgery providers that meet certain minimum requirements. However, a significant barrier in selecting performance measures is that while there are a large number meant as indirect indicators of health outcomes (e.g., readmissions, process measures), there is uncertainty about their association with health outcomes (i.e., mortality, quality of life, or function). The purpose of this rapid evidence review was to determine whether such performance measures are associated with health outcomes and to compare their measurement burden and unintended consequences. Findings will help inform the development of value-based community care purchasing.

Investigators with VA's Evidence-based Synthesis Program Coordinating Center located in Portland, OR searched multiple data sources (i.e., MEDLINE®, CINAHL, Cochrane Register of Controlled Trials, NHS Economic Evaluation, etc.) for relevant articles through June 19, 2017. After reviewing more than 3,000 potentially relevant articles, they included 38 studies (6 systematic reviews, 1 randomized controlled trial, and 31 observational studies) in their analysis.

Summary of Findings
Among performance measures meant as indirect indicators of health outcomes, 30-day readmission is the strongest indicator of 30-day mortality for CABG (coronary artery bypass grafting) and hip replacement, and is feasible to measure . CABG is consistently (although modestly) associated with 30-day mortality. For hip replacement, the evidence is weaker but also points toward 30-day readmission as being a valid indicator of 30-day mortality. In addition, 30-day readmission rates would likely be feasible to collect as all community Medicare providers are already mandated to collect this information. However, 30-day readmission has potential limitations that must be considered; for example, variability in and lack of consensus on methods to adjust for case-mix, controversy about including socioeconomic status, and difficulties accounting for readmissions to another hospital.

Also, contrary to recent criticism, the 30-day mortality measure is likely a valid surrogate for long-term survival, with a lower than expected risk of gaming (i.e., to meet a 30-day metric). Use of a robust and widely utilized composite measure of direct and indirect indicators of health outcomes also may be a highly feasible and comprehensive approach to determining eligibility of Choice Program providers.

Studies in this review also show that adherence to standardized CABG wait-time protocols, a set of cardiac surgery process measures, and a specific cardiac surgery antibiotic prophylaxis guideline-based protocol, but not other studied individual process measures, have decreased the likelihood of mortality and complications in cardiac surgery. For orthopedic surgery, multiple individual and composite CMS NQF (Centers for Medicare & Medicaid National Quality Forum) and SCIP (Surgical Care Improvement Project) quality measures were not valid indicators of either mortality or complications. Moreover, investigators did not find any studies of performance measures collected in the inpatient setting that directly assessed the quality of post-operative bundles of care.

Implications
Possible minimum requirements for Choice Program providers include:

• Acceptable performance on national rankings,
• Compatible operational infrastructure, and the
• Ability to comply with an agreed-upon wait-time threshold.

In addition to these minimum requirements, the Office of Community Care could consider the added value of the performance measures that this review has identified as being indicators of desirable health outcomes. These include the single measures of 30-day mortality, a direct health outcome measure, and/or 30-day readmission, the indirect measure with the strongest association with mortality – both of which are commonly measured by surgery programs.

Another option is to use a composite performance measure that includes mortality, readmission, and other process measures – such as the Society of Thoracic Surgeons' composite CABG measure. Also, investigators recommend the use of public reporting program participation and measures of efficiency as additional considerations for Choice Program community provider minimum standards. However, an unintended effect of stricter performance measure-based criteria to identify eligible Choice providers may be an undersupply of providers, which could diminish the Program's effect on reducing Veterans' wait times.

Future Research
To improve evidence about the validity of performance measures as indicators of surgical quality and to increase its applicability to broader populations, investigators recommend the following high priorities for future research evaluations:

• Broader array of modern surgical populations;
• Utility of social and environmental factors in risk prediction models;
• Associations with longer-term patient-important outcomes;
• Unintended consequences including gaming and risk-based patient selection, evaluation of appropriateness of care, changes in disparities, and spillover effects; and
• Post-operative plan of care measures, such as presence of pre-discharge assessment and ensuring every patient is scheduled for a follow-up visit.

Further, to improve capabilities to monitor Veterans' outcomes in the community and to compare to care within VA, Secretary Shulkin and others have called for the creation of data standards and standardized electronic data systems that would better permit aggregation of data across sites.

Reference
Peterson K, Anderson J, Bourne D, Boundy E, Helfand M. Evidence Brief: Use of Performance Measures as Criteria for Selecting Community Cardiac and Orthopedic Surgical Providers for the Veterans Choice Program. VA ESP Project #09-199; 2017.

Please feel free to forward this information to others!

Read past HSR&D Management e-Briefs on the HSR&D website.

ESP is currently soliciting review topics from the broader VA community. Nominations will be accepted electronically using the online Topic Submission Form. If your topic is selected for a synthesis, you will be contacted by an ESP Center to refine the questions and determine a timeline for the report.

This Management e-Brief is provided to inform you about recent HSR&D findings that may be of interest. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. If you have any questions or comments about this Brief, please email CIDER. The Center for Information Dissemination and Education Resources (CIDER) is a VA HSR&D Resource Center charged with disseminating important HSR&D findings and information to policy makers, managers, clinicians, and researchers working to improve the health and care of Veterans.

This report is a product of VA/HSR&D's Quality Enhancement Research Initiative's (QUERI) Evidence-Based Synthesis Program (ESP), which was established to provide timely and accurate synthesis of targeted healthcare topics of particular importance to VA managers and policymakers – and to disseminate these reports throughout VA.

See all reports online.