2011 National Meeting

2009 — Validation of Process Quality Measures

Sox-Harris A (VA Palo Alto) , Finney JW (VA Palo Alto)

Workshop Objectives:
Improving health care is contingent on having valid definitions and measures of quality. Quality measures are used to identify high- and low-performing facilities, to monitor the effects of system-wide initiatives, to reward effective management, and to incentivize particular patient care practices and outcomes. Process of care (PoC) quality measures attempt to quantify the extent to which certain health care practices and procedures occur when and for whom they are intended. However, such measures are often developed and implemented without proper validation. Using PoC quality measures without direct evidence of their validity has serious risks, including incentivizing poor or incomplete care, diverting clinical and financial resources away from higher impact activities, and creating skepticsm towards the entire quality improvement enterprise. Validation in this context refers to checking that the specifications of the measures faithfully identify the targeted processes and patients, that the assumed links between process and outcomes exist, and that these two types of validity do not significantly vary from site to site. The workshop has the following objectives: 1) To describe the inherently multi-level nature of quality measures and their applications (hence the need for multi-level data and models); 2) Summarize methods for evaluating how well quality measure specifications and available data capture the intended care process; 3) Present specific, multi-level statistical models for assessing the associations between PoC measures and outcomes, 4) Introduce advanced quality measure diagnostics, including checking site-level moderation, comparing the measurement characteristics of competing specifications, and assessing the presence of gaming by selective case identification.

Activities:
Workshop material will be presented in lecture format, using real examples involving VA health care data and quality measures. Ample time will be allowed for audience questions and participation.

Target Audience:
The workshop will be useful for individuals involved in the development, validation, implementation, and interpretation of quality measures. Other stakeholders in the validity of quality measures (e.g., clinicians, clinical managers, policy makers, hospital directors) will also benefit by learning how even seemingly highly reasonable process measures can have serious problems.

Assumed Audience Familiarity with Topic:
Some background in statistical regression models is useful but not necessary.