HSR&D Home » Research » DIS 99-221 – HSR&D Study
Evaluation of VISTA Performance Profiles and Non-VISTA Measures
Eve A. Kerr, MD MPH
VA Ann Arbor Healthcare System, Ann Arbor, MI
Ann Arbor, MI
Funding Period: December 1999 - November 2001
The VA has incorporated extensive diabetes measures in their External Peer Review Program (EPRP) quality measurement system, which is based on medical record abstraction.
While EPRP is VA’s gold standard and provides the most reliable diabetes quality information currently available, chart abstraction is very expensive. Certain process and intermediate outcome data (for example, whether a lab test was performed and its value) are available from the Veteran’s Integrated Health Systems Technology and Architecture (VISTA) system. However, the validity and reliability of these data are unknown.
The specific goals of the project include: (1) to examine the correlation of diabetes measures derived from VISTA measures with those derived from medical record and patient survey, and (2) to assess the ability of each of the measures to accurately identify facilities with higher or lower quality.
The project compares diabetic quality measures in the VA obtained from three existing data sources: VISTA, medical records and patient surveys (using data already collected by the VA Office of Quality and Performance (OQP) as part of a quality assurance activity). We have compare the quality measures derived from each data source to determine how they correlate and how each contributes to the variation in quality scores across facilities.
Success rates were higher for process measures derived from medical record versus automated data (e.g., 78% vs. 68% for LDL measured; 84% vs. 78% for A1c measured). This difference narrowed for intermediate outcome measures (e.g., 79% vs. 76% for LDL<130; 86% vs. 88% for A1c<9.5%). Agreement for measures derived from the medical record compared to automated data was moderate for process measures (e.g.,A1c measured, kappa=0.61) but high for intermediate outcome measures (e.g., A1c<9.5%, kappa=0.92). Hybrid measures, which use automated data supplemented with medical record data, yielded success rates similar to those of medical record based measures. Hybrid process measures would require medical record review in only 50% of cases. Process measures, but no intermediate outcome measures, showed significant variation attributable to the facility, regardless of the data source.
We found that agreement between medical record and automated data was generally high. Nonetheless automated data tended to underestimate the success rate in process measures for diabetes. Applying hybrid methodology yielded results consistent with the medical record but required less data to come from medical record reviews. Despite the high rates in overall performance, further research should examine the underlying reasons for facility level variation in diabetes process measures in order to craft appropriate quality improvement programs.
External Links for this Project
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Health Systems
DRE: Technology Development and Assessment
Keywords: Diabetes, Quality assessment, Risk adjustment
MeSH Terms: none