Study Compares Data Sources for Provider Financial Incentives
Quality improvement initiatives, including report cards and financial incentives linked to performance, must use reliable clinical performance measures. Understanding how well data from electronic health records (EHRs) reflect what is known about the patient is critical as national quality reporting programs transition from manual review to automated reports, since they are largely doing so without natural language processing of free text. Collecting these data via manual review is considered the gold standard, but is very time- and resource-intensive. This study examined how well data from automated processing of EHRs (AP-EHR) reflect data collected via manual chart review, and assessed the potential impact of data collection methods on incentive earnings for physicians and provider groups participating in a trial evaluating pay-for-performance for hypertension care. Participants included primary care physicians and practice team members (e.g., nurses) at 12 VA hospital-based outpatient clinics and 2,840 Veterans with hypertension who were treated between April and July 2009. Providers in the incentive groups earned payments for prescribing guideline-recommended anti-hypertensive medications, achieving blood pressure (BP) thresholds, and appropriately responding to uncontrolled BP; these were the main outcome measures. Incentive payments also were examined.
The total amount of incentives disbursed to providers would have been lower (by 10%) using the AP-EHR data to reward performance because this method under-reported the number of Veterans receiving appropriate medications – compared to manual review.
Regarding how well the AP-EHR reflect data from manual review, results show almost perfect agreement for the BP control measure: manual review indicated 70% of Veterans had controlled BP compared to 67% by AP-EHR review. Moderate agreement was found between the data sources for the use of guideline-recommended anti-hypertensive medication: manual review showed 72% of Veterans were considered to have received guideline-recommended anti-hypertensive medications compared to 65% by AP-EHR. And low agreement was found for the appropriate response to uncontrolled BP: manual review showed that 52% of Veterans received an appropriate response for uncontrolled BP compared to 40% by AP-EHR review.
- Pay-for-performance plans that use only EHR-derived data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement.
- Given the large amount of resources needed for chart review endeavors, investigators feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared to manual review is acceptable. Findings suggest that this data collection method could be implemented efficiently in VA pay-for-performance programs.
- Investigators were not able to examine why there was low agreement for appropriate response to uncontrolled BP.
- Quality measures relying on more than one data source may experience lower agreement.
This study was partly funded by HSR&D (IIR 04-349). Ms. Urech, Drs. Woodard and Virani, Ms. Lutschg, and Dr. Petersen are part of HSR&D's Center for Innovations in Quality, Effectiveness & Safety, Houston, TX.
Urech T, Woodard L, Virani S, Dudley RA, Lutschg M, and Petersen L. Calculations of Financial Incentives for Providers in a Pay-for-Performance Program: Manual Review Versus Data from Structured Fields in Electronic Health Records. Medical Care. October 2015;53(10): 901-907.