Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
Publication Briefs

Study Compares Data Sources for Provider Financial Incentives

Quality improvement initiatives, including report cards and financial incentives linked to performance, must use reliable clinical performance measures. Understanding how well data from electronic health records (EHRs) reflect what is known about the patient is critical as national quality reporting programs transition from manual review to automated reports, since they are largely doing so without natural language processing of free text. Collecting these data via manual review is considered the gold standard, but is very time- and resource-intensive. This study examined how well data from automated processing of EHRs (AP-EHR) reflect data collected via manual chart review, and assessed the potential impact of data collection methods on incentive earnings for physicians and provider groups participating in a trial evaluating pay-for-performance for hypertension care. Participants included primary care physicians and practice team members (e.g., nurses) at 12 VA hospital-based outpatient clinics and 2,840 Veterans with hypertension who were treated between April and July 2009. Providers in the incentive groups earned payments for prescribing guideline-recommended anti-hypertensive medications, achieving blood pressure (BP) thresholds, and appropriately responding to uncontrolled BP; these were the main outcome measures. Incentive payments also were examined.


  • The total amount of incentives disbursed to providers would have been lower (by 10%) using the AP-EHR data to reward performance because this method under-reported the number of Veterans receiving appropriate medications – compared to manual review.
  • Regarding how well the AP-EHR reflect data from manual review, results show almost perfect agreement for the BP control measure: manual review indicated 70% of Veterans had controlled BP compared to 67% by AP-EHR review. Moderate agreement was found between the data sources for the use of guideline-recommended anti-hypertensive medication: manual review showed 72% of Veterans were considered to have received guideline-recommended anti-hypertensive medications compared to 65% by AP-EHR. And low agreement was found for the appropriate response to uncontrolled BP: manual review showed that 52% of Veterans received an appropriate response for uncontrolled BP compared to 40% by AP-EHR review.


    • Pay-for-performance plans that use only EHR-derived data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement.
    • Given the large amount of resources needed for chart review endeavors, investigators feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared to manual review is acceptable. Findings suggest that this data collection method could be implemented efficiently in VA pay-for-performance programs.


    • Investigators were not able to examine why there was low agreement for appropriate response to uncontrolled BP.
    • Quality measures relying on more than one data source may experience lower agreement.

    This study was partly funded by HSR&D (IIR 04-349). Ms. Urech, Drs. Woodard and Virani, Ms. Lutschg, and Dr. Petersen are part of HSR&D's Center for Innovations in Quality, Effectiveness & Safety, Houston, TX.

    PubMed Logo Urech T, Woodard L, Virani S, Dudley RA, Lutschg M, and Petersen L. Calculations of Financial Incentives for Providers in a Pay-for-Performance Program: Manual Review Versus Data from Structured Fields in Electronic Health Records. Medical Care. October 2015;53(10): 901-907.

  • Related Briefs

    » next 21 Data Use Briefs...

    What are HSR Publication Briefs?

    HSR requires notification by HSR-funded investigators about all articles accepted for publication. These journal articles are reviewed by HSR and publication briefs or summaries are written for a select number of articles that are then forwarded to VHA Central Office leadership to keep them informed about important findings or information. Articles to be summarized are selected by HSR based on timeliness of the findings, interest of leadership, or potential impact on the organization. Publication briefs are written for only a small number of HSR published articles. Visit the HSR citations database for a complete listing of HSR articles and presentations.

    Questions about the HSR website? Email the Web Team

    Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.