Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

IIR 15-436 – HSR Study

IIR 15-436
Using Data Integration and Predictive Analytics to Improve Diagnosis-Based Performance Measures
Katherine J Hoggatt, PhD MPH
VA Greater Los Angeles Healthcare System, Sepulveda, CA
Sepulveda, CA
Funding Period: January 2017 - March 2021
VA performance monitoring makes extensive use of diagnosis-based quality measures that track delivery of care only among patients who have qualifying ICD-9 diagnosis codes. However, diagnosis-based measures can have critical validity problems if the targeted condition is under- or over-diagnosed to differing degrees across facilities. Use of diagnosis-based process measures can therefore undermine one of the primary purposes of quality measurement: The comparison of facilities and systems. In addition, diagnosis-based measures cannot be used to detect gaps in access to care for patients who have a targeted condition but no qualifying diagnosis code. Finally, when diagnosis rates vary across patient subgroups, diagnosis-based measures cannot be used to detect and act on healthcare disparities. Problems with diagnosis-based measures could be remedied if true prevalence data were available; however, for many conditions, the electronic health record (EHR) does not contain data on true prevalence.

We propose to build a model for predicting prevalence using multiple sources of existing data and to validate it through a one-time collection of survey-based SUD prevalence data. Focusing on substance use disorder (SUD) care as an example, the objectives of this study are to: (a) assess the degree of SUD under- or over-diagnosis; (b) refine and validate a model for predicting SUD prevalence among VA patients; and (c) assess disparities in SUD diagnosis.

We will collect data on SUD among VA patients using a validated instrument. We will compare observed diagnosis rates to survey-based prevalence estimates and will refine a prototype SUD prediction model and will compute facility performance rankings using diagnosis rates versus predicted prevalence to assess the extent to which variation in performance may reflect variation in diagnosis or coding. Finally, we will assess possible disparities in diagnosing by comparing the gap between diagnosis and estimated prevalence across patient groups.

This project has not yet produced findings.

In this project, we will develop methods to improve diagnosis-based measures to ensure the integrity and effectiveness of VA performance monitoring. These methods can be readily adapted for other conditions where performance monitoring depends on diagnosis-based measures and gold-standard prevalence data are not readily available.

External Links for this Project

NIH Reporter

Grant Number: I01HX002128-01A1

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


Journal Articles

  1. Williams EC, Fletcher OV, Frost MC, Harris AHS, Washington DL, Hoggatt KJ. Comparison of Substance Use Disorder Diagnosis Rates From Electronic Health Record Data With Substance Use Disorder Prevalence Rates Reported in Surveys Across Sociodemographic Groups in the Veterans Health Administration. JAMA Network Open. 2022 Jun 1; 5(6):e2219651. [view]
  2. Hoggatt KJ, Harris AHS, Washington DL, Williams EC. Prevalence of substance use and substance-related disorders among US Veterans Health Administration patients. Drug and Alcohol Dependence. 2021 Aug 1; 225:108791. [view]
  3. Boden MT, Hoggatt KJ. Substance Use Disorders Among Veterans in a Nationally Representative Sample: Prevalence and Associated Functioning and Treatment Utilization. Journal of studies on alcohol and drugs. 2018 Nov 1; 79(6):853-861. [view]
Journal Other

  1. Hill J, Hoggatt KJ. The Tenability of Counterhypotheses: A comment on Bross' discussion of statistical criticism. Observational studies. 2018 Jan 1; 4(2018):34-41. [view]

DRA: Mental, Cognitive and Behavioral Disorders
DRE: Epidemiology, Diagnosis, TRL - Applied/Translational
Keywords: Disparities, Ethnicity/Race, Predictive Modeling, Quality Indicators, Research Measure Development, Substance Use and Abuse
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.