VA performance monitoring makes extensive use of diagnosis-based quality measures that track delivery of care only among patients who have qualifying ICD-9 diagnosis codes. However, diagnosis-based measures can have critical validity problems if the targeted condition is under- or over-diagnosed to differing degrees across facilities. Use of diagnosis-based process measures can therefore undermine one of the primary purposes of quality measurement: The comparison of facilities and systems. In addition, diagnosis-based measures cannot be used to detect gaps in access to care for patients who have a targeted condition but no qualifying diagnosis code. Finally, when diagnosis rates vary across patient subgroups, diagnosis-based measures cannot be used to detect and act on healthcare disparities. Problems with diagnosis-based measures could be remedied if true prevalence data were available; however, for many conditions, the electronic health record (EHR) does not contain data on true prevalence.
We propose to build a model for predicting prevalence using multiple sources of existing data and to validate it through a one-time collection of survey-based SUD prevalence data. Focusing on substance use disorder (SUD) care as an example, the objectives of this study are to: (a) assess the degree of SUD under- or over-diagnosis; (b) refine and validate a model for predicting SUD prevalence among VA patients; and (c) assess disparities in SUD diagnosis.
We will collect data on SUD among VA patients using a validated instrument. We will compare observed diagnosis rates to survey-based prevalence estimates and will refine a prototype SUD prediction model and will compute facility performance rankings using diagnosis rates versus predicted prevalence to assess the extent to which variation in performance may reflect variation in diagnosis or coding. Finally, we will assess possible disparities in diagnosing by comparing the gap between diagnosis and estimated prevalence across patient groups.
This project has not yet produced findings.
In this project, we will develop methods to improve diagnosis-based measures to ensure the integrity and effectiveness of VA performance monitoring. These methods can be readily adapted for other conditions where performance monitoring depends on diagnosis-based measures and gold-standard prevalence data are not readily available.
- Hill J, Hoggatt KJ. The Tenability of Counterhypotheses: A comment on Bross' discussion of statistical criticism. Observational studies. 2018 Jan 1; 4(2018):34-41.