BACKGROUND/RATIONALE:
In order to evaluate the quality and effectiveness of care, health care systems increasingly rely on process measures of treatment provision constructed from administrative data. The VA Office of Quality and Performance (OQP) Continuity of Care measure is the primary metric through which VA SUD specialty care quality is insentivized. The HEDIS performance measures of substance use disorder (SUD) care (Initiation and Engagement) have been widely adopted by many state and private health care agencies to evaluate the quality of their SUD treatment performance. Further, as the VA Mental Health Strategic Plan approved by the Secretary recommends greater use of these measures, greater analysis of them has high VA policy relevance. However, two important aspects of the validity of these measures have not been evaluated: 1) The extent to which meeting these process-based criteria are associated with other quality measures, such as patient clinical outcomes, and 2) The extent to which the criteria as specified accurately index bonafide SUD care.

OBJECTIVE(S):
Aim1: To determine the degree of association between meeting the process of care criteria (i.e., OQP Continuity of Care measure, HEDIS Initiation and Engagement) at the patient level with outcome quality measures (i.e., change in patient symptoms).
Aim 2: To evaluate the extent to which the specifications for process measures of quality accurately index bonafide SUD care.

METHODS:
Method Aims 1: We examined pre-existing samples of VA SUD patients and evaluate the degree of association between the three process-of care quality measures and outcome quality measures (e.g., SUD symtom and psychosocial improvement)

Method Aim 2: Receipt of SUD care as determined by the HEDIS specifications (using CPT, DRG, and ICD9 -CM codes) will be compared to receipt of SUD care as determined by the specialty treatment codes available in the SAS Medical Datasets (e.g., clinic stop, bedsection). The HEDIS coding system for identifying SUD care and the clinic stop/ bedsection approach utilized in the OQP COC measure, will be evaluated by calculating thier concordance with each other and SUD care documented in progress notes.

FINDINGS/RESULTS:
Findings from this project are presented in 9 published papers and 2 others in review, and have been presented in 18 talks and posters. The main findings are these:

1. The presumed links between addiction process quality measures and outcomes are weak to non-existent. This work has energized the search for better quality measures.

2. The statistical models used to validate process quality measures have ignored important multi-level details.

3. The process by which process quality measures are developed (expert panels without process/outcome link validation) is flawed. We have proposed an alternative strategy.

4. Strategies for identifying addiction-related care with administrative data (DSS codes, diagnosis, and procedure codes) have several, previously unknown caveats.

IMPACT:
Anticipated Impacts on VA Patient Care:

This project has impacted the status of current addiction quality measures and has energized the development of other approaches to assess quality, both within VHA and beyond. We have discussed the results with OMHS, SUD QUERI, and the Program Evaluation Resouce Center (PERC). Outside of VA, SAMSHA, RAND, and NIDA have taken notice of this work and have invited us to present it at national quality measurement forums.

---

External Links for this Project

---

PUBLICATIONS:

Journal Articles

1. Dragoo JL, Braun HJ, Bartlinski SE, Harris AH. Acromioclavicular joint injuries in National Collegiate Athletic Association football: data from the 2004-2005 through 2008-2009 National Collegiate Athletic Association Injury Surveillance System. The American Journal of Sports Medicine. 2012 Sep 1; 40(9):2066-71. [view]
2. Thomas CP, Garnick DW, Horgan CM, McCorry F, Gmyrek A, Chalk M, Gastfriend DR, Rinaldo SG, Albright J, Capoccia VA, Harris AH, Harwood HJ, Greenberg P, Mark TL, Un H, Oros M, Stringer M, Thatcher J. Advancing performance measures for use of medications in substance abuse treatment. Journal of substance abuse treatment. 2011 Jan 1; 40(1):35-43. [view]
3. Harris AH, Reeder RN, Ellerbe L, Bowe T. Are VHA administrative location codes valid indicators of specialty substance use disorder treatment? Journal of rehabilitation research and development. 2010 Nov 9; 47(8):699-708. [view]
4. Harris AH, Finney JW, Moos RH. Baseline Abstinence Status May Moderate Intervention Effects. Journal of drug issues. 2010 Jan 1; 40(1):141-154. [view]
5. Harris AH, Bowe T. Characteristics of VA Substance Use Disorder (SUD) Patients Who Initiate and Engage in Treatment. Psychological Services. 2008 Jan 1; 5(1):228-238. [view]
6. Harris AH, Kivlahan DR, Bowe T, Finney JW, Humphreys K. Developing and validating process measures of health care quality: an application to alcohol use disorder treatment. Medical care. 2009 Dec 1; 47(12):1244-50. [view]
7. Harris AH, Humphreys K, Bowe T, Tiet Q, Finney JW. Does meeting the HEDIS substance abuse treatment engagement criterion predict patient outcomes? The journal of behavioral health services & research. 2010 Jan 1; 37(1):25-39. [view]
8. Harris AH, Bowe T, Finney JW, Humphreys K. HEDIS initiation and engagement quality measures of substance use disorder care: impact of setting and health care specialty. Population health management. 2009 Aug 1; 12(4):191-6. [view]
9. Dragoo JL, Braun HJ, Durham JL, Chen MR, Harris AH. Incidence and risk factors for injuries to the anterior cruciate ligament in National Collegiate Athletic Association football: data from the 2004-2005 through 2008-2009 National Collegiate Athletic Association Injury Surveillance System. The American Journal of Sports Medicine. 2012 May 1; 40(5):990-5. [view]
10. Harris AH, Humphreys K, Bowe T, Kivlahan DR, Finney JW. Measuring the quality of substance use disorder treatment: evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. Journal of substance abuse treatment. 2009 Apr 1; 36(3):294-305. [view]
11. Humphreys K, Harris AH, Kivlahan DR. Performance monitoring of substance use disorder interventions in the Veterans Health Administration. The American journal of drug and alcohol abuse. 2009 Jan 1; 35(3):123-7. [view]
12. Braun HJ, Wilcox-Fogel N, Kim HJ, Pouliot MA, Harris AH, Dragoo JL. The effect of local anesthetic and corticosteroid combinations on chondrocyte viability. Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA. 2012 Sep 1; 20(9):1689-95. [view]
13. Harris AH, Reeder RN, Ellerbe LS, Bowe TR. Validation of the treatment identification strategy of the HEDIS addiction quality measures: concordance with medical record review. BMC health services research. 2011 Apr 11; 11:73. [view]
14. Harris AH, Humphreys K, Finney JW. Veterans Affairs facility performance on Washington Circle indicators and casemix-adjusted effectiveness. Journal of substance abuse treatment. 2007 Dec 1; 33(4):333-9. [view]

VA Cyberseminars

1. Finney JW. Are misinterpreted, hospital-level relationships between process performance measures and outcomes undermining evidence-based patient care? [Cyberseminar]. 2013 May 22. [view]

Conference Presentations

1. Harris AH. Aspects of Process Quality Measure Validation. Presented at: RAND Distinguished Speaker Series Monthly Seminar; 2010 Nov 17; Santa Monica, CA. [view]
2. Harris AH. Aspects of Process Quality Measure Validation. Paper presented at: VA HSR&D National Meeting; 2011 Feb 27; National Harbor, MD. [view]
3. Harris AH, Finney JW. Benefits of Partnerships Between HSR&D, QUERI, OQP, and Operations in the Development and Validation of Quality Measures. Poster session presented at: VA “One Team toward One Dream” National Quality, Patient Safety and Systems Redesign Conference; 2010 Jun 22; San Francisco, CA. [view]
4. Harris AH. Can We Strengthen the Links Between Process Quality Measures and Outcomes? Performance Measurement for Substance Use Disorders: A Research Agenda. Paper presented at: Substance Abuse and Mental Health Services Administration / National Institute on Drug Abuse Meeting; 2010 Mar 24; Washington, DC. [view]
5. Harris AHS, Bowe T, Humphreys KN. Characteristics of VA substance use disorder (SUD) patients who initiate and engage in treatment. Poster session presented at: American Psychological Association Annual Convention; 2007 Aug 19; San Francisco, CA. [view]
6. Harris AH, Humphreys KN, Oliva EM, Fernandes-Taylor S, Bowe T, Kivlahan D, Trafton JA. Comparing Alternative Metrics for Monitoring Pharmacotherapy of Alcohol Use Disorders: Effects on Overall Performance and Facility Rank Order. Poster session presented at: AcademyHealth Annual Research Meeting; 2011 Jun 15; Seattle, WA. [view]
7. Harris AHS, Humphreys KH, Bowe T, Kivlahan D, Hyun J, Finney JW. Developing a New Quality Measure of Outpatient Addiction Treatment for the VA. Presented at: AcademyHealth Annual Research Meeting; 2008 Jun 8; Washington, DC. [view]
8. Harris AH, Humphreys KN, Bowe T, Kivlahan D, Hyun J, Finney JW. Developing a New Quality Measure of Outpatient Addiction Treatment for the VA. Paper presented at: Addiction Health Services Research Conference; 2008 Oct 21; Boston, MA. [view]
9. Finney JW, Humphreys KN, Kivlahan DR, Harris AH. Differences between patient-and facility-level performance measure-outcome relationships: Implications for health care quality management and research. Paper presented at: Kaiser Permanente Division of Research Annual Meeting; 2010 Jan 5; Oakland, CA. [view]
10. Finney JW, Humphreys KN, Kivlahan DR, Harris AH. Differences between patient-level and facility-level performance measure-outcome relationships: Implications for quality measurement and management. Poster session presented at: Treatment of Addictive Behaviors Annual International Conference; 2010 Feb 9; Santa Fe, NM. [view]
11. Harris AH, Kivlahan D, Barnett PG, Finney JW. Do Substance Abuse Residential Rehabilitation Treatment Programs with Longer Average Lengths of Stay Have Better Outcomes? Poster session presented at: VA HSR&D Field-Based Mental Health and Substance Use Disorders Meeting; 2010 Apr 28; Little Rock, AR. [view]
12. Harris AH. HEDIS Initiation and Engagement: Who, Where, and Links to Outcomes. Paper presented at: Addiction Health Services Research Conference; 2009 Oct 29; San Francisco, CA. [view]
13. Finney JW, Humphreys KN, Kivlahan DR, Harris AH. Incongruence between patient-level and facility-level performance measure-outcome relationships: Implications for quality management and research. Poster session presented at: Addiction Health Services Research Conference; 2009 Oct 29; San Francisco, CA. [view]
14. Thomas CP, Seidmann MR, Horgan C, Miller K, Harris AH. Measuring Performance in Use of Pharmacotherapy for Substance Use Disorders in Different Health Systems. Paper presented at: AcademyHealth Annual Research Meeting; 2009 Jun 28; Boston, MA. [view]
15. Harris AH, Humphreys K, Bowe T, Kivlahan D, Finney JW. Measuring the Quality of Addiction Treatment: Assessing the VA Continuity of Care Quality Measure. Paper presented at: Addiction Health Services Research Conference; 2007 Oct 16; Athens, GA. [view]
16. Harris A, Humphreys K, Bowe T, Kivlahan D, Finney J. Measuring the Quality of Substance Use Disorder Treatment: Assessing the VA Continuity of Care Performance Measure. Poster session presented at: VA HSR&D National Meeting; 2008 Feb 13; Baltimore, MD. [view]
17. Harris AH, Finney JW. Multi-level modeling (and thinking) in the development and validation of health care quality measures. Paper presented at: Health Policy Statistics Annual International Conference; 2010 Jan 1; Washington, DC. [view]
18. Harris AH, Finney JW. Strategies to Improve Behavioral Healthcare (A Working List). Paper presented at: Quality of Behavioral Healthcare: A Drive for Change Through Research Conference; 2010 Apr 13; Clearwater Beach, FL. [view]
19. Harris AH. The Quality of Quality Measures: What Happens When Enthusiasm Precedes Science. Presented at: National Institute on Drug Abuse Division of Epidemiology, Services, and Prevention Research Meeting; 2010 Mar 1; Washington, DC. [view]
20. Harris AH. The VA's Experience with a Retention Measure. Paper presented at: Washington Circle Public Sector Biannual Workgroup Meeting; 2009 May 4; Alexandria, VA. [view]
21. Harris AH, Reeder RN, Ellerbe L, Bowe T. Validity of the HEDIS system of Identifying Substance Use Disorder Treatment in the Medical Record? Poster session presented at: Quality of Behavioral Healthcare: A Drive for Change Through Research Conference; 2010 Apr 13; Clearwater Beach, FL. [view]
22. Harris AHS, Humphreys KN, Finney JW. Veterans Affairs facility performance on Washington circle indicators and casemix-adjusted effectiveness. Poster session presented at: American Psychological Association Annual Convention; 2007 Aug 19; San Francisco, CA. [view]
23. Harris AH, Finney JW. When Process Outcome Associations Differ at the Patient and Facility-levels: Implications for Quality Measurement and Management. Paper presented at: Quality of Behavioral Healthcare: A Drive for Change Through Research Conference; 2010 Apr 13; Clearwater Beach, FL. [view]

DRA: Substance Use Disorders
DRE: none
Keywords: none
MeSH Terms: none