HSR&D Home » Research » IIR 11-326 – HSR&D Study
Evaluating Collaborative Care Using a Stepped Design and National Electronic Data
Melissa M Farmer, PhD MS
VA Greater Los Angeles Healthcare System, Sepulveda, CA
Funding Period: April 2013 - March 2015
Depression is a major cause of morbidity and mortality in primary care patients, and depression affects approximately 12% of Veterans attending VA primary care practices. To address this heavy burden, a series of depression quality improvement efforts (Translating Initiatives in Depression into Effective Solution-TIDES and ReTIDES) based on collaborative care management (CCM) were implemented at the VA beginning in 2002. In 2007, the national primary care/mental health integration (PC-MHI) mandate (VA Uniform Mental Health Services Handbook) introduced both co-located mental health specialists and CCM into larger VA primary care sites. This mandate required implementation and spread of CCM to translate from a research innovation to the roll-out of a national program making evaluation strategies based on randomization not feasible. Also, electronic performance measures for assessing the level at which the population of Veterans with depression received appropriate care had not proven robust enough to support rigorous evaluation of VA's depression management. Therefore, there was a critical need for longitudinal electronic depression care quality measures capable of evaluating the implementation, spread and sustainability of CCM.
We developed longitudinal electronic population-based measures of depression quality of care and validated the measures conceptually and operationally using panel judgment by VA and non-VA experts; and demonstrated the feasibility of using a modified stepped wedge design approach with the new measures for evaluating TIDES/ReTIDES.
We used the VA National Patient Care Database and Pharmacy Benefits Management data for primary care patients from 1999-2011 from nine Veteran Integrated Service Networks (VISNs). We developed four population-based depression care quality metrics that incorporated a six-month look back and one-year follow-up: detection of a new episode of depression, 84 and 180 days follow-up, and minimally appropriate treatment one-year post-detection. The development of longitudinal population-based quality measures required systematic referencing of final measures to the full primary care population, attention to timing of the measures in relationship to a patient's index visit, establishing continuity of care, and determining evidence-based cut-points for health care visits, prescription medications and refills.
We developed a modified stepped wedge design based on a general linear model and published stepped wedge design methodology to improve the precision of comparisons between intervention and control periods when randomized designs are not feasible. Our model is multi-level, requiring inclusion of three levels of variables: time, site-level, and patient-level covariates. We fit the model for all primary care patients with a new episode of depression seen at the 34 sites within the nine VISNs that implemented TIDES or ReTIDES from 2000-2010. Implementation data on TIDES/ReTIDES start date, facility type (medical center vs. community-based clinic) and facility size, as well as patient-level covariates (age, gender, race/ethnicity, marital status, and PTSD) were used to test the utility and validity of a modified stepped wedge design for evaluating depression follow-up and treatment. We used a modified Delphi expert panel process with VA and non-VA experts in the area of depression care quality, PC-MHI, program evaluation, and performance measurement to assess overall acceptability of the measures and the modified stepped wedge design, future implications for VA's CCM program implementation, and optimal statistical features for broader use.
There was a substantial increase in the number of patients seen in primary care - 1.19 million in 2000 to over 2.26 million in 2010. Despite rapid growth, the detection of new episodes of depression (7-8%) and minimally appropriate treatment rates (82-84%) remained stable over the decade indicating that VA was able to maintain a standard of care while treating significantly more patients each year. Follow-up at 84 and 180 days were 37% and 45% respectively in 2000 and increased to 56% and 63% by 2010 indicating improvements in timely treatment initiation and follow-up.
In the modified stepped wedge model, 53% of the sites were medical centers, and 21% were the original TIDES sites. The primary care patient population was 9% women, 30% white and 10% black patients,18% less than 44 years, 51% ages 45-64, and 31% 65 years or older. We found improvements in depression care quality over time. However, after correcting for covariates, the interactions of TIDES and ReTIDES with time (year) were not significant, indicating that the effect for sites engaging in the programs over the ten years compared to the effect seen in control periods did not significantly vary by calendar year. Women had increased odds of treatment compared to men (OR=1.10; p<.05), whereas blacks had increased odds of follow-up (84 days OR=1.11 and 180 days OR=1.14; p<.05) compared to whites. Younger patients had higher odds of follow-up and treatment than older age groups, and patients with PTSD had increased odds of follow-up (84 days OR=2.79, 180 days OR=4.31; p<.01) and treatment (OR=3.91; p<.01) than patients without PTSD.
We convened an expert panel in March 2015. Prior to the meeting, our 14 panelists received a measures development monograph and an on-line survey to evaluate our methods. Pre-meeting survey results showed a high level of agreement on most cohort construction and measures development methods, including using antidepressants prescribed in primary care as a signal for depression detection. Experts reached consensus on our definition of "minimally appropriate treatment" as the lowest level of activity that would indicate treatment for depression in a 12-month period. For the modified stepped wedge, panelists agreed that additional measures with stricter treatment requirements would be useful, especially considering the mental health related complexity of the Veteran population. Panelists noted the importance of measuring non-face-to-face encounters for evaluating CCM; however, because reliable coding of telephone, tele-health, and secure messaging has only recently become available, inclusion of these was not feasible in these historical analyses.
These electronic depression care quality measures meet a widely-recognized need for longitudinal metrics that span the course of depression clinical care from screening through treatment. By using these measures to demonstrate the feasibility of using a modified stepped wedge model to evaluate collaborative care roll-out and sustainability over a ten-year period, the results inform use of comparative evaluation designs for nonrandomized spread interventions and on-going national implementation of CCM as well as advance the science of depression quality measurement.
External Links for this Project
NIH ReporterGrant Number: I01HX000821-01A1
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Mental, Cognitive and Behavioral Disorders, Health Systems
DRE: Treatment - Comparative Effectiveness
MeSH Terms: none