The VA is underway with a transformative effort to promote evidence-based, recovery-oriented mental health services. One such program is Illness Management and Recovery (IMR), an evidence-based practice (EBP) aimed at promoting recovery in people with severe mental illness by teaching illness self-management strategies. To date, there has been no effort to assess the implementation of IMR within the VA. Previous research has indicated the difficulty in implementing evidence-based practices in general and IMR in particular.
The overall objective of this research was to conduct a formative evaluation of the implementation of IMR within VA mental health care services.
Aim 1: Assess penetration of IMR within VA mental health services.
Aim 2: Identify facilitators and barriers to implementing IMR within mental health services.
Aim 3: Assess penetration, facilitators, and barriers specific to PRRC programs.
Aim 4. To examine the content validity of the IMR treatment integrity scale (IT-IS) via an expert survey.
Aim 5. To assess the acceptability of IT-IS ratings and feedback to VA IMR clinicians.
The proposed formative evaluation included four phases. Phase I utilized an on-line survey to identify current levels of self-reported penetration and implementation of IMR as well as barriers and facilitators. Participants included local recovery coordinators (LRCs) and any additional staff the LRC identified as a local IMR coordinator. Participants completed an on-line survey regarding IMR penetration, components of the IMR program utilized, and facilitators/barriers to implementing IMR. Phase II consisted of qualitative analysis of semi-structured telephone interviews with a stratified subsample of respondents from Phase I. Participants included LRCs, PRRC coordinators, and local IMR experts. Interviews garnered more in-depth information on current and planned IMR implementation as well as needs and preferences for future tools to support IMR implementation. Phase III included an on-line survey of IMR experts (published, senior clinicians, and peer-providers) regarding the importance of putative elements of IMR. Phase IV elicited feedback on the usefulness, acceptability, and perceived accuracy of the IT-IS from current IMR clinicians via semi-structured interviews
Aim 1: Regarding penetration of IMR, 47% of the 97 VAMCs responding to our online survey reported having IMR anywhere. PRRCs had the highest penetration, with 69% of PRRCs reporting IMR implementation, while only 20% of MHICM teams and <10% of other clinics reporting IMR.
Aim 2: Preliminary findings indicate barriers and facilitators at multiple levels, including outside the clinic implementing IMR, within the clinic, regarding IMR itself, and specific to the individuals providing IMR. We have developed a codebook of codes relevant to IMR implementation and are producing site summaries for all sites participating in our Phase II interviews. Notably, less than 20% of Phase I providing IMR reported receiving IMR training.
Aim 3: Most, but not all PRRCs implemented IMR. Barriers to implementation included lack of IMR training availability, lack of awareness of IMR, and lack of fit between PRRC group schedules (generally on 3 or 6 month semesters) and IMR length (over 9 months).
Aim 4: The expert survey found three items in need of clarity of wording. We examined importance of each element in three ways: criticality, definitiveness, and impact on outcomes. All items were rated higher than a distractor item which was intentionally irrelevant to the program model. Definitiveness produced the most variability amongst items, with three items showing the highest level of being "defining" of IMR: structured curriculum, goal setting, and recovery orientation.
Aim 5: For the IT-IS acceptability interview, we made attempts to recruit participants from 47 VA sites. Nearly half (47%) of those potential recruits did not respond to multiple email and voice mail requests for participation. Regarding the personnel we were able to contact, two frequently cited reasons for non-participation were: an IMR group was not being offered within the study time frame (23%); privacy issues concerning staff and/or veterans (13%). We were only able to recruit one VA participant; we recruited an additional two community clinicians. Participants generally agreed with the acceptability and accuracy of the IT-IS as a whole and each item individually. The more poignant message was the feasibility of obtaining audio-recordings across VA sites. In future research, more time will be required to liaise with each facility's mental health leadership and information security personnel.
The transformative effort taken by the VA aims to create a system where Veterans have the right to direct their own treatment and are encouraged to develop recovery-oriented action plans for themselves, all of which are core tenets of IMR. IMR helps Veterans develop and pursue personal goals, which is consistent with the PRRCs' intention to encourage Veterans to define and pursue a self-determined personal mission and vision for their lives. Careful examination of the implementation of IMR will likely replicate the VA's success in previous EBP roll-outs, thereby informing future implementation both within the VA and in community services.
The current work highlights the remaining implementation gap existing within the VA. Most Veterans with IMR still do not have access to IMR within their particular VAMC and/or clinic. Moreover, too few supports exist (most especially training) for the dissemination of IMR or the assurance of high quality IMR services. Our work will greater clarify the barriers to wider IMR implementation as well as assess acceptability and anticipated utility of potential implementation tools. Regarding one particularly tool, the IT-IS, stakeholders agree on its usefulness and the importance of the elements assessed by it; however, substantial barriers currently prevent its widespread use. Future work will focus on modifying the IT-IS protocol to maximize acceptability and developing ways to overcome systemic barriers in its application.
- McGuire AB, Luther L, White D, White LM, McGrew J, Salyers MP. The "Critical" Elements of Illness Management and Recovery: Comparing Methodological Approaches. Administration and policy in mental health. 2016 Jan 1; 43(1):1-10.
- McGuire AB, Salyers MP, White DA, Gilbride DJ, White LM, Kean J, Kukla M. Factors affecting implementation of an evidence-based practice in the Veterans Health Administration: Illness management and recovery. Psychiatric Rehabilitation Journal. 2015 Dec 1; 38(4):300-5.
- McGuire AB, White DA, White LM, Salyers MP. Implementation of illness management and recovery in the Veterans Administration: an on-line survey. Psychiatric Rehabilitation Journal. 2013 Dec 1; 36(4):264-71.
- McGuire AB, Stull LG, Mueser K, Santos M, Mook A, Nicksic C, Rose N, White L, Salyers MP. Illness management and recovery treatment integrity scale. Paper presented at: National Institutes of Health Conference on the Science of Dissemination and Implementation: Policy and Practice Conference; 2011 Mar 21; Bethesda, MD.