RRP 09-171 – HSR&D Study
Career Development Projects
Formative Evaluation of the Self-Help Toolkit
John D. McKellar PhD
VA Palo Alto Health Care System, Palo Alto, CA
Palo Alto, CA
October 2009 -
Self-help groups are an important component of the system of care for patients with substance use disorders (SUDs) in the VA. Self-help groups can augment continuing care services in the VA, reduce rates of relapse, and reduce cost of care. Twelve-step facilitation is among the evidence-based psychosocial interventions for SUD identified within the Handbook on Uniform Mental Health Services. To improve referral of patients to and retention in self-help, the SUD QUERI Retention in Continuing Care Workgroup created a website for clinicians called the "Self-Help Toolkit and 3-Step Referral Method."
The objectives of this study included (1) receiving feedback from treatment managers on the feasibility of implementing the 3-step referral algorithm (2) integrating the feedback into our website, (3) pilot testing the new version of the website with 6 programs, (4) and then receiving further feedback from treatment managers and frontline staff (anonymously) from the 6 SUD outpatient programs.
For phase 1 27 VA SUD treatment programs. We selected only one program per VA facility. The goal for this phase was to receive feedback on the feasibility of our self help referral process outlined in the website and to identify potential barriers/solutions. Qualitative analyses were used to code suggested barriers and solutions which were integrated into the website. For Phase 2 of the project, we selected 6 programs from among those that participated in Phase I of the project. Three of the sites were recruited did not feel that implementation would difficult and 3 were recruited because they believed that implementation would be difficult. During Phase 2 we piloted the website with the 6 programs for 3 months (4 sites actually completed). During months 1 and 2 we contacted program managers and inquired about success or problems associated with website use, suggesting solutions as needed. Near the end of month 3, we distributed an anonymous questionnaire as requesting feedback on aspects such as ease of using the website, usefulness, and likelihood they will use it in the future with two (optional) open-ended questions about ways to improve the website at end of the questionnaire.
We have completed stage one of our project, by obtaining feedback from 27 SUD treatment providers to improve our website and integrating suggestions into our referral algorithm. The clearest message we received in Phase 1 was that we needed to create a script/manual for the referral algorithm. Providers preferred explicit step-by step directions for the referral method to general descriptions of our referral method with greater flexibility for implementation (e.g., "just tell us what to do"). For Phase 2 we recruited 6 programs but 4 programs participated in monthly calls with the PI to identify problems and solutions.. The calls with Phase 2 program managers did not identify any problems or issues that might be addressed by changes to the website or the 3-Step Referral Method. Brief questionnaire feedback from frontline providers was positive with regard to usability of the website but more equivocal about adherence to the 3-step Referral Method.
The current study suggests that the Self Help Toolkit is ready for evaluation in a formal implementation study. Our pre-implementation study suggests that the future study should focus on testing implementation strategies aimed at increasing use of the 3-Step Referral Method.
None at this time.
Substance Abuse and Addiction
Treatment - Observational
Addiction, Self-care, Telemedicine