A comparison of scalable routine clinical materials and observer ratings to assess CBT fidelity.

Calloway A, Creed TA, Gumport NB, Gutner C, Marques L, Hernandez S, Song J, Johnson C, Youn SJ, Elhusseini S, Deguzman-Lucero RM, Laskot T, La Bash H, Silvan YA, Cassotte C, Park AL, Dean K, Bartuska AD, Jo B, Barnett P, Kuhn E, DeRubeis R, Vogt D, Stirman SW. A comparison of scalable routine clinical materials and observer ratings to assess CBT fidelity. Behaviour Research and Therapy. 2025 Jan 1; 184:104655.

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

Decades of research have demonstrated the efficacy of cognitive behavioral therapies (CBTs) for a wide variety of psychiatric diagnoses, resulting in the inclusion of CBT as a first-line evidence-based practice (EBP) in treatment guidelines for mood and anxiety disorders. However, some research suggests that many providers do not implement EBPs as intended. Ongoing quality monitoring is needed to support EBP implementation and sustainability, but "gold standard" fidelity monitoring (e.g. observer ratings) is time-consuming, requires extensive training, and may feel intrusive to providers and clients. In the current study, we aimed to develop a scalable method of assessing CBT fidelity that leverages information generated in routine clinical care (e.g. session worksheets and clinician checklists). Ratings of adherence based on worksheets were not correlated with ratings of adherence based on observer ratings. However, ratings of competence based on worksheets were significantly correlated with observer ratings of competence. Ratings of adherence based on clinician checklist ratings were also significantly correlated with observer-rated adherence. Results did not indicate a strong relationship between adherence or competence measured by worksheet ratings or observer and symptom change. However, adherence as measured by clinician checklists were associated with subsequent depression symptom change. Findings have a strong potential to impact fidelity monitoring strategies for a variety of CBTs. Given the limited resources and time to do full audio review in routine care settings, findings suggest that using routine materials generated in session to assess therapist competence may be a feasible alternative to the "gold standard" audio review. The trial is registered at ClinicalTrials.gov, number NCT03479398.