BACKGROUND/RATIONALE:
The organizational readiness to change assessment (ORCA) is a survey instrument designed to assess organizational readiness to change in preparation for testing interventions designed to implement evidence-based changes in clinical practice. The scales are intended for diagnostic use, to identify needs or conditions that can be targeted by implementation activities or resources. They were field-tested in 3 quality improvement projects, but had not been previously validated.

OBJECTIVE(S):
Conduct three sets of psychometric validation analyses: 1) scale reliability tests; 2) factor analysis; and 3) criterion validation.

METHODS:
We used secondary data from three quality improvement projects to conduct cross-sectional analyses. For Aims 1 and 2, we conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects conducted in the Veterans Health Administration (n = 80).
For Aim 3, we used secondary data from one of the three quality improvement projects (n=30) to test the predictive validity of the ORCA scales where the outcomes were 2 items from a survey of Chiefs' of Cardiology and Medicine: (1) their skepticism about the mandate for the quality improvement initiative, and (2) the resources available to implement it.

FINDINGS/RESULTS:
We find general support for the reliability and factor structure of the ORCA. Reliability for the evidence scale was poor and two subscales were not supported by the factor analysis. However, overall reliability and factor analysis findings support the reliability and structure of the ORCA. Criterion validation findings were mixed, though we view them as positive and suggest that the ORCA subscales were associated with implementation measures.
The next critical step is to use outcomes from implementation activities for criterion validation. This should provide more information about which items and scales are the most useful candidates for a revised readiness to change instrument.

IMPACT:
Additional validation of the ORCA is currently underway, and the research team is available to collaborate with quality improvement staff interested in using the survey to obtain a baseline assessment of organizational readiness to change. At this time, we have limited evidence to recommend this survey as an assessment tool for quality improvement projects.

---

External Links for this Project

---

PUBLICATIONS:

Journal Articles

1. Holt DT, Helfrich CD, Hall CG, Weiner BJ. Are you ready? How health professionals can comprehensively conceptualize readiness for change. Journal of general internal medicine. 2010 Jan 1; 25 Suppl 1:50-5. [view]
2. Helfrich CD, Li YF, Sharp ND, Sales AE. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework. Implementation Science. 2009 Jul 14; 4:38. [view]

Conference Presentations

1. Helfrich CD, Li Y, Sharp ND, Sales AE. Organizational Readiness to Change Assessment (ORCA). Presented at: Knowledge Utilization Annual Colloquium; 2009 Jun 15; Deganwy, Wales. [view]
2. Helfrich CD. Organizational readiness to change assessment: Evaluating Context Using Different Lenses Conference Panel. Presented at: Knowledge Utilization Annual Colloquium; 2010 Jun 10; Halifax, Canada. [view]
3. Helfrich CD, Li YF. Psychometric Validation of Readiness-to-Change Scales Based on the Promoting Action on Research Implementation in Health Systems (PARiHS) Model. Poster session presented at: VA HSR&D National Meeting; 2008 Feb 13; Baltimore, MD. [view]
4. Helfrich CD. Validation of Readiness-to-Change Scales Based on the Promoting Action on Research Implementation in Health Systems (PARIHS) Model. Poster session presented at: University of Washington Department of Health Services Annual Research Meeting; 2011 Jun 15; Seattle, WA. [view]

DRA: Health Systems
DRE: none
Keywords: Organizational issues
MeSH Terms: none