HSR&D Home » Research » RRP 07-280 – QUERI Project
Psychometric Validation of an Organizational Readiness-to-Change Scale
Christian D. Helfrich, PhD MPH BA
VA Puget Sound Health Care System Seattle Division, Seattle, WA
Funding Period: October 2007 - June 2010
The organizational readiness to change assessment (ORCA) is a survey instrument designed to assess organizational readiness to change in preparation for testing interventions designed to implement evidence-based changes in clinical practice. The scales are intended for diagnostic use, to identify needs or conditions that can be targeted by implementation activities or resources. They were field-tested in 3 quality improvement projects, but had not been previously validated.
Conduct three sets of psychometric validation analyses: 1) scale reliability tests; 2) factor analysis; and 3) criterion validation.
We used secondary data from three quality improvement projects to conduct cross-sectional analyses. For Aims 1 and 2, we conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects conducted in the Veterans Health Administration (n = 80).
For Aim 3, we used secondary data from one of the three quality improvement projects (n=30) to test the predictive validity of the ORCA scales where the outcomes were 2 items from a survey of Chiefs' of Cardiology and Medicine: (1) their skepticism about the mandate for the quality improvement initiative, and (2) the resources available to implement it.
We find general support for the reliability and factor structure of the ORCA. Reliability for the evidence scale was poor and two subscales were not supported by the factor analysis. However, overall reliability and factor analysis findings support the reliability and structure of the ORCA. Criterion validation findings were mixed, though we view them as positive and suggest that the ORCA subscales were associated with implementation measures.
The next critical step is to use outcomes from implementation activities for criterion validation. This should provide more information about which items and scales are the most useful candidates for a revised readiness to change instrument.
Additional validation of the ORCA is currently underway, and the research team is available to collaborate with quality improvement staff interested in using the survey to obtain a baseline assessment of organizational readiness to change. At this time, we have limited evidence to recommend this survey as an assessment tool for quality improvement projects.
External Links for this Project
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Health Systems
Keywords: Organizational issues
MeSH Terms: none