» Back to Table of Contents

Response to Commentary

Three HRO Evaluation Priorities

Drs. Stone and Lieberman outlined the journey we are on within VHA to become a High Reliability Organization (HRO). This high priority initiative takes on even more urgency in the face of unexpected, enormous disruptions like the COVID-19 pandemic. Many facilities, including the lead 18 sites and others, have already been on the journey to HRO maturity. This journey is a marathon, not a sprint; this depth of cultural transformation will take time and long-term commitment by everyone throughout the system.

Learning is at the heart of an HRO in its laser focus on Zero Harm to patients. The HRO rests on three key pillars as shown in Figure 1. The journey starts with leaders at all levels who visibly demonstrate commitment to HRO through their actions. This commitment leads to a robust safety culture that embraces Just Culture, creating psychological safety to encourage error reporting and learning from mistakes.1 Teams are strengthened as they engage in Continuous Process Improvement to optimize patient care processes. Mature HROs promote teamwork and increase a sense of purpose and meaning in work for all employees.

We have been fortunate in building partnerships with key leaders throughout the VHA to do the groundwork necessary to evaluate progress toward HRO in a way that helps the system learn. We convened a panel to develop a research agenda to address HRO evaluation priorities at the HSR&D Annual Meeting in October 2019 in collaboration with Dr. Gerard Cox (Deputy Under Secretary for Health for Organizational Excellence), Dr. William Gunnar (Executive Director, National Center for Patient Safety), Dr. B. Vince Watts (Director, Systems Redesign and Improvement Program Office), and Christopher Mannozzi (Supervisory Analyst, RAPID-Performance Measurement Office). Participants and partners identified three key priorities: 1) measurement and prediction; 2) effective approaches to implement HRO; and 3) pragmatic yet robust study designs. HSR&D and QUERI have quickly contributed funds to support foundational work to help advance knowledge across these domains.

Measurement and Prediction

Pragmatic, simple measures are needed to monitor progress, understand relationships between key organizational attributes and HRO maturity, and to develop and affirm a causal pathway from HRO implementation to safety outcomes. Leaders throughout VHA need metrics to help guide their efforts toward HRO maturity; metrics must balance the need to demonstrate clinical impact while, at the same time, ensuring that robust safety culture is a key mediator to positive impacts. Our team is working to identify indicators of HRO based on longitudinal changes in items within the All-Employee Survey (AES) as the first 18 sites continue their HRO journey. There is also a need to validate the safety culture measures added to the AES in 2019 and assess their relationship with event reporting. A guiding hypothesis is that error reporting will increase as organizations mature and that the ratio of potential errors versus safety events will increase; this indicates transparency, a key characteristic of the safety culture.

Effective Approaches to Implement HRO

Effective implementation strategies are needed to help organizations mature as HROs. As mentioned above, HSR&D funded a Rapid Review Evidence Brief that identified major knowledge gaps in how to effectively implement HRO. Our team is partnering with Systems Redesign and Improvement (SR&I) in an evaluation of newly updated Lean belt training curriculum and QUERI’s Learn. Engage. Act. Process. (LEAP) QI training program. We are using evolutionary learning methods to compare and learn from delivering these two training approaches. Our goal is to engage frontline clinical teams in continuous process improvement, a key HRO pillar. Much more is needed to design effective strategies to address all three HRO pillars. Other specific topics identified by the HSR&D partnered panel members included learning from other organizations outside VHA, designing incentives that encourage team-based development and training, and identifying strategies to leverage necessary process changes to align with HRO in response to the Cerner Electronic Health Record Migration.

Pragmatic Yet Robust Study Designs

HRO as an intervention is highly complex in several ways: it is a complex intervention with multiple highly abstract components with unclear definitional boundaries (e.g., safety culture embraces concepts of psychological safety, just culture, and more); it relies on complex pathways comprising multiple feedback loops, mediators, and moderators; it impacts both employee experience and Veteran safety outcomes; the contexts within which HRO is being implemented are complex environments that may dynamically change over time (e.g., impact of the COVID-19 pandemic), which in turn demands complex, adaptive, multi-component implementation strategies. In the face of these realities, panel members recognized the need and the challenge of identifying pragmatic, feasible, and yet, robust evaluation designs. For example, our team is using qualitative, quantitative, and mixed methods approaches in our formative evaluation work. The way forward will likely include robust trial designs to enable comparisons of HRO versus non-HRO organizations.

Reference

1. Reason, J. “Achieving a Safe Culture: Theory and Practice,” Work & Stress 1998; 12(3):293-306.