Formative Evaluation of PACT Implementation in VISN-23: A Mixed Methods Approach

Solimeo S, Ono SS, Adams SL, Paez MB, Stewart GL. Formative Evaluation of PACT Implementation in VISN-23: A Mixed Methods Approach. Poster session presented at: VA HSR&D National Meeting; 2011 Feb 17; National Harbor, MD.

Objectives: The VISN 23 Patient Aligned Care Team (PACT) Demonstration Lab is conducting a formative evaluation of the PACT implementation process across 21 sites. Three levels of analysis: site, team, and role, are being conducted to develop a broad-based model for understanding the connections between role clarity, team function, and patient outcomes. Methods: The reflexive nature of implementation requires that formative evaluation be proactive in obtaining data that are timely, site and role specific, and yet comparable and measurable across sites. A set of interrelated methods are being employed to gather both experience-near and site-level data. These methods include: role-based group discussions; early and late implementation role-specific interviews; site visits; team-based open-ended discussions; a work role survey; observation of learning sessions, and telephone conferences; and a novel approach to traditional "diary" methods. These integrated data streams are analyzed using content analysis, traditional qualitative inductive theory building, and statistical methods. Results: Early implementation work role measurements of 147 team members indicate significant role-based differences in satisfaction, role overload, and initiated and received task interdependence. On a 5-point scale, providers report lower job satisfaction than other team members (means: provider, 3.8; RN, 4.1; LPN, 4.2; clerk, 4.0). Job satisfaction derives in part from role overload (means: provider, 3.8; RN, 3.3; LPN, 2.9; clerk, 3.5), which in turn draws upon initiated task interdependence (means: providers, 3.81; RN, 3.1; LPN 3.4; clerk, 3.6) and received task interdependence (means: provider, 4.2; RN, 3.5; LPN, 3.5; clerk, 4.0). Preliminary analyses of role-based discussion groups support and contextualize the survey findings, provide insight into cross-site variation in implementation and team-member perspectives on barriers to implementation. Implications: Early implementation measures demonstrate the value of mixed methods analyses as well as provide an understanding of barriers to PACT implementation. The VISN 23 PACT Demonstration Lab's mixed methods approach connects quantitative and qualitative data and identifies potential barriers and facilitators to effective team development. Impacts: PACT is currently being implemented in select VHA sites. Timely, targeted analyses of the role of team function in predicting patient outcomes support the anticipated broader rollout of PACT throughout the VHA and ultimately serve to promote accessible, efficient, and patient-centered care for our veterans.