HSR&D Home » Research » IIR 12-383 – HSR&D Study
Linking Clinician Interaction and Coordination to Clinical Performance in VA PACT
Sylvia J. Hysong, PhD MA BA
Michael E. DeBakey VA Medical Center, Houston, TX
Funding Period: April 2014 - September 2017
Care coordination is a fundamental component of Patient-Aligned Care Teams (PACTs) and lies at the heart of their ability to deliver higher quality care than what is possible with traditional clinic models. However, best practices on how to coordinate successfully in health care environments are scarce. Okhuysen & Bechky's (O&B) integrative model of coordination identifies the integrating conditions needed to make collective work possible (predictability, accountability, and common understanding), and the mechanisms and processes that facilitate such conditions (e.g., defining responsibilities for tasks, resource allocation, hand-off work). The O&B framework can thus be used to develop best practices for coordination. Previous research indicates that clinical performance measures found in the research to require more complex interaction amongst clinical personnel (e.g., depression screening) take longer to reach target performance levels than those found to require simpler interactions (e.g., cervical cancer screening) regardless of clinical condition, and that considerable variation exists among VA medical centers (VAMCs) in their performance of measures at a given level of clinician interaction. It is therefore imperative to identify and understand the elements of coordination most accountable for performance variability, and thus most ripe for intervention. To help identify said practices, we used the O&B framework as our criterion standard for measuring and characterizing coordination.
The goal of the proposed research is to test the proposition that elements of coordination (as defined by O&B) will interact with clinician interaction to predict incremental variance in clinical performance. This goal was accomplished via the following objectives:
1) determine the complexity of clinician interaction required for each outpatient clinical performance measure;
2) identify the specific practices employed by VA PACTs indicative of effective coordination as defined by the O&B model; and
3) assess the extent to which the PACTs employing practices indicative of improved coordination exhibit improved clinical performance for outpatient measures of varying levels of clinician interaction.
Five primary care physicians (3 resident, 2 attending) served as subject matter experts (SMEs) to support assessment of clinician interaction. We used functional job analysis (FJA) to assess each outpatient clinical performance measure on clinician interaction. We conducted structured FJA focus groups with the SMEs to identify the tasks required to satisfy the performance criteria for each External Peer Review Program (EPRP) measure. The research team then rated each task using the Worker Interaction scale from FJA (scale range 1-6, the simplest being solitary work, the most complex being systems adaptation), consistent with the methods detailed in Hysong et al., 2016. For each EPRP measure, we calculated a composite rating of worker interaction from the individual tasks.
AIMS 2 and 3:
SURVEY DEVELOPMENT: We developed web-based survey of coordination practices, based on an extensive literature review of coordination in and outside of healthcare. An independent set of 10 participants evaluated the survey for clarity, readability and usability. The survey was deployed in two phases, a survey-development phase and a hypothesis-testing phase using REDCap, a VA-approved web-based survey platform. Items intended to capture integrating conditions were revised between the two phases based on their observed intraclass correlations during survey development phase.
SURVEY DEPLOYMENT: We invited the members of 2100 currently existing PACT teamlets (provider, nurse care manager, LVN/LPN, clerk) at 150 VAMCs nationwide to complete our newly-developed the web-based survey of coordination practices: 1200 during our survey development phase (November 2015-April 2016), 900 during the hypothesis-testing phase (August-December 2016).
OUTCOME DATA COLLECTION: We obtained PACT-level outpatient FY 2016 clinical quality performance measure data from the eQM and Corporate Data Warehouse data sources maintained by Office of Analytics and Business Intelligence (OABI), and obtained or created composites to examine four clinical domains: Behavioral Health, Ischemic Heart Disease (IHD), Diabetes Mellitus (DM), and Prevention. We also obtained PACT Integration metrics and related organizational characteristics from the Patient Aligned Care Teams Compass Cube to use as covariates in our models.
DATA ANALYSIS - Survey Psychometric Work: To evaluate the psychometric soundness of our instrument we conducted intraclass correlations (ICC) and multi-level confirmatory and exploratory factor analyses (MCFA, MEFA). We limited our sample to teams with 3 or more respondents to facilitate ICC calculation. Factors resulting from adequate fit according to either MCFA or MEFA were used for hypothesis tests.
DATA ANALYSIS -- Tests of Hypotheses: Based on our initial psychometric work, to examine the impact of the coordination practices on our four clinical performance domains we conducted multiple regression models using maximum likelihood (ML) estimation, and full-information maximum likelihood (FIML) to handle missing data. PACT Integration, was used as a covariate in the models.
Coordinative complexity ratings across clinical performance measures ranged from 1.80-2.57 (out of a possible maximum of 6), with over half of the measures having complexity scores between 2.12 and 2.57 (mean = 2.33, SD =.21). The most complex of the measures examined was hypertension control (diagnosis of hypertension and BP >140/90, part of the IHD composite), where tasks mostly involve unilateral and bilateral asynchronous communication amongst clinicians. The least complex was breast cancer screening, where most tasks involve solitary work. No task involved in accomplishing any of the measures studied exceeded a coordinative complexity level of 3, where the clinician both receives and returns the communication and/or resources required to complete the task.
Survey Response Rates:
A total of 3405 individual teamlet members out of 8284 invited (41%) responded to our survey, representing primary care PACTs housed at 677 VA Healthcare facilities to include 157 VA Medical Centers (41% of respondents), 20 Health Care Systems, (6% of respondents) 319 Primary Care CBOCs (29% of respondents) and 169 Multi-Specialty CBOCs (24% of respondents).
320 viable PACTs (i.e., those with responses from 3 or more teamlet participants) out of 1200 PACTs invited responded during the survey development phase of the project (27%). An additional 387 teams with two respondents per team (32%) and 243 single-respondent teams (20%) also supplied responses; thus 79% of the PACTs invited were represented in the data during the survey development phase.
For the hypothesis-testing phase we relaxed our criteria to include any team with 2 or more respondents, consistent with current practice in teams research. 427 viable PACTs out of 900 PACTs invited responded during the hypothesis-testing phase of the project (47%). An additional 244 single-team respondents (27%) also supplied responses; thus 74% of the PACTs invited were represented in the data during the hypothesis-testing phase.
Survey Development and Psychometric Work:
We used the 300 viable PACTs available at the time analyses began to evaluate the instrument's psychometrics. Intraclass correlations for each construct measured by the survey were sufficiently high (.09-.13) to warrant a multilevel approach to our factor analysis work. Initial MCFAs showed poor fit to the expected constructs proposed by Okhuysen and Bechky's model of coordination (SRMR-between =.11-.73; SRMR-within = .04-.08; p-values for all Chi-square tests <.0001)). Follow-up MEFAs yielded 2 integrating conditions of coordination (accountability and common understanding), rather than the 3 proposed by O&B; and 7 processes of coordination (defining responsibilities/developing agreement; creating a common perspective; direct information sharing and scaffolding; bringing groups together; updating; monitoring and role substitution; developing trust and anticipating/responding) rather than the 16 originally proposed by O&B. Thus the model proposed by O&B may operate somewhat differently in the VA primary care setting, given the high levels of proceduralization and standardization already present in VA health care delivery.
Tests of Hypotheses:
PACTS reporting higher scores in the practice of creating a common perspective among team members, both through their usual routines and through the objects and representations employed to accomplish this (R-sq =.038, p=.046), exhibited higher clinical performance scores in the Behavioral Health composite (consisting of measures of mental depression, post-traumatic stress disorder, and alcohol misuse screenings). PACTS reporting greater common understanding within their team exhibited higher clinical performance scores in the Ischemic Heart Disease composite (consisting of multiple blood pressure control measures). No other significant relationships were found.
As shown by the Aim 1 findings, little variability existed in the coordinative complexity of the clinical performance measures examined. Due to its lack of variability, this variable provided insufficient predictive utility and thus no further analyses were carried out with this variable. However, it is noteworthy that the clinical composites with significant effects of coordination practices were also the two composites with the highest average coordinative complexity. The newly created standardized task statements and FJA ratings were added to the existing bank of primary care tasks and can now be utilized in future research and organizational application.
VA has invested considerably to transition primary care into PACTs, a paradigmatic shift in the way it delivers primary care. However, the degree of alignment of current practice with PCMH principles it is not clear. Coordination is critical to PACT performance, and requires a more complex level of interaction among clinical staff than the traditional model of care; without a good understanding of best practices in coordination, PACTs will not be successful. Insights from fields outside of health care, such as industrial/organization psychology, are needed to advance our understanding of coordination among disciplines.
Until recently, clinicians received little formal training about effective coordination practices, or feedback on their care coordination abilities. Our study suggests the traditional tools of coordination such as routines, policies, and procedures are necessary but not sufficient to facilitate coordination, and highlights need for teams to create common understanding amongst its members to more effectively provide effective, coordinated care to the veterans they serve. To realize the full potential of the PACT/ PCMH model, interventions and best practices for improving coordination should target the development of shared mental models of the coordinated work the team needs to accomplish together.
External Links for this Project
NIH ReporterGrant Number: I01HX001110-01A1
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Health Systems
MeSH Terms: none