Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

IIR 11-356 – HSR Study

IIR 11-356
Examining the Relationship of Culture Change, Adverse Events and Costs in CLCs
Jennifer L. Sullivan, PhD MS BA
VA Boston Healthcare System Jamaica Plain Campus, Jamaica Plain, MA
Boston, MA
Funding Period: March 2013 - February 2016
Person-Centered Care (PCC) implementation underway in VA, combined with the use in VA of both the Artifacts of Culture Change Tool and MDS-based Quality Indicators (QIs), represents a unique research opportunity to assess the impact of implementing PCC on quality in a large, integrated delivery system with a sizable number of sites. Using pilot cross-sectional data, we had found that there is a statistically significant relationship between the extent of PCC and a MDS-based composite measure of quality; and, there is preliminary evidence that higher levels of quality are associated with lower costs.

The purpose of this longitudinal mixed-methods study is to understand the effects of PCC on quality and quality on costs and develop a better understanding about variations in PCC/quality and characteristics distinguishing Community Living Centers (CLCs). Our specific objectives were to: 1) Identify the extent to which PCC implementation leads to lower adverse event rates; 2) Identify the extent to which quality leads to costs; and 3) Understand what distinguishes the high from the low performing CLCs by conducting site visits at selected facilities.

In Objective 1, we evaluated the relationship between the MDS-based quality composite score and Artifact Tool PCC scores using a Bayesian model; and assessed the relationship between the individual MDS-based quality indicators and Artifact Tool PCC scores from FY2008-2012.
In Objective 2, we examined if higher quality is associated with lower patient-level costs adjusting for case mix through both individual and time-series regressions from FY2008-2012.
In Objective 3, we conducted 12 site visits in order to understand variations in key health systems factors (structural characteristics, PCC/quality processes, and organizational infrastructure) for CLCs in high performance versus low and mixed performance categories. Sites were selected according to their performance status (PCC implementation and Quality). All study participants (including site visitors) were blinded to hospitals' quality and PCC performance. Verbatim interview transcripts were the primary data source for analysis. A team of five analysts coded the transcripts for evidence of 21 a priori domains within the 5 conceptual areas established in the conceptual framework. We used an inductive approach to identify 9 important themes relevant to the study's goals that were not anticipated prior to data collection.

Objective 1. Over the study period, the mean MDS Composite quality measure score remained stable during the study period at 0.14. The Artifacts of Culture Change Tool scores increased from a mean score of 217 in Fy08 to 255.4 in FY12. We found that 28% of CLCs had a statistically significant change in their quality score over time. In addition, 52% of CLCs had a statistically significant change in the Artifacts Tool score over time. We found the overall relationship between PCC and quality was not statistically significant (p=0.23). Nevertheless, we did find a statistically significant relationship between quality and PCC for about 15% of CLCs (p<.05).
Objective 2. We used Type 3 score statistics for probabilities, adjusted RUGs score (a variable controlling for case mix) was highly significant (p<0.0001), while quality (p=0.0459) and PCC (p=0.0671) were both marginally significant, all being positively related to total patient costs.
Objective 3. We conducted 120 interviews with 141 staff members at 12 VA CLCs. Staff members included senior leaders, middle managers, and front-line staff.Of the thirty health system factors identified, high and low performers were similar in 16 domains and varied in 9 domains. There was insufficient data to make comparisons in 5 domains. Across all 12 CLCs, regardless of performance status, we found similarities in the ways PCC was defined and implemented and the types of quality improvements that were being conducted. All sites had senior leaders that prioritized PCC and quality. Sites reported similar barriers to implementing PCC and utilized both PCC and quality data to address issues. High performers had more positive perceptions about organizational culture, alignment/coordination, and senior leadership support from the middle and front-line staff perspective. In addition, high performers reported more PCC policies in place, conducted more PCC and quality trainings, and provided more recognition of good PCC practices. High performers also reported fewer barriers to providing high quality care.

The project generated knowledge that can be used to strengthen and faciliate PCC and quality in VA CLCs. Literature regarding the relationship of quality to PCC has been mixed to date, and our findings of no relationship over time adds to this body of literature. We found relatively little variation in CLC quality over time which may partially explain our results in Objective 1. However, we did observe that there were individual CLCs that did improve or get worse over time that may be worth investigating.
The nature of the cost-quality tradeoff in these CLCs suggests that more quality issues or adverse events increase total patient costs. In addition, more PCC practices also tend to be more costly, though not strongly significantly. Our findings suggest the importance of higher quality, not only as an end in itself but as a way to reduce costs in CLCs. Improvements in PCC are associated with small increases in costs, but further research is needed to understand which components of PCC are related to costs (e.g., staffing levels) and to assess patient value gains engendered by PCC initiatives.
We found similarities in the way CLCs implemented PCC and focused on quality which was expected given that all sites were part of the same integrated healthcare system. We found that high performers on both PCC and quality reported greater use of PCC practices, such as training, policies, and recognition, in relation to both outcomes. To improve PCC and quality, CLCs and other nursing homes should consider making improvements in the health systems factors that differentiated high and low performers. In particular, improvements in certain domains (e.g. training, recognition, organizational culture, alignment/coordination) may provide the most impact but will require resources from leaders within the organization and support from staff throughout the organization.

External Links for this Project

NIH Reporter

Grant Number: I01HX000808-01A1

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


Journal Articles

  1. Sullivan JL, Shwartz M, Stolzmann K, Afable MK, Burgess JF. A Longitudinal Assessment of the Effect of Resident-Centered Care on Quality in Veterans Health Administration Community Living Centers. Health services research. 2018 Jun 1; 53(3):1819-1833. [view]
  2. Benzer JK, Cramer IE, Burgess JF, Mohr DC, Sullivan JL, Charns MP. How personal and standardized coordination impact implementation of integrated care. BMC health services research. 2015 Oct 2; 15(1):448. [view]
  3. Engle RL, Tyler DA, Gormley KE, Afable MK, Curyto K, Adjognon OL, Parker VA, Sullivan JL. Identifying barriers to culture change: A qualitative analysis of the obstacles to delivering resident-centered care. Psychological Services. 2017 Aug 1; 14(3):316-326. [view]
  4. Sullivan JL, Engle RL, Tyler D, Afable MK, Gormley K, Shwartz M, Adjognon O, Parker VA. Is Variation in Resident-Centered Care and Quality Performance Related to Health System Factors in Veterans Health Administration Nursing Homes?. Inquiry : A Journal of Medical Care Organization, Provision and Financing. 2018 Jan 1; 55:46958018787031. [view]
  5. Parker V, Engle RL, Afable MK, Tyler DA, Gormley K, Stolzmann K, Shwartz M, Sullivan JL. Staff-perceived Conflict between Resident-centered Care and Quality in the Skilled Nursing Facility: Are Both Possible?. Clinical Gerontologist. 2019 May 1; 42(3):267-276. [view]
  6. Burgess JF, Shwartz M, Stolzmann K, Sullivan JL. The Relationship between Costs and Quality in Veterans Health Administration Community Living Centers: An Analysis Using Longitudinal Data. Health services research. 2018 Oct 1; 53(5):3881-3897. [view]

DRA: Aging, Older Veterans' Health and Care
DRE: Epidemiology, Treatment - Comparative Effectiveness
Keywords: none
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.