1023 — Using Machine-Learning Methods to Identify Key Drivers of Family Dissatisfaction with VA End-of-Life Care

Thorpe J, PROMISE Center, Philadelphia, PA; Kutney-Lee A, PROMISE Center, Philadelphia, PA; Smith D, PROMISE Center, Philadelphia, PA; Lu H, PROMISE Center, Philadelphia, PA; Kuzla N, PROMISE Center, Philadelphia, PA; Johnson M, PROMISE Center, Philadelphia, PA; Ersek M, PROMISE Center, Philadelphia, PA;

Objectives:
The value of surveys on the health care experiences of Veterans and family members depends on how well these multi-item surveys can be transformed into actionable knowledge. The objective of this study was to apply machine-learning methods to identify the key drivers of overall dissatisfaction with End-of-Life (EOL) care as measured by VHA's Bereaved Family Survey Performance Measure (BFS-PM).

Methods:
The sample includes next of kin (NOK) who completed the BFS-PM survey for Veterans who died in VA facilities from October 2010-September 2014. BFS-PM items included one global satisfaction item (poor-excellent) and 13 items describing aspects of EOL care. Dissatisfaction was defined as a response of fair/poor on the global item. Factor analysis identified 5 BFS-PM items representing 5 distinct dimensions. Several socio-demographic, health, and facility predictors were also included. Classification and Regression Tree (CART) was used to develop and validate a prediction model and evaluate keys drivers of EOL dissatisfaction.

Results:
Of the 52,778, respondents, 7.4% reported being dissatisfied with EOL care. CART identified 3 BFS-PM items (staff listens, post-death emotional support, spiritual support) and 3 non-BFS variables (age, setting of death, VISN) that identified nine risk factor profiles and probabilities of dissatisfaction (range: 0.7% dissatisfied to 33.1% dissatisfied). The area under the ROC curve for these 6 predictors was 0.87 (sensitivity = 0.85 specificity = 0.79). The highest dissatisfaction rates were observed in NOK who were dissatisfied with both staff listening and the level of emotional support provided by VHA post-death (33.1%). Positive experiences with post-death emotional support mitigated the negative effects of dissatisfaction with staff listening; particularly among Veterans over 78 years who died in hospice and nursing home settings (4.7% dissatisfied).

Implications:
CART analysis identified a subset of six key risk factors for predicting NOK dissatisfaction with EOL care. Additionally, these risk factor combinations would be difficult to detect using traditional statistical methods.

Impacts:
Machine-learning methods indicated that not all survey items may be needed to accurately identify Veterans and family members at greatest risk of dissatisfaction. These methods may be useful for reducing respondent burden and increasing response rates while preserving predictive accuracy.