2012 HSR&D/QUERI National Conference Abstract
3105 — Data Mining: A Novel Method for Evaluating and Incorporating Survey Data
Glasgow JM, Iowa City VA Healthcare System and University of Iowa; Yano EM, Greater Los Angeles VA; Kaboli PJ, Iowa City VA Healthcare System and University of Iowa;
VA supports a survey culture that produces numerous research and organizational surveys. Responses to these surveys can potentially be of significant use for evaluating how organizational characteristics impact patient outcomes. Despite this potential utility, it can be difficult to evaluate and incorporate complex survey data. The objective of this study is to demonstrate the utility of data mining, an established engineering methodology but a novel technique in health services research, for evaluating the relationship between survey responses and hospital performance during a quality improvement (QI) collaborative.
First, a conceptual model was created to explain how organizational characteristics identified in the VA Clinical Practice Organizational Survey (CPOS) interact to create an environment that predictably responds to QI efforts. Second, the Waikato Environment for Knowledge Development (WEKA) version 3.6.4 was used to develop four data mining decision trees that modeled hospital performance during the FY07 Flow Improvement Inpatient Initiative (FIX). Performance of the decision trees was evaluated using 10-fold analysis and full decision trees were reviewed to detect additional patterns.
Overall, the 10-fold analysis indicated that knowledge about hospital characteristics, as measured in the CPOS survey, was poorly predictive of performance during FIX (kappa = 0 for all decision trees). However, review of the decision trees highlighted four variable categories that regularly appeared in the data mining models: sufficient staff, performance monitoring, inpatient resources, and guideline adherence. These four categories represent areas where organizational structure impacts the likelihood of successful QI.
This study indicates that data mining can be usefully applied to analyze survey data. While the results from this study were predominately hypothesis generating, it indicates a potential approach for parsing large amounts of survey data in a manner that improves understanding of the role of organizational characteristics in patient outcomes.
Data mining is an important tool for VA researchers to be aware of and consider in their toolbox of data analytic approaches. While it has its own weaknesses, its strengths make it an ideal tool for analyzing large data sets, such as survey data, and identifying how to use VA’s wealth of survey data to improve patient outcomes.