3002 — Delayed Follow-up of Abnormal Test Results: A Multifaceted Solution for a Multifaceted Problem
Smith MW, Murphy DR, Laxmisan A, and Reis B, Houston VA HSR&D, Michael E. DeBakey VA Medical Center; Sittig DF, University of Texas Health Science Center at Houston; Singh H, Houston VA HSR&D, Michael E. DeBakey VA Medical Center;
Despite CPRS-based communication via “View Alerts”, abnormal outpatient test results do not always receive timely follow-up. A combination of several factors, including software, user interface and workflow, is likely responsible. To guide the development of a CPRS-based software system (Alert Watch And Response Engine, or AWARE) to reduce future missed follow-up of results, we conducted a multifaceted formative evaluation of utility and usability to address these factors.
Our study involved multiple types of potential AWARE users in VA. We conducted structured interviews with 19 providers (residents/attendings), focusing on how they sorted and filtered alerts and how alerts fit into their workflow. We observed the participants perform specific clinical tasks, including simulated interruption, using a prototype AWARE reminder tool. Eight Patient Safety Managers and clinical supervisors participated in structured interviews about alert follow-up related quality improvement (QI), and used the prototype AWARE QI tool to explore a simulated database of alerts and follow-up actions. To evaluate technical support requirements, we conducted structured interviews and evaluations of both prototype tools with 9 Clinical Application Coordinators. Furthermore, we evaluated the technical feasibility to operationalize our product in the CPRS/VistA environment by installing AWARE on the CPRS test account at our local facility. Responses to the structured interview questions were aggregated. Usability test performance was scored by trained observers based on the ease of task completion.
Providers found the reminder system useful and compatible with their workflow. The staff who evaluated the QI tool found many features relevant to their work. The iterative usability testing led to improvements in both the reminder and the QI tools. AWARE was successfully installed in the CPRS test account and thus is technically compatible.
We successfully conducted the formative evaluation of a novel software product by addressing multiple aspects of successful health IT design and implementation.
As opposed to the development and evaluation of solutions with only one type of end-user, our approach involved addressing tasks, workflows, and technical issues related to multiple types of users. This “socio-technical” methodology could be useful to others as they design, develop, and implement new software applications in electronic health records.