HSR&D Home » Research » PPO 09-298 – HSR&D Study
Redesigning Medication Alerts to Support Prescriber Workflow
Alissa Lynn Russ, PhD
Richard L. Roudebush VA Medical Center, Indianapolis, IN
Funding Period: September 2010 - August 2011
The VA's medication alert system automatically warns prescribers about potential drug allergies, drug-drug interactions, etc during order entry. These alerts are particularly important given that an ever-increasing number of medications are available for treatment and preventable medication errors contribute to many adverse drug events (ADEs). However, VA data suggest that many alerts do not aid prescriber decision-making. In addition, several barriers identified during routine VA prescribing relate to the design of medication alerts. The feasibility of prototyping and evaluating alert redesigns has not been previously demonstrated.
The objective of this pilot proposal was to investigate whether interface redesigns can effectively address some common problems associated with medication alerts and evaluate the feasibility of these methods for a larger, more complex investigation of alert redesigns. The specific aims for this study were as follows: Aim 1: Design new medication alert prototypes and evaluate the usability of these redesigns. Aim 2: Assess whether alert redesigns reduce common barriers associated with current medication alerts.
For Aim 1, we combined literature findings with input from an expert panel to create new medication alert prototypes. We redesigned the presentation of allergy data; navigation to drug-drug interaction details; timing of drug-laboratory alerts; and mechanism to enter a reason for overriding alerts. In addition, we attempted to streamline the presentation of information on alerts and reduce textual information overload. We also developed three complex, clinically-relevant patient scenarios to evaluate the redesigns. Scenarios were crafted to prompt 5 drug-allergy alerts, 11 drug-drug interaction alerts, and 3 drug-laboratory alerts. Safety probes were included in the scenarios to test potential alert design weaknesses. For Aim 2, simulation experiments were conducted with 20 VA outpatient, primary care prescribers (physicians, nurse practitioners, and pharmacists) in a within-subject comparison of current alerts versus redesigned alerts. Prescribers completed two 30 minute sessions, one for each alert design; in the first session, half of the prescribers received the VA alert design in the first session and half the prescribers received the redesigned alerts. Prescribers were asked to 'think aloud' and were video recorded. After the scenarios, prescribers also completed a computer system usability questionnaire (CSUQ) (7-point Likert scale; 1 = strongly disagree and 7 = strongly agree) and NASA TLX workload assessment. Summed responses to questionnaire subscales were analyzed with paired t-tests. Individual items that were added to the questionnaire to assess specific design features were analyzed with the Wilcoxon signed-rank test. NASA TLX was evaluated with a two-way analysis of variance (ANOVA). For safety probes and alerts, we analyzed how the two alert designs affected decision-making error rates. These results were analyzed with chi-squared McNemar tests and the Wilcoxon signed-rank test. For all statistical analyses, a p-value < 0.05 was considered significant.
The alert design type did not have a significant main effect on the overall NASA TLX workload assessment. However, the redesigned alerts improved usability across several dimensions, based upon a qualitative usability analysis of the video recordings as well as quantitative ratings from the usability questionnaire. Qualitative findings indicated that the layout of information on the redesigned alerts enhanced usability. In addition, the following CSUQ scores were significantly greater for redesigned alerts compared to current alerts: 'overall satisfaction', (redesigned alerts: Mean = 5.2, SD = 1.4 vs. current alerts: Mean = 4.3, SD = 1.2, p = .033); 'information quality' (redesigned alerts: Mean = 5.2, SD = 1.4 vs. current alerts: Mean = 4.1, SD = 1.2; p = 0.013); and 'interface quality' (redesigned alerts: Mean = 5.2, SD = 1.5 vs. current alerts: Mean = 4.3, SD = 1.3; p = 0.005). In addition, for redesigned alerts, the efficiency of viewing allergy histories was rated significantly higher (redesigned alerts: Mean = 6.0, SD = 1.3 vs. current alerts: Mean = 3.6, SD = 1.9; p = 0.001), as was the efficiency of entering the reason for overriding alerts (redesigned alerts: Mean = 5.6, SD = 1.6 vs. current alerts: Mean = 4.4, SD = 1.7; p = 0.032). Ratings were not significantly different across the two designs for the efficiency of viewing laboratory results or drug interaction monographs. Prescribers rated information quality greater for redesigned alerts, although much of the clinical information was consistent across the two alert designs. Modifying alerts to reduce textual information overload may enhance prescribers' perceptions about the quality of information presented by alerts and improve workflow.
For safety probes and decision-making, appropriate decisions were significantly greater for alert redesigns compared to current alerts for all three alert types: allergy alerts (redesigned alerts: Median = 5.0, SE = 0.1 vs. current alerts: Median = 3.5, SE = 0.2, p = 0.001); drug-laboratory alerts for low creatinine clearance (redesigned alerts: Median = 2.0, SE = 0.2 vs. current alerts: Median = 0.5, SE = 0.2, p = 0.001); and drug-drug interaction alerts (redesigned alerts: Median = 4.5, SE = 0.2 vs. current alerts: Median = 3.0, SE = 0.3, p = 0.028). Finally, the percentage of override reasons that were appropriate were significantly greater for redesigned alerts that offer a drop-down list of options (Median = 85%, SE = 3.0) compared to current alerts that rely on free-text entry (Median = 50%, SE = 6.0, p = 0.001). Thus, several aspects of decision-making were improved with the redesigned alerts, indicating that these new designs may enhance workflow and patient safety.
This study addressed common design problems that weaken the effectiveness of VA medication alerts. Results can help the VA understand how to design alerts to better support clinical workflow during medication order entry. Findings may be used to improve medication alert design for physicians, nurse practitioners, and pharmacists, and may ultimately be used to reduce ADEs and enhance medication safety for veterans. Study results may also inform alert designs in other institutions and computerized provider order entry (CPOE) systems.
External Links for this Project
NIH ReporterGrant Number: I01HX000343-01
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Health Systems
DRE: Research Infrastructure, Treatment - Observational, Technology Development and Assessment
Keywords: Decision support, Pharmaceuticals, Safety
MeSH Terms: none