Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
FORUM - Translating research into quality health care for Veterans

» Back to Table of Contents


Improving Safety Measurement in Outpatient Surgery: A Case Study Applying a Trigger Dashboard to Detect Adverse Events

Key Points


  • The VA Surgical Quality Improvement Program (VASQIP) measures quality in VA surgical care and prioritizes procedures for nurse review.

  • The newly developed Trigger Dashboard has been a useful adjunct to VASQIP review, allowing adverse events (AEs) not previously identifiable to be collectively monitored.

  • Detecting safety events and identifying specialties with high AE rates supports earlier intervention.

As VA transforms into a high reliability organization, accurate measurement of safety events will be critical.1 This is particularly true for the 28 ambulatory surgery centers (ASCs) that perform thousands of VA outpatient surgeries annually, yet are not subject to the rigorous quality measurement efforts in hospitals. The VA Surgical Quality Improvement Program (VASQIP) was designed to measure quality in VA surgical care and prioritizes high-risk procedures for nurse review. Trained VASQIP nurses review cases to identify adverse events (AEs) and these data are used to drive quality improvement efforts. Implementation and continuous use of VASQIP has lowered surgical morbidity and mortality throughout VA and is hailed as a significant contribution to patient safety.

Although VASQIP is applicable from complex to ambulatory care settings, fewer cases are VASQIP-eligible in lower complexity settings. In addition, some service lines could perform 10 to 20 percent of total surgical volume without performing VASQIP-eligible cases. Half of surgical site infections identified in a study of VA outpatient surgeries occurred in VASQIP-ineligible procedures.2 To address this gap in surgical AE detection, Dr. Hillary Mull was awarded a VA Health Services Research & Development Career Development Award to develop and validate an AE surveillance tool for outpatient surgery.3 The tool assigns predicted probabilities of an AE to outpatient surgeries, using patient, procedure, and facility characteristics as well as triggers, electronic algorithms based on medical record data, to flag cases with patterns of healthcare utilization consistent with an AE. The surveillance tool identified an overall outpatient surgical AE rate of 9 percent.3

In Spring 2019, we commenced a staff-initiated implementation of the AE surveillance model for outpatient surgery in one ASC (Figure 1). Phase 1 of this effort involved taking three months of cases at the pilot site from the prior fiscal quarter and comparing the AEs detected through the usual VASQIP review process with the predicted probabilities of an AE generated by the surveillance model. We determined that the predicted probabilities of an AE aligned well with the cases that had a true AE, but that many cases with a high probability of an AE were not assigned VASQIP review. These encouraging findings led us to develop a local, internal mechanism to identify outpatient surgeries with a possible AE.

Phase 2 of the project involved working with Knowledge Management & Analytics (KM&A) at the VISN level to develop a program that would flag outpatient surgeries for VASQIP nurse review. Dr. Mull provided the SAS code used to run the triggers and apply coefficients from the logistic regression model used to assign predicted probabilities; however, the KM&A team used SQL coding, and applying the logistic regression model was deemed infeasible. Instead, we decided to program the triggers in a user-friendly format that the VASQIP nurse could assess and determine whether further review might identify an AE. The Trigger Dashboard built by KM&A could classify the outpatient surgeries so cases could be separated by service lines to allow tracking of patients as a quality initiative.

The following triggers from Dr. Mull’s surveillance tool were retained: a postoperative emergency department (ED)/urgent care visit, a postoperative hospital admission, and more than three visits to surgical clinics postoperatively. With input from the VASQIP nurse, we modified the AE surveillance algorithm. The triggers were programmed to cover a 30-day range instead of 15 days and the list of Current Procedure Terminology (CPT) procedures qualifying as outpatient surgeries was expanded. Next, the “2 or more visits to the urology clinic” trigger was dropped because the ASC did not perform many urology surgeries, and the telephone triage trigger – which was tested but ultimately rejected in Dr. Mull’s earlier work – was added to the Trigger Dashboard. Lastly, we added columns to the dashboard indicating whether the patient died within 30 days of the procedure, the number of post-operative days between surgery and ED visit or telephone call, and the length of stay for the admission. The Phase 2 AE Trigger Dashboard required 13 hours of effort from the KM&A team – three hours to write the code and 10 hours to make user-driven modifications and an acceptable interface.

Following a use period of three months, we revised the trigger dashboard again to improve the detection of AEs and remove trigger flags with limited utility. We also added a new trigger that was not used in the AE surveillance system Dr. Mull created – a flag for whether the patient had any emergency care covered on a fee-for-service basis in the community. We removed the telephone triage trigger and added back the urology clinic trigger. While it requires more time than reviewing only the VASQIP-targeted cases, the Trigger Dashboard helped to identify five outpatient surgeries with an AE compared to zero AEs detected in the VASQIP-eligible procedures. These AEs include surgical site infections, urinary tract infections, and urinary retention, all of which represent potentially preventable surgical complications.

The present version of the Trigger Dashboard has been a useful adjunct to VASQIP review. Tracking of events with the triggers has allowed events not previously identifiable to be collectively monitored. The tool also allows different triggers to be prioritized by population or specific event concerns. For example, all podiatry patients could be tracked for ED visits or clinic-based joint injections for hospital admissions. In many ASCs, podiatry surgeries make up a significant proportion of cases and the patients are often high risk with multiple comorbidities. This tracking would be labor intensive and cumbersome if performed without the dashboard. Perhaps the most useful contribution of the dashboard is the inclusion of podiatry procedures as this specialty is largely excluded from VASQIP. Detecting safety events and identifying specialties with high AE rates supports earlier intervention.

The Trigger Dashboard is facilitating ongoing quality measurement of surgical care in an ASC. This close collaboration between research, clinical care, and KM&A is a case study in how VA translates research into practice and furthers our goal to become an enterprise-wide high reliability organization.

Description of Implementation Process at One VA ASC*
  1. Agency for Healthcare Research and Quality (AHRQ). Patient Safety Primer: High Reliability, 2019.
  2. Linsenmeyer K, Branch-Elliman W, Kalver E, Mull HJ. “Surgical Site Infections in Outpatient Surgeries: Less Invasive Procedures Contribute Substantially to the Overall Burden,” Infection Control & Hospital Epidemiology 2019; 40(10):1191-3.
  3. Mull HJ, Itani KMF, Pizer SD, et al. “Development of an Adverse Event Surveillance Model for Outpatient Surgery in the Veterans Health Administration,” Health Services Research 2018; 53(6):4507-28.

Previous | Next



Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.