Catheter-associated urinary tract infection (CAUTI) is one of the most common hospital-acquired infections. Limiting catheter use and accurate monitoring of urinary catheter-days are key components of CAUTI prevention, as are identifying positive urine cultures and linking these to catheter use. The VHA mandates reporting of urinary catheter-days and number of CAUTI by each facility to the national Inpatient Evaluation Center (IPEC). Since urinary catheter documentation often appears only as free text in nursing notes, electronic capture of urinary catheter-days is not currently possible. Natural language processing (NLP) is a potential approach to streamlining reporting of urinary catheter use and CAUTI while improving accuracy. NLP refers to the various ways in which computers analyze written or spoken language.
Our hypothesis was that computer-facilitated surveillance of urinary catheter use and urine cultures would reduce labor intensive processes and would be more accurate than current manual methods.
Aim 1 To develop and validate an algorithm using NLP and machine learning techniques to count urinary catheter-days and to link urine culture results to the presence of a urinary catheter for specific patients.
Aim 1a To explore the feasibility of using NLP and machine learning techniques to detect urinary tract infection symptom information, linked temporally to a urinary catheter and to a positive urine culture.
Aim 2 To field test implementation feasibility for automated reports produced by NLP in parallel with existing methods to assess opportunities to replace current reporting strategies with fully-automated reporting.
Aim 1 First, 12 months of patient data from two VA hospitals in an existing database of urinary catheter-days and urine cultures were linked to the corresponding electronic medical records (EMR) in VINCI. We then developed a lexicon, or terms and phrases, used in medical documentation that refer to urinary catheters, urine cultures, and symptoms of CAUTI. NLP modules searched for catheter lexicon terms in notes written by nurses and other healthcare workers, and the concepts extracted were classified as evidence of either catheter presence or catheter absence. A reference standard set of 1595 randomly selected documents from inpatient admissions were annotated by human reviewers to identify all positively and negatively asserted concepts. An NLP algorithm was tuned using 100 documents from the set. Tuning enabled development of algorithms to infer presence or absence of a catheter on days with inadequate documentation. The NLP algorithm was then validated on the remaining 1495 documents to determine agreement between NLP and human reference standard, sensitivity and positive predictive value (PPV). EMR note titles with the highest hit rate for concepts were identified.
Aim 2 Development of the automated urinary catheter report format employed iterative design, in which the research team showed a sample NLP-generated data report to potential end-users, surveyed them about usability, and then revised the report accordingly. We went through 4 such cycles, grouping the first 2 iterations into Version 1 and the second two into Version 2 because most design changes were made between iterations 2 and 3. Our prototype report format contained a month of actual data from an acute care ward in our hospital. The survey consisted of 10 questions exploring the following domains: layout, understandability, completeness of data, and ability to replace current reporting methods. We recorded the time participants spent looking at the report and asked one quiz question to assess the participant's interpretation of the data provided.
Aim 1 The overall cohort included 5,589 unique patients (both acute and long-term care) with 77,938 bed-days and 572,419 clinical documents. The lexicon contained 590 concepts for catheter presence (eg. Foley catheter was placed) and 18 for evidence of absence (eg. patient has bathroom privileges). The overall agreement between the NLP and reference standard was 71%. With 348 instances of 'evidence of catheter presence,' the system found 246 for a sensitivity of 87%. With 84 false-positive concepts associated with catheter presence, the PPV was 59%. For 'evidence of catheter absence', the agreement was 72% (450 instances), sensitivity was 77% and PPV was 68%. Overall, nurses' notes were the most frequent inpatient notes and yielded the highest number of concepts with respect to urinary catheters.
Aim 2 The 40 participants surveyed included physicians, ward nurses, nurse CAUTI champions, quality managers, and infection control specialists. The average time spent looking at the report was 47.2 seconds. Report Version 1: 45% answered the quiz question correctly. Of the 4 domains, the lowest score was in layout, receiving an average of 3.3/5 points. We made the display of catheter days more graphical in response to users' comments. Report Version 2: 76% answered the quiz question correctly. The lowest scoring domain was ability to replace current reporting methods, with an average score of 3.2/5 points. User comments suggested that the report will need to meet the disparate needs of different provider types.
This RRP has brought together informatics research and clinical epidemiology to support a high priority operational need. Although the work is ongoing, we have accomplished the first key step in developing computer protocols to assist humans in large-scale review of patient charts for CAUTI. This project is likely to increase detection and reporting of the presence of urinary catheters and eventually CAUTI, while decreasing staff burden. Our project is a model for future work to detect other high-impact infections, including central line-associated bloodstream infections, and potentially other non-infectious harms, such as venous-thromboembolism. Rapid, real-time detection and reporting of these harms could reduce hospital-associated morbidity and mortality.
- Gundlapalli AV, Divita G, Redd A, Carter ME, Ko D, Rubin M, Samore M, Strymish J, Krein S, Gupta K, Sales A, Trautner BW. Detecting the presence of an indwelling urinary catheter and urinary symptoms in hospitalized patients using natural language processing. Journal of Biomedical Informatics. 2017 Jul 1; 71S:S39-S45.
- Skelton F, Campbell B, Horwitz D, Krein S, Sales A, Gundlapalli A, Trautner BW. Developing a user-friendly report for electronically assisted surveillance of catheter-associated urinary tract infection. American journal of infection control. 2017 May 1; 45(5):572-574.
- Mody L, Greene MT, Saint S, Meddings J, Trautner BW, Wald HL, Crnich C, Banaszak-Holl J, McNamara SE, King BJ, Hogikyan R, Edson BS, Krein SL. Comparing Catheter-Associated Urinary Tract Infection Prevention Programs Between Veterans Affairs Nursing Homes and Non-Veterans Affairs Nursing Homes. Infection control and hospital epidemiology. 2017 Mar 1; 38(3):287-293.
- Fakih MG, Gould CV, Trautner BW, Meddings J, Olmsted RN, Krein SL, Saint S. Beyond Infection: Device Utilization Ratio as a Performance Measure for Urinary Catheter Harm. Infection control and hospital epidemiology. 2016 Mar 1; 37(3):327-33.
- Carter ME, Divita G, Redd A, Rubin MA, Samore MH, Gupta K, Trautner BW, Gundlapalli AV. Finding 'Evidence of Absence' in Medical Notes: Using NLP for Clinical Inferencing. Studies in health technology and informatics. 2016 Jan 1; 226:79-82.
- Divita G, Carter M, Redd A, Zeng Q, Gupta K, Trautner B, Samore M, Gundlapalli A. Scaling-up NLP Pipelines to Process Large Corpora of Clinical Notes. Methods of Information in Medicine. 2015 Nov 4; 54(6):548-52.
- Trautner B. How Practice and Surveillance Affect Your CAUTI Efforts. [Cyberseminar]. 2014 Jun 15.
- Skelton F, Campbell B, Horwitz DJ, Saint S, Krein S, Sales A, Trautner B. Developing a User-Friendly Report for Electronically-Assisted CAUTI Surveillance. Poster session presented at: Society for Healthcare Epidemiology of America Annual Scientific Meeting; 2016 May 19; Atlanta, GA.