Health Services Research & Development

Veterans Crisis Line Badge
Go to the ORD website
Go to the QUERI website

CRE 12-320 – HSR&D Study

New | Current | Completed | DRA | DRE | Portfolios/Projects | Centers | Career Development Projects

CRE 12-320
Veterans Like Mine Cognitive Support for Therapeutic Decision Making
Matthew H. Samore MD
VA Salt Lake City Health Care System, Salt Lake City, UT
Salt Lake City, UT
Funding Period: October 2014 - September 2018

Clinicians make decisions about treatment in the face of uncertainty and under the constraints of time. Rare diseases or common diseases with unusual features are examples of situations where clinicians must make decisions in the absence of applicable recommendations from clinical trials or practice guidelines. We propose to develop and implement a novel cognitive support and population-analytic system, called Veterans Like Mine (VLMine) to aid clinicians in therapeutic decision-making. The clinical domain focus of this project is on infectious diseases.

Our goal is to develop an informatics tool that will retrieve and present information on other patients similar to the patient being treated. The VLMine tool will facilitate: the management of diagnostic uncertainty; the assessment of treatment options; and the prediction of clinical outcomes. This tool will provide guidance for therapeutic management by addressing knowledge and experience gaps, inadequately filled by traditional information resources. The Veterans Like Mine project will achieve these goals through the following aims:

Aim 1 - Retrieve and analyze population data relevant to therapeutic decisions at the individual patient level
Aim 2 - Execute and display case-matched population inquiries of VLMine
Aim 3 - Conduct a demonstration study of the VLMine tool for clinical problems in infectious diseases (ID)

Aim 1.1 - Cognitive Task Analysis
Cognitive task analysis (CTA) techniques were used to examine the strategies used by ID experts to manage cases that were challenging or unfamiliar. Ten ID experts were interviewed at the University of Utah and Salt Lake City VA Medical Center. ID experts were asked to recall a critical or vivid antibiotic prescribing incident that they considered complex. Using four iterations of the Critical Decision Method (CDM), a type of CTA, questions were posed to explore the components which underlie clinical complexity. In a follow-up pilot study, the use of population-based data to support decision-making for complex cases was also examined. Ten clinicians were presented with a graphical display of results of a population inquiry to assess impact on treatment decisions in a case vignette.

Aim 1.2 - Population Inquiry Tool
We have developed a prototype tool, built on the JavaFX platform, to perform population inquiries. To improve retrieval performance, we created specialized retrieval aids, such as customized microbiology tables. The query tool leverages the OMOP Common Data Model for retrieval and display.
We have refined the query and cohort building application. We have improved the interface the clinicians see as well as the input options available to them (along with the "behind the scenes" functionality) for building a cohort of similar clinical patients for the PICOT analysis stage of the tool.

Aim 2.2 Case-Matching Methods
The prototype application includes the ability to select diagnoses, procedures, demographic information, and structured microbiology data according to the characteristics of an individual patient. microbiology data can be retrieved by specimen site location, organism, and susceptibility results. This is accomplished by building customized microbiology databases and by leveraging existing VINCI resources such as the OMOP data, thus allowing manual case-matching inquiries. Case matches are viewable at a "population level" where all case matches and relevant microbiology data are scrollable. In addition, users are able to select an individual event result (such as a microbiology finding) or patient to delve into further detail, as desired.

Aim 2.2 - Interactive User Interface
Users can execute case queries for clinical events and include dimensions such as age-range, gender, location, and timeframe. One can build faceted searches, and include comorbidities, drug therapies and procedures, using concept strings or clinical codes, to accommodate complex cases.
The VLMine software platform and customized resources have also been improved. The user interface of the application has been reengineered for better reliability and PHI policy compliance. The customized microbiology tables have been rebuilt using applied antibiogram data and with guidance from a clinician experienced in antimicrobial stewardship. A windows design, facilitates viewing and interaction with results. Users may move, hide and re-launch separate windows containing demographic, event, and PICOT results. Users may sort results, filter results by clicking on a patient id or entering terms in search boxes, or prune output with menus that pop up with a right-click of a mouse.
The population inquiry tool has full text view and full text search capabilities, accommodating search in the TIU ReportText fields. The full text search can be executed 'on the fly' using a restricted set of notes rather than retrieving documents from a full text index across all patients. This is a slightly restricted procedure due to current resource and scale-out limitations. Queries can be a string of words, terms, phrase or sentence. Each term is queried to retrieve a set of relevant documents. The intersection of these results is created. The resulting set of relevant documents maps back to a reduced set of patient identifiers, which can be used to highlight specific patients in the line lists, as well as be displayed separately by patient/relevant documents. Relevant search terms are highlighted in the results.

Aim 2.3 Availability of Population Data on Decision-Making for Vignettes
We will test the influence of population data on clinical decision-making using vignettes, using Judgment Analysis as the methodological framework.

Aim 3.1 Implement the Inquiry Tool for Infectious Disease Management at six VA Medical Centers
We will identify suitable sites and recruit at least two clinicians from each. A "soft launch" will be carried out at each site.

Aim 3.2. Evaluate adoption and perceived usefulness of VLMine tool.
We will analyze basic measures of adoption over time.

Aim 1.1 - Cognitive Task Analysis
Three themes were identified as contributing to uncertainty in clinical reasoning: overall clinical picture does not match a pattern; lack of comprehension of the situation; social and emotional pressures such as fear and anxiety. Five types of strategies to manage complexity were ascertained: 1) watchful waiting with respect to antibiotic prescribing; 2) theory of mind to simulate other clinicians' perspectives; 3) reliance on simple heuristics to reduce complexity; 4) anticipatory thinking to plan and re-plan events; solicitation of opinions from consultants. Evaluation of responses to a test scenario suggested that an approach to decision support based on population analytics holds significant promise. Preferences about the design of population information displays were elicited. Techniques to control the level of view, such zoom and filtering tools, were requested.

Aim 1.2 - Population Inquiry Tool
As described in the Methods section, the Population Inquiry Tool enables layered, faceted searches, while providing multiple types of data visualization for interactive appraisal of the results. We have run queries to evaluate test-case categories such as endocarditis (ICD9 424.90) and prosthetic joint infection (ICD9 996.66). Microbiology data that were returned included: organism, collection dates, collection sites, antibiotic, antibiotic resistance values, and patient identifier. Diagnosis (Condition) values included condition diagnosed, start date, end date (when available), visit type (inpatient, etc.) and patient identifier. Patient demographic data included gender, birth year, city, state, death date (if available), and patient identifier. Total patient count and deceased patient count were included. Bar chart data included visualized results of organisms and counts, and collection sites. The design of the tool has been mapped to epidemiological workflow to answer PICOT (Population, Intervention, Comparison, Outcome, Time) - type questions. We have devised various strategies to address barriers in the VINCI/OMOP environment.

Aim 2.1 - Case-matching Methods
The test-case categories (described in results for Aim 1.2) retrieved initial case-matched populations.

Aim 2.2 - Interactive User Interface
The mechanism to build faceted searches is in place, as well as interactive line-lists and bar charts. We continue to work on the full-text component, which will be fully integrated in the application. Once all work is complete, usability testing will take place. Currently, project team members have used internal iterative testing and feedback to improve interface flow and intuitiveness of use for clinician users, in addition to clarifying the labeling and descriptions for query options. Work has shifted to building the PICOT window to display epidemiologic characteristics of the analysis cohort and summary statistics. These descriptive summary statistics will include information such as frequency of treatment by user-selected medication regimen(s), patient outcomes by regimen, information on mean duration of treatment with each regimen, etc. This design process is ongoing and utilizes both information on clinician display preferences identified through interviews completed in Aim 1.1 and team expertise in bioinformatics. Iterative design will be employed to determine the optimal display content, style, and organization.

Work for sub-aims 2.3, 3.1, and 3.2 will be carried out in the future.

This project is developing a proof-of-concept prototype with the potential to greatly improve patient care by providing an additional decision support tool that identifies protocols that work, as well as less effective protocols, in near-real time, for a Veteran like mine. Upon completion, we expect to have a prototype that can be adapted at the bedside.


Journal Articles

  1. Roosan D, Del Fiol G, Butler J, Livnat Y, Mayer J, Samore M, Jones M, Weir C. Feasibility of Population Health Analytics and Data Visualization for Decision Support in the Infectious Diseases Domain: A pilot study. Applied clinical informatics. 2016 Jun 29; 7(2):604-23.
  2. Islam R, Weir CR, Jones M, Del Fiol G, Samore MH. Understanding complex clinical reasoning in infectious diseases for improving clinical decision support design. BMC medical informatics and decision making. 2015 Nov 30; 15(1):101.
Center Products

  1. Gawron A, Gundlapalli AV, Gawron LM, Samore MH, Rubin MA, Jones MM, Weir CR. Natural Language Processing (NLP) Clinical Extraction Method. 2017 Jul 1.
  2. Weir CR, Samore MH, Chapman WW, Jones MM. Population health Dashboards. 2017 Jul 1.
  3. Samore MH. Veterans Like Mine. 2016 Oct 15.

DRA: Infectious Diseases
DRE: Diagnosis, Treatment - Comparative Effectiveness, Prognosis
Keywords: Care Management Tools, Computational Modeling, Data Visualization, Decision Support, Healthcare Algorithms, Medication Management, Natural Language Processing
MeSH Terms: none