BACKGROUND/RATIONALE:
The number of patients with cirrhosis and advanced liver disease has been growing in the VA system and general population of the US. As of 2008, the prevalence of chronic liver disease in the US reached 15%. Complications of cirrhosis frequently require hospital admission, and each year, cirrhosis is responsible for >150,000 hospitalizations at a cost of approximately $4 billion. Among patients who survive the initial hospitalization, nearly half are re-hospitalized within 1 year. The use of clinical decision support (CDSS) in clinical dashboards has potential for facilitating more robust identification of high risk patients, risk stratification and tailored clinical care interventions. Clinical decision support has been disseminated into general clinical practice slowly because of the sophisticated underlying data requirements and a lack of focus on clinical workflow and efficiency optimization.


OBJECTIVE(S):
The overall objective of this project is to develop the informatics infrastructure and tools to facilitate improved evidence based quality care delivery to patients with cirrhosis that will impact readmission and mortality rates. More specifically, we have these SPECIFIC AIMS: 1) develop and validate near real-time natural language processing (NLP) tools in order to extract information that is relevant for case finding and risk factor modification among these patients, 2) develop and validate a robust family of logistic regression prediction models for readmission and mortality following hospitalization for use in the identification of high risk patients, 3) development of a clinical dashboard with imbedded clinical decision support and patient data visualization tools to support clinical care delivery, and conduct a pre-post clinical pilot to evaluate the efficacy and adoption of the dashboard when used.

METHODS:
This proposal will analyze national retrospective cohort data among adult hospitalized patients for Specific Aims 1 and 2, beginning with all hospitalized patients and identifying the cohort of patients with cirrhosis in order to develop predictive models for readmission and mortality. All variables will be extracted from structured data in the CDW, Medical SAS, and Medicare files, with real time NLP used to extract risk factors from unstructured data. Augmented case finding will be used in Aim 1 to detect additional cirrhotic patients, and the discussed risk factors, social history factors, and modifiable clinical variables will be used to generate logistic regression models for readmission and mortality, internally validated with bootstrapping. The NLP pipeline and prediction models will be incorporated into a clinical dashboard that will be developed to support the clinical care delivery needs of these patients, as described in Specific Aim 3 using established implementation science framework methods through observation and interview of the providers using the dashboard. Finally, a pre-post clinical pilot will be conducted to evaluate efficacy and adoption of the dashboard for use in inpatient and outpatient cirrhotic patient care at the San Diego and TVHS VA facilities.

FINDINGS/RESULTS:
Aim 1: To test the utility of candidate variables for inclusion in future NLP algorithms for identification of patients with cirrhosis we performed a retrospective study of 270 consecutive patients who underwent liver biopsy from 2009-2012. This study confirmed the utility of a NLP strategy for more accurate identification of patients with cirrhosis. (Kung R et al. A natural language processing algorithm for identification of patients with cirrhosis from electronic medical records. Gastroenterology 2015;148:S1071-S1072). Subsequently we studied an algorithm to identify patients with cirrhosis in 504 patients at risk for cirrhosis hospitalized between 2005 and 2013. From hepatologist chart review, 151 were negative, 237 were indeterminate, and 116 were positive for cirrhosis. We used 148 candidate predictor variables from the electronic medical record and performed variable selection using penalized logistic regression (LR),validated through bootstrapping an unpenalized model. The LASSO selected 8 variables out of the 148: age; number of ultrasound examinations in the past year; history of ascites, coagulopathy, and alcohol abuse; and laboratory values for albumin, AST, and platelets. The model only counting 'Clinically Yes' as true positives had an AUC of 0.91 (95% CI: 0.87 - 0.95); whereas the FIB4, APRI, Bonacini, and API were 0.75 (0.69 - 0.81), 0.73 (0.67 - 0.79), 0.79 (0.73 - 0.84), and 0.79 (0.73 - 85) respectively. The model including hepatologist cirrhosis determinations of 'yes' and 'indeterminate' as true positives had slightly worse performance (0.87, 95% CI: 0.82 - 0.92); other risk scores were similarly worse. This cirrhosis risk model based on a large set of candidate predictor variables had excellent performance compared to existing risk models. This algorithm could be automated for use in the electronic medical record to alert providers (Jejo KD et al. Algorithm for identification of patients at high risk for cirrhosis from administrative data. AASLD meeting 2017). Hepatorenal Syndrome (HRS) is a severe manifestation of cirrhosis, but phenotyping algorithms have not yet been developed using large electronic health record (EHR) databases. We evaluated and compared multiple phenotyping methods to achieve an accurate algorithm for HRS identification. A national retrospective cohort of patients with cirrhosis and renal disease admitted to 124 Veterans Affairs hospitals from 2005 to 2013. Five hundred and four hospitalizations were manual chart reviewed and served as the gold standard. EHR based predictors were identified using structured and free text clinical data, subjected through NLP from the clinical Text Analysis Knowledge Extraction System. With the combined structured and NLP variables, we analyzed five phenotyping algorithms: penalized logistic regression, naïve Bayes, support vector machines, random forest, and gradient boosting. Calibration and discrimination metrics were calculated using 100 bootstrap iterations. The area under the receiver operating characteristic curve (AUC) for the different models ranged from 0.73 to 0.93; with penalized logistic regression having the best discriminatory performance. Calibration for logistic regression was modest, but gradient boosting and support vector machines were superior. NLP identified 6985 variables; a priori variable selection performed similarly to dimensionality reduction using high-throughput phenotyping and semantic similarity informed clustering (AUC of 0.81 - 0.82). This study demonstrated improved phenotyping of a challenging renal disease etiology, HRS, over ICD-9 coding. (Koola JD et al. Development of an automated phenotyping algorithm for hepatorenal syndrome. J Biomed Inform 2018; Apr;80:87-95. doi: 10.1016)
Aim 2: We analyzed a retrospective cohort of 76,815 patient hospitalizations from among 124 VA medical centers between January 1, 2005 and December 31, 2013 who survived to discharge. The 30-day, 90-day, 1-year, and 5-year mortality in our cohort was 7%, 15%, 29%, and 52% respectively. The median follow up was 780 days (IQR: 254-1608). 91 factors were statistically significant, of which 14 increased the hazard ratio by greater than 20%. A history of TIPS was one of the few variables to dramatically lower mortality (0.70, [0.59-0.83]) This study, of a national VA cohort, identified 91 variables that significantly influence mortality of patients hospitalized with cirrhosis including novel factors not reported in the literature. mortality (Koola J et al. Post-Discharge Mortality in a National Cohort of Veterans Affairs Patients with Cirrhosis. Presented at AASLD, Boston, MA, November 14, 2016). We analyzed 193,674 index hospital admissions comprising 72,737 patients, and of those, there were 43,674 readmissions within 30 days, for a readmission rate of 22.6%. We identified several modifiable risk factors such as choice of antibiotics when treating infections and using rifaximin for treatment of hepatic encephalopathy. Though several variables are likely proxies for disease severity, it is possible that some of the significant variables could be modifiable to reduce risk of re-admission or identify patients that warrant population health management following discharge. (Koola J et al. Machine Learning Models to Predict Readmission for Patients with Cirrhosis. AMIA, meeting 2017). We worked to develop and validate mortality risk prediction model following the hospitalization of cirrhotic patients. We developed and validated a prognostic score predicting post-hospitalization mortality for cirrhotic patients and compared performance with the MELD, MELD-Na, and the CLIF-C AD scores in a retrospective cohort of 73,976 patients hospitalized between 2006-2013 at any of the VA hospitals. We included 140 predictor variables and built a time-dependent Cox proportional hazards model with all-cause mortality as the outcome and censoring occurring at liver transplant or last contact with the VA. We compared performance to the three extant models, and report discrimination and calibration using bootstrapping. Furthermore, we analyze differential utility using the net reclassification index (NRI). The C-statistic for the final model was 0.839, and a significant improvement over the MELD, MELD-Na, and the CLIF-C AD, which had C-statistics of 0.676, 0.693, 0.688, respectively. Multiple risk factors were significant in our model, including variables reflecting disease severity, cirrhosis complications, and hemodynamic compromise. Important modifiable risk factors were use of opiates, Transjugular Intrahepatic Portosystemic Shunts, and mental healthcare use. The event-specific NRI showed significantly improved survival and mortality prediction for low- and high-risk patients, respectively, compared to other models. This study developed a more accurate risk prediction score in predicting subsequent mortality in a general cohort of hospitalized cirrhotic patients compared to existing models. The model may be used to identify a high-risk cohort for targeted higher intensity post discharge management. (Koola JD et al. submitted)
Aim 3: We completed an observational work-flow study and a multi-site interdisciplinary collaborative Design Workshop consisting of 14 clinicians, graphic designers, informatics specialists, and health service researchers to develop a CDS tool prototype using a human factor engineering approach. Workflow observation study: At Site A, 26 physicians, 3 nurses and 1 clerk were observed in: 1) gastrointestinal (GI) outpatient clinics (31%); 2) inpatient medical rounds (69%). At Site B, 14 physicians were observed in: 1) GI outpatient (14%) & Primary Care clinics (29%); 2) Emergency Department (21%); 3) inpatient medical rounds (29%) and 4) inpatient GI consult rounds (7%). 2/3 of participants were trainees & 1/3 were attending physicians. We categorized 220 of 341 unique topics into CDS-relevant themes applicable to CDS tool construction: 1) Clinical work is distributed across people, space, and time; 2) communication and coordination hub roles may amplify the usefulness of CCDS interventions; 3) integrate CCDS with clinical assessment & planning processes; and 4) provide CCDS in electronic & hardcopy forms (Miller A et al. Application of contextual design methods to inform targeted clinical decision support interventions in sub-specialty care environments. International J Biomedical Informatics 2018 in press). Then an interactive CDS tool to improve the risk stratification and care of patients with advanced liver disease was developed based on these findings. The tool prototype was converted into a functioning URL based tool and interfaced with the electronic medical record. It was interfaced with the EMR such that a patient's cirrhosis-related medical history is pulled and presented by category, along with a cirrhosis-related risk score and guideline-based treatment recommendations. We completed formative evaluations to validate the usability and efficacy of the Cirrhosis CDS in the test EMR environment. User feedback and observations of potential safety risks from that evaluation informed further changes in the Tool's user interface and clinical content. (Garvin JH et al Design and formative evaluation of cirrhosis order set and clinical decision support for improved care in the Department of Veterans Affairs, in preparation). We will evaluate the impact of the CCDS Tool use at two hospitals on provider acceptance, usability, and compliance with quality of care indicators for patients with cirrhosis. We have obtained approvals for CCDS implementation at SD and TVHS. IT and facility permissions to deploy are being negotiated to prepare for the final live pilot trial. Technical verification of tool functionality, including the mapping of site-specific identifiers, has been completed. We plan on implementing the CCDS in routine clinical practice and evaluate the impact on cirrhosis quality of care measures using a pre-post study design.

IMPACT:
The innovation of this proposal is that it brings together a full spectrum of informatics and health services research tools to achieve clinical decision support for population management, with cirrhosis care as the exemplar. This research has the potential to directly impact national VA clinical care through implementation of NLP tools directly into clinical care and extending knowledge gained from the design, development, and usability testing of dashboard tools within the VA context. The latter is relevant and timely to the development of the new VA EHR, which has a specific goal of supporting population care management. This informatics research will be done in partnership with OABI, hi2, IRDC, HMP, and iEHR operational collaborators, largely through existing partnerships and collaborations within ongoing projects. This collaboration and information sharing will make the national operational use of the products of this proposal much more likely as components of the tools are transferred from the research domain to operational systems. Our progress to date indicates that we can more accurately stratify the risks for patients with cirrhosis, and also have clinical decision support tools that can be integrated within usual work flows to improve quality of care for these patients. Formative evaluations of the CDS tool indicate that it supports workflow for clinicians and facilitates clinical decision making through display of relevant information. When fully implemented, this has the potential to impact VA clinical care through identifying high risk patients and suggesting clinical care changes in inpatient and outpatient clinical care.

---

External Links for this Project

NIH Reporter

Grant Number: I01HX001284-01A1
Link: https://reporter.nih.gov/project-details/8675417

---

PUBLICATIONS:

Journal Articles

1. Miller A, Koola JD, Matheny ME, Ducom JH, Slagle JM, Groessl EJ, Minter FF, Garvin JH, Weinger MB, Ho SB. Application of contextual design methods to inform targeted clinical decision support interventions in sub-specialty care environments. International journal of medical informatics. 2018 Sep 1; 117:55-65. [view]

Conference Presentations

1. Garvin JH. workflow-guided development of a clinical decision support tool. Poster session presented at: American Medical Informatics Association Annual Symposium; 2015 Nov 15; San Francisco, CA. [view]

DRA: Digestive Diseases, Infectious Diseases
DRE: Prevention, Prognosis
Keywords: Care Management Tools, Decision Support, HIV/AIDS, Hepatitis C, Natural Language Processing, Predictive Modeling, Risk Adjustment, Technology Development
MeSH Terms: none