Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic.

Parikh RB, Zhang Y, Kolla L, Chivers C, Courtright KR, Zhu J, Navathe AS, Chen J. Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic. Journal of the American Medical Informatics Association : JAMIA. 2023 Jan 18; 30(2):348-354.

Related HSR&D Project(s)

• SDR 20-361 – Racial Bias in a VA Algorithm for High-Risk Veterans

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.