Lead/Presenter: Ying Yin, Washington DC VA Medical Center
All Authors: Yin Y (Washington DC VA Medical Center; The George Washington University, Biomedical Informatics Center), Garvin JH (VA HSR&D Center for Health Information and Communication) Zeng-Treitler Q (Washington DC VA Medical Center; The George Washington University, Biomedical Informatics Center) Nelson SJ (The George Washington University, Biomedical Informatics Center)
Objectives:
A challenge for informaticians is the need to determine a cohort for examination in a study, often using standard codes. While traditionally EHR data quality is measured by 5 dimensions: completeness (e.g., the amount of missing data), correctness, concordance (e.g., the use of terminology standards), plausibility (e.g., out-of-range values), and currency (e.g., the timeliness or recency of data), none of these measures looks at the quality of coding. Potential biases might occur when the interpretation of a given clinical situation might be represented using separate sets of codes, or where a set of identical codes can be used to represent two different situations. We call such situations a violation of representational sematic (RS) integrity, a generalization of the Cimino desiderata of avoiding redundancy and ambiguity. The transition from ICD-9CM to ICD-10CM on October 1, 2015, led to an abrupt disruption of efforts by researchers to develop and maintain consistent cohorts for their research. Despite the development of a General Equivalence Mapping by CMS, marked changes in the frequency of certain conditions were observed. The objective of this study is to use machine learning models to identify the RS violation during ICD-9CM to ICD-10CM transition within VA CDW.
Methods:
After forming clusters of ICD-9CM and ICD-10CM codes of similar meaning, two machine learning methods (transformer model and SARIMA model), using co-occurring laboratory, medication, and procedure codes, were used to predict the frequency of code use.
Results:
At the time of the transition, dramatic changes in the frequency of occurrence of some of the clusters were observed. 47% of the 700 clusters studied had changes in the frequency of greater than 4 standardized residuals. The two methods were similar in their findings.
Implications:
How many of these dramatic changes are due to violations of RS integrity requires further investigation. Other possibilities include unfamiliarity with a new coding system, poor quality mapping, or aberrancy in training the machine learning models.
Impacts:
Though further investigations are needed, the findings could indicate that the violations of RS integrity can be a widespread issue, which could have a significant impact on the quality of medical informatics research.