HSR&D Home » Research » RRP 12-184 – HSR&D Study
Developing Electronic Surveillance Measures of Overuse of Colorectal Cancer Screening
Sameer D. Saini, MD MS
VA Ann Arbor Healthcare System, Ann Arbor, MI
Ann Arbor, MI
Funding Period: July 2012 - September 2013
Colorectal cancer (CRC) is the second-leading cause of cancer-related death in the United States. Screening for CRC has been shown to be effective. While numerous studies have demonstrated that CRC screening is underused in screen-eligible patients, concurrent work has demonstrated that screening is also overused. Overuse of screening can lead to tests, procedures, and treatments that not only have little probability of benefit, but can also cause discomfort, diminished quality of life, and injury or death due to procedure-related complications. Yet, current VA CRC screening performance measures are focused exclusively on underuse of screening. Improving the quality of CRC screening in VA will require that we not only increase screening in patients likely to benefit from it, but that we also reduce screening in patients who are unlikely to benefit or may experience net harm.
(1) to develop a high-specificity electronic surveillance measure of CRC screening overuse, using data from the Corporate Data Warehouse (CDW); (2) to validate the reliability of this overuse measure through comparison to manual chart review; (3) to describe the prevalence of CRC screening overuse and the variation of overuse across VA facilities; (4) to develop and validate a natural language processing (NLP) algorithm to extract colonoscopy indication from colonoscopy procedure reports.
We accessed CDW to identify all Veterans who underwent screening fecal occult blood testing (FOBT) or screening colonoscopy at a VA facility in fiscal year (FY) 2013. We then determined whether the screening test was appropriate according to definitions developed collaboratively by an expert workgroup comprising VA leaders in gastroenterology (GI), quality measurement, and implementation science. Screening tests were classified as appropriate, probable overuse, or possible overuse. The electronic CDW algorithm was validated by comparing CDW results to those obtained by chart review (for a stratified random sample of 4,000 Veterans). Prevalence of overuse was tabulated by facility and VISN, and variation in overuse across facilities and VISNs was also examined. Finally, in collaboration with colleagues at the Salt Lake City VA, we developed an NLP algorithm to extract colonoscopy indication from colonoscopy reports. The reliability of the standard CDW algorithm was compared to the reliability of an NLP-enhanced CDW algorithm.
An expert workgroup collaboratively developed and agreed upon the definitions and elements of overuse measures for FOBT and colonoscopy. Compared to gold-standard chart review, the sensitivity and specificity of the CDW algorithm for overuse FOBT were 71% and 81%, respectively (AUROC = 0.76, 95% CI: 0.72-0.80). Sensitivity and specificity for overuse colonoscopy were 20% and 97%, respectively (AUROC = 0.58, 95% CI: 0.56-0.61). In FY13, 342,563 screening FOBTs were identified, and 95,176 (27.8%) of these met the definition for probable (10.1%) or for possible (17.7%) overuse. Over the same time period, 88,754 screening colonoscopies were identified, and 20,530 (23.1%) of these met the definition for probable (17.2%) or for possible (5.9%) overuse. Substantial variation in colonoscopy overuse was noted between facilities and between VISNs, with a nearly 8-fold difference between the maximum and minimum rates of overuse at the facility level and a nearly 2-fold difference at the VISN level. A reliable NLP algorithm was developed for extraction of colonoscopy indication from colonoscopy procedure notes (sensitivity = 83%, specificity = 98%). The addition of NLP to the CDW algorithm modestly enhanced sensitivity for overuse colonoscopy (sensitivity increased from 20% to 34%) without compromising specificity, improving overall reliability (AUROC increased from 0.58 to 0.64).
The overuse measures developed by this study can be utilized by VHA to examine overuse and improve the appropriateness of CRC screening in VA. The knowledge gained from this work can be extended to other preventive services, ultimately enhancing appropriate use of preventive care in VHA at large.
External Links for this Project
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
MeSH Terms: none