Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
Publication Briefs

Study Suggests Comparisons between VA and Non-VA Hospitals May Not Accurately Account for Mental Health Diagnoses

Patients, purchasers, and policymakers want to compare hospitals on a wide array of performance metrics. The Centers for Medicare & Medicaid Services (CMS) publish many metrics on their Hospital Compare website; VA- and DoD-affiliated hospitals also contribute data to this website. While CMS publishes performance metrics on Hospital Compare, the risk-adjustment algorithms underlying these metrics are often unclear. Further, recently published literature questions whether existing risk-adjustment algorithms accurately adjust for mental health comorbidities. This study sought to determine whether current risk-adjustment algorithms fairly compare VA hospitals with non-federal hospitals. In their analysis, investigators used the CMS Hierarchical Condition Categories (HCC) risk adjustment system version 21 (V21) because it is publicly available and has been used to adjust metrics published on the CMS Hospital Compare website. Investigators also measured mental health comorbidities using the PsyCMS (Psychiatric Case Mix System). Performed from 9/8/2015 to 10/22/2018, this study included 5.5 million Veterans who received inpatient or outpatient VA care in 2012.


  • Current comparisons between VA and non-VA hospitals are flawed because the risk adjustment algorithms used to make patients comparable do not adequately control for mental health issues. Of 5,472,629 VA patients, the V21 model identified 694,706 as having mental health or substance use HCCs. The PsyCMS identified another 1,266,938 Veterans with mental health diagnoses.
  • The top 10 mental health diagnoses missed by the V21 model included nicotine dependence (40%), depression not otherwise specified (31%), PTSD (27%), and anxiety (10%).
  • Overall, the V21 model under-estimated costs for patients with low costs and over-estimated costs for patients with above average costs except for the top decile. For Veterans with a mental health diagnosis, the V21 model under-estimated the cost of care by $2,314 per patient.


  • Risk scores may need to be developed based on a broader set of hospital data. Without such efforts, safety net hospitals, such as VA, may be penalized and patients and policymakers misled.


  • This study relied on data with different coding practices by VA and non-VA hospitals.
  • This study only tested the V21 model. Other approaches that may work for disadvantaged patients need to be assessed.

Dr. Wagner is an HSR&D Research Career Scientist Awardee. Drs. Wagner (Director), Chee, and Jacobs are part of HSR&D's Health Economics Resource Center (HERC); Drs. Wagner and Jacobs also are part of HSR&D's Center for Innovation to Implementation (Ci2i), both in Palo Alto, CA. Drs. Francis and Almenoff are with VA's Office of Reporting, Analytics, Performance, Improvement, and Deployment (RAPID) in Washington, DC.

Wagner T, Almenoff P, Francis J, Jacobs J, and Chee CP. Assessment of the Medicare Advantage Risk Adjustment Model for Measuring Veterans Affairs Hospital Performance. JAMA Network Open. December 14, 2018;1(8):e185993.

Related Briefs

» next 21 Data Use Briefs...

» next 225 Mental Health Briefs...

What are HSR Publication Briefs?

HSR requires notification by HSR-funded investigators about all articles accepted for publication. These journal articles are reviewed by HSR and publication briefs or summaries are written for a select number of articles that are then forwarded to VHA Central Office leadership to keep them informed about important findings or information. Articles to be summarized are selected by HSR based on timeliness of the findings, interest of leadership, or potential impact on the organization. Publication briefs are written for only a small number of HSR published articles. Visit the HSR citations database for a complete listing of HSR articles and presentations.

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.