Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website
Publication Briefs

Study Examines Relationship between Diagnostic Accuracy and Confidence Using Case Study Vignettes


BACKGROUND:
Diagnostic errors can lead to patient harm, but the factors contributing to diagnostic errors have not been explored as thoroughly as other patient safety concerns. While physician overconfidence has been identified as a potential contributor to diagnostic error, little is known about the relationship between physicians' diagnostic accuracy and their confidence (defined as diagnostic calibration) or how that relationship might change with the evolution of diagnosis, such as through patient history, examination, and testing. Even less is known about how confidence and diagnostic accuracy relate to resource use (e.g., specialized tests, referrals, second opinions). This case vignette-based study evaluated diagnostic accuracy, confidence, and diagnostic calibration during the evolution of the diagnostic process. It also evaluated how resource use might be related to those factors. Investigators recruited 118 general internists from an online physician community (www.QuantiaMD.com) and asked them to diagnose four previously validated case vignettes of variable difficulty (two "easier"; two "more difficult"). Cases were presented in a web-based format. Investigators assessed physicians' diagnostic accuracy, confidence in diagnostic accuracy, as well as diagnostic calibration and whether they would use any additional resources to diagnose the case. Diagnostic accuracy was scored by comparing participating physicians' responses to the true diagnoses provided by case study authors.

FINDINGS:

  • Physicians correctly diagnosed 55% of the easier and 6% of the more difficult cases.
  • Physicians' confidence averaged 7 out of 10 (10 being highest confidence) for easier cases and 6 for more difficult cases. The small decrease in confidence between easier and more difficult cases was unlikely to be practically or clinically meaningful in the face of a much larger drop in accuracy.
  • Physicians' level of overconfidence increased with difficult cases (diagnostic calibration was worse).
  • Although confidence increased as cases evolved through the diagnostic process and more data became available, diagnostic accuracy did not.
  • Fewer diagnostic tests were requested when confidence was high, even when accuracy was low.

LIMITATIONS:

  • Findings from an artifical diagnostic environment may not translate to real-world settings.
  • Some physicians may have been unwilling to disclose diagnostic uncertainty.

IMPLICATIONS:

  • If confirmed in real clinical settings, findings provide evidence needed to develop interventions to improve physicians' diagnostic calibration and resource use, and related diagnostic outcomes.

AUTHOR/FUNDING INFORMATION:
Drs. Meyer, Payne, Meeks, and Singh are part of HSR&D's Houston Center for Quality of Care and Utilization Studies.


PubMed Logo Meyer AND, Payne VL, Meeks DW, Rao R, and Singh H. Physicians’ Diagnostic Accuracy, Confidence, and Resource Requests: A Vignette Study. JAMA Internal Medicine August 26, 2013;e-pub ahead of print.

Related Briefs

» next 8 Provider Education Briefs...


What are HSR Publication Briefs?

HSR requires notification by HSR-funded investigators about all articles accepted for publication. These journal articles are reviewed by HSR and publication briefs or summaries are written for a select number of articles that are then forwarded to VHA Central Office leadership to keep them informed about important findings or information. Articles to be summarized are selected by HSR based on timeliness of the findings, interest of leadership, or potential impact on the organization. Publication briefs are written for only a small number of HSR published articles. Visit the HSR citations database for a complete listing of HSR articles and presentations.


Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.