skip to page content
Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Go to the ORD website
Go to the QUERI website

HSR&D Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

Statistical methods for MRMC ROC studies.

Skaron A, Li K, Zhou XH. Statistical methods for MRMC ROC studies. Academic Radiology. 2012 Dec 1; 19(12):1499-507.

Related HSR&D Project(s)

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
   Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions


RATIONALE AND OBJECTIVES: In radiology, multireader, multicase (MRMC) receiver-operating characteristic studies are commonly used to evaluate the accuracy of diagnostic imaging systems. The special feature of an MRMC receiver-operating characteristic study requires that the same set of patients' images be examined by the same set of doctors. One main difficulty of analyzing MRMC data is a complicated correlation structure. Four commonly used methods are available for dealing with this complicated correlation structure. The authors conducted an extensive simulation study to assess the performance of these methods in finite sample sizes. They summarize the relative strengths and weaknesses of these methods and make recommendations on the use of these methods. MATERIALS AND METHODS: A comprehensive simulation study was conducted to assess finite-sample performance of these methods with continuous data. The use of these methods for magnetic resonance imaging to predict extracapsular extension of prostate gland tumors is also illustrated. RESULTS: The results indicate that when test outcomes are continuous, all four methods perform well for estimating the difference in areas under the curves for two diagnostic tests. On the basis of these results, it seems that any of these approaches is appropriate for analyzing an MRMC data set with continuous or pseudocontinuous data. CONCLUSIONS: The Dorfman-Berbaum-Metz method is the most practical analysis method to implement in a wide variety of scenarios. Also, in MRMC studies, radiologists should be encouraged to use the entire rating scale rather than tending toward a binary "diseased" or "not diseased" decision.

Questions about the HSR&D website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.