Simultaneous evaluation of the imprecision and inconsistency domains of GRADE can be performed using prediction intervals.

Murad MH, Morgan RL, Falck-Ytter Y, Mustafa RA, Sultan S, Dahm P, Siedler MR, Altayar O, Davitkov P, Naqvi SAA, Riaz IB, Wang Z, Lin L. Simultaneous evaluation of the imprecision and inconsistency domains of GRADE can be performed using prediction intervals. Journal of clinical epidemiology. 2024 Nov 1; 175:111543.

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

OBJECTIVES: To explore the use of prediction interval (PI) for the simultaneous evaluation of the imprecision and inconsistency domains of Grading of Recommendations, Assessment, and Evaluation using stakeholder-provided decision thresholds. STUDY DESIGN AND SETTING: We propose transforming the PI of a meta-analysis from a relative risk scale to an absolute risk difference using an appropriate baseline risk. The transformed PI is compared to stakeholder-provided thresholds on an absolute scale. We applied this approach to a large convenience sample of meta-analyses extracted from the Cochrane Database of Systematic Reviews and compared it against the traditional approach of rating imprecision and inconsistency separately using confidence intervals and statistical measures of heterogeneity, respectively. We used empirically derived thresholds following Grading of Recommendations, Assessment, and Evaluation guidance. RESULTS: The convenience sample consisted of 2516 meta-analyses (median of 7 studies per meta-analysis; interquartile range: 5-11). The main analysis showed the percentage of meta-analyses in which both approaches had the same number of certainty levels rated down was 59%. The PI approach led to more levels of rating down (lower certainty) in 27% and to fewer levels of rating down (higher certainty) in 14%. Multiple sensitivity analyses using different thresholds showed similar results, but the PI approach had particularly increased width with a larger number of included studies and higher I values. CONCLUSION: Using the PI for simultaneous evaluation of imprecision and inconsistency seems feasible and logical but can lead to lower certainty ratings. The PI-based approach requires further testing in future systematic reviews and guidelines using context-specific thresholds and evidence-to-decision criteria.