Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Go to the ORD website
Go to the QUERI website

VA Statisticians' Association

Interpretation of P-values: Challenges for the Replication and Comparison of Statistical Results


by Ilana Belitskaya-Levy, PhD
Seminar date: 10/29/2013

Description: Large-scale genome-wide association studies rely on extremely small p-values, on the order of 10-8, to achieve positive findings. Little attention has been given to the implications of this shift from the traditional 0.05 significance level on the likely success of such studies. In this seminar, we address unappreciated properties of extreme p-values that may contribute to the failure to replicate many findings from genome-wide association studies. Specifically, we address the following questions. (1) How variable is the sampling distribution of a p-value? (2) Given a SNP with a specific, observed p-value, what is likely to happen for the same SNP in an independent replication study? (3) Given two observed p-values for different SNPs from the same or different studies, how certain can we be that the effect of one SNP is greater than the effect of the other? We obtain explicit numerical results showing that even extremely small p-values provide surprisingly little information as to the likelihood of subsequent replication or the relative importance of the underlying associations. While we frame our findings in terms of GWAS and SNPs, they are valid for p-values produced by most large-sample statistical tests applied to any number of hypothesis tests. Our results suggest that unjustified over-interpretation of very small initial p¬-values are an important factor contributing to the frequency of subsequent non-replications. The target audience would be mostly statisticians and researchers who work with data, especially those working with high-dimensional or genetic data. The level is not basic, but somewhat advanced. VASA Website: http://www.hsrd.research.va.gov/for_researchers/sig/vasa/

We're sorry - this video format is no longer supported. You may still download the resources below.



DOWNLOAD: Request PDF Handout | Audio only (mp3) | transcript


Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.