01What probability is being calculated?

NIST defines the p-value in terms of obtaining a test statistic at least as extreme as the observed value when the null hypothesis is true. The calculation also relies on the statistical model's assumptions. It does not reverse the question to calculate the probability that the null hypothesis is true after seeing the data.NIST p-value definition (opens in a new tab)

For an original illustration, assume a fair coin and ten independent tosses. The probability of ten heads is 1/1,024. A two-sided test using outcomes equally or more extreme in either direction includes ten tails as well, giving 2/1,024, about 0.00195. This is a probability of specified data patterns under the fair-coin model, not the probability that the coin is fair.

The example is deliberately simple. Real study tests may involve different statistics, assumptions and definitions of extremeness. Before interpreting a reported value, identify which test was actually used.

02It is not the probability that the finding is real

The American Statistical Association warns that p-values do not give the probability that a hypothesis is true or that the data were produced by chance alone. A small value can indicate incompatibility with a specified model; it does not, by itself, identify which alternative explanation is correct.ASA interpretation principles (opens in a new tab)

In the coin example, the unusual sequence might prompt you to question the fairness assumption. It could also prompt checks of independence, recording or the way the experiment was selected for attention. The calculation does not perform those checks for you.

03Significance is not effect size

The ASA also distinguishes a p-value from the magnitude or importance of an effect and advises against basing a scientific conclusion only on whether a threshold is crossed. A numerical estimate and its uncertainty are needed to understand what the observed difference might mean.ASA interpretation principles (opens in a new tab)

As an original reading exercise, imagine one report describing a very small measured change precisely and another describing a larger change imprecisely. A smaller p-value in the first report would not, on its own, establish the more important biological result. You still need the outcome's scale and the study context.

Likewise, p = 0.049 and p = 0.051 fall on opposite sides of a common threshold but are close numerically. Treating them as complete proof and complete absence of an effect would exaggerate what that boundary can tell you.

04Check how the result was selected

Ask whether the comparison was specified before examining the data, how many outcomes or analyses were considered and whether the reported result represents the whole analysis. A single selected number can hide a much larger set of attempted comparisons. That context belongs beside the p-value.

For example, a sentence that reports one favourable endpoint without mentioning several other measured endpoints leaves the reader unable to assess the selection process. The solution is to find the methods, protocol where available and complete results, rather than interpreting the isolated number more confidently.

A result above the chosen threshold also deserves careful wording. “Did not meet the stated significance criterion” is not automatically equivalent to “proved no effect”. Consider which effect sizes remain compatible with the estimate and uncertainty before drawing a stronger conclusion.

05A useful reading checklist

• ✓What null hypothesis and statistical test were used?
• ✓What assumptions make the calculation meaningful?
• ✓What is the effect estimate, in which units, and how uncertain is it?
• ✓How many outcomes or comparisons were considered?
• ✓Does the conclusion stay within the study's design and model?

You do not need to dismiss a p-value to read it properly. Give it its specific role, then combine it with the rest of the evidence. That produces a more informative account than a binary significant/not-significant label.