01Read power as a conditional probability

NC3Rs defines power as the probability of rejecting a false null hypothesis and describes its dependence on effect size, variability, significance level and sample size for a specified test. Those inputs are essential to interpreting the reported percentage.NC3Rs EDA — Group and sample size (opens in a new tab)

In an original illustration, a design has 80% power for a defined effect under its assumed model. Across many repetitions in which that effect and those assumptions hold, the test would detect it in about 80% of experiments. The remaining experiments could miss it.

This does not mean that a significant result from the study has an 80% probability of being true. It also does not mean the test has 80% power for every possible effect, outcome or subgroup.

02Distinguish few units from little useful information

All else equal, fewer independent units commonly reduce sensitivity to a specified effect. But sample size alone does not determine power: an outcome with large variation presents a different problem from one measured with little variation.

The relevant count is the number of experimental units contributing to the comparison. Repeating an instrument reading many times may improve measurement of existing units without supplying the additional biological information assumed by a larger unit count.

A small study may still identify a large, consistent effect or expose a practical measurement problem. It should not be dismissed solely because its sample is small. The question is which conclusions the design and resulting uncertainty can support.

03Check the assumptions rather than accepting a target number

A statement that the study was powered at 80% is incomplete without the planned effect and analysis. A design can reach that target for a large difference while having much less sensitivity to a smaller difference that would still matter.

Look for the rationale behind the selected effect, the source of the variability estimate and any allowance for missing units. Assumptions borrowed from a different model or a very uncertain pilot deserve explanation.

Power for a primary outcome does not automatically transfer to every secondary measurement. Different outcomes can vary differently, and an interaction or subgroup comparison can require different information from the main comparison.

A useful sensitivity analysis considers several plausible assumptions rather than implying that one estimated input is known exactly. The reader can then see which design conclusions are robust and which depend strongly on an uncertain quantity.

04Use the observed estimate to interpret the completed study

NC3Rs cautions against interpreting a completed experiment through observed power calculated using its own observed effect. That calculation is directly related to the observed significance result and does not add independent information.NC3Rs EDA — Group and sample size (opens in a new tab)

Instead, inspect the estimated effect and its interval. Which sizes and directions remain unresolved? Does the evidence rule out a difference that matters, or is the study simply too imprecise to answer that question clearly?

An original reading comparison illustrates the distinction: two studies may both miss a significance threshold, while one gives a narrow interval around a negligible change and the other gives a broad interval extending to consequential effects. Their implications differ despite the shared label.

Summarise planned sensitivity separately from achieved evidence. The former describes an expectation under assumptions; the latter comes from the observed data, their uncertainty and the actual execution of the experiment.