A literature search shows the evidence it can retrieve, not every experiment ever performed. If the chance of a result becoming available depends on what it found, the visible literature can give a misleading impression. In a small field, each missing study may represent a substantial part of the evidence base.
Two ways evidence can be missing
Cochrane distinguishes missing studies from missing results within known studies. A whole experiment may remain unavailable, or a published study may omit an outcome relevant to a synthesis. Bias arises when that availability is related to the result, so the observed set no longer represents the evidence you intended to assess.Cochrane missing-evidence guidance (opens in a new tab)
| Gap | Example question |
|---|---|
| An unlocated study | Does a registry, protocol or conference record identify work without a full result? |
| An unreported outcome | Does the methods section name a measurement absent from the results? |
| An incomplete result | Are the numbers needed for interpretation unavailable? |
| A repeated report | Do several papers describe the same underlying experiment? |
These possibilities are prompts for checking, not proof of misconduct. A missing detail can have several explanations. Record what is observable before deciding how it affects confidence in the evidence.
Why a small evidence base is sensitive
Consider an original hypothetical field with three visible studies, each reporting a favourable result. If you later locate two additional studies with different findings, the description of the evidence changes substantially. The original three papers have not necessarily become false; the set available for interpretation has changed.
This example does not assume that every small field has exactly two hidden negative studies. It illustrates why counting published positive papers alone cannot establish the balance of all research. The missing part is unknown unless you find information about it.
Small numbers also make patterns harder to interpret. A striking result from one research group may dominate the available literature. Several publications may involve similar methods, overlapping samples or the same underlying experiment, so paper count should not be treated automatically as independent replication.
Look beyond the first set of papers
Cochrane recommends considering sources that can reveal unavailable or selectively reported evidence, including study registrations and protocols where relevant. Comparing planned outcomes with reported outcomes can make some gaps visible, although it cannot reveal every unreported experiment.Cochrane missing-evidence guidance (opens in a new tab)
For a practical reading task, keep a record of conference abstracts, preprints, protocols and later full papers, and note relationships between them. Do not give each record the status of a separate completed study. Equally, do not remove a relevant record merely because the full report is difficult to obtain.
If a published paper says additional data are available elsewhere, follow that reference when possible. State when it could not be accessed. A transparent access limit is more useful than an implied claim that the complete evidence base was inspected.
Do not turn suspicion into a numerical correction
You cannot recover an unknown body of results just by imagining how many studies might be missing. A sensitivity analysis can explore assumptions, but those assumed values must remain labelled as assumptions. They are not newly discovered observations.
Similarly, possible publication bias does not establish that every favourable result is wrong. The appropriate conclusion may be reduced confidence, a need for additional evidence or an explicitly qualified synthesis. Keep the strength of that conclusion aligned with what you actually found.
Describe the evidence base honestly
- Count independent studies separately from publications.
- Record known protocols, registrations and incomplete reports.
- Identify outcomes mentioned but not fully reported.
- Explain search and access limitations.
- Distinguish observed gaps from hypothetical missing evidence.
A useful summary might say: “Three studies were located; two related abstracts could not be linked to full reports, and completeness remains uncertain.” That description gives a reader a concrete reason for caution without inventing the direction or size of unobserved results.
Sources and further detail
- Cochrane Handbook — Bias due to missing evidence (opens in a new tab)
Chapter 13. Used for missing-study/missing-result distinctions and ways of investigating availability. Hypothetical study counts in this article are not findings about a peptide field.
Sources checked 19 September 2026. Worked examples are illustrative unless a supplied report is explicitly identified. This article has not undergone independent scientific peer review.