Two fictional results
Imagine that a difference of five points on a particular scale would matter for a defined decision. One study estimates a difference of zero, with an interval from minus twelve to plus twelve. Another estimates zero, with an interval from minus one to plus one.
Both point estimates are zero. The first interval remains compatible with differences much larger than five points in either direction. The second is much narrower. Subject to the design and analysis being appropriate, it provides a stronger basis for ruling out differences of that size.
The five-point boundary is an assumption in this illustration, not a universal threshold. In real work, a meaningful difference needs justification before it becomes a basis for interpretation.
What “not significant” leaves out
The phrase does not tell you how large the estimated effect was, what range remains compatible with the data, or whether the study was designed to test equivalence. A small study with noisy measurements and a large, precise study can both produce a p-value above a conventional cutoff.
Cochrane's interpretation guidance encourages attention to estimates and uncertainty rather than treating statistical significance as a complete verdict. That distinction is especially useful when a headline says a treatment “doesn't work.”
Language matched to the result
- Wide uncertainty: “The study did not resolve whether a meaningful difference exists.”
- Narrow uncertainty around a small effect: “The results argue against a difference as large as the specified threshold.”
- A formal equivalence study: describe its prespecified margin and whether the interval met that criterion.
None of these statements turns a flawed design into reliable evidence. Precision concerns random uncertainty; it does not remove systematic error. A careful conclusion needs both the interval and a credible comparison.
