Prevention & care
Certainty and uncertainty: confidence intervals, meaningful effects and evidence appraisal
On this page
Confidence in a body of evidence is different from the precision of one estimate. GRADE distinguishes evidence certainty from recommendation strength and considers more than study design. A statistical result, a certainty judgment and a decision are related steps rather than interchangeable labels.[3]
Separate the statistical summary from the interpretation question
| Concept | What it describes | What it does not establish alone |
|---|---|---|
| Confidence interval | Interval estimate under specified assumptions | A personal outcome or posterior probability[1] |
| P value | Data compatibility with a specified statistical model | Probability that a hypothesis is true[2] |
| Effect importance | Whether an effect matters for the question | A consequence of statistical significance alone[2] |
| Evidence certainty | Confidence in a relevant body of evidence | The strength of a recommendation by itself[3] |
| Imprecision appraisal | Interval relative to a threshold of interest | One automatic verdict from crossing the null[4] |
A confidence interval has a repeated-study interpretation
Greenland and colleagues explain that the usual frequentist confidence level describes the interval procedure over hypothetical repetitions under its assumptions. It is not a 95% posterior probability assigned to the particular reported interval. That different probability interpretation needs additional Bayesian modeling assumptions. The interval also depends on design, analysis and reporting choices; it does not automatically include every source of bias or missing information. Its width and location help describe uncertainty, while its endpoints are not sharp boundaries between possible and impossible effects.[1]
Statistical significance does not measure effect size or importance
The ASA emphasizes context: statistical significance alone cannot establish effect size or importance.[2]
A certainty framework evaluates more than one design label
The original GRADE rating paper describes high, moderate, low and very low categories for a body of evidence. Risk of bias, inconsistency, indirectness, imprecision and publication bias can reduce confidence, while other considerations can increase it. The relevant question and outcomes must be clear. Evidence certainty and recommendation strength are separate judgments. Explaining this framework is not the same as completing its systematic assessment. Atlas guides retain their stated pending review and formal-grading status; a citation to GRADE does not independently grade this article or its sources.[3]
Imprecision depends on the effect question and threshold
The 2022 GRADE guidance on minimally contextualized imprecision emphasizes absolute-effect confidence intervals relative to a threshold of interest, alongside information size in relevant circumstances. An interval can be narrow around an effect too small to matter, or wide enough to remain compatible with importantly different effects. Whether it crosses zero or a ratio of one is not the only consideration. The threshold and context need justification; this guide does not select a universal minimum important benefit or apply a formal certainty downgrade.[4]
Failure to detect a difference is not proof of equivalence
Lakens explains equivalence testing as a different question from a conventional test of no difference. It evaluates whether effects can be bounded within a prespecified range judged small enough for the purpose. Those bounds require justification and sufficient information. A nonsignificant conventional result can arise because the estimate is too imprecise to discriminate among important possibilities. Conversely, evidence for a sufficiently small effect is not a claim that the true effect is exactly zero. The methodological primer is not a clinical rule for choosing treatment-equivalence margins.[5]
Evidence & uncertainty
In experiments about public statistics, numeric ranges had little or no measured effect on trust in the source, while verbal uncertainty could reduce it. These findings concern particular messages and audiences; they do not guarantee trust or good health decisions in every setting.[8]
Selected-source overview · Independent review pendingPublished questions
Does a reported 95% confidence interval give a 95% posterior probability for the true effect?
Selected-source overview · Independent review pending. Read the evidence context.
No. The frequentist confidence level concerns a repeated-study procedure under assumptions. A Bayesian probability interval requires additional assumptions and a different interpretation.[1]
Evidence behind this answer
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
original statistical interpretation essay · 2016
Selected original full-text methods, results and limitations inspected; no independent systematic appraisal · Frequentist interval procedure, posterior distinction, model assumptions and endpoints not absolute exclusions
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Does a small P value measure the probability that the hypothesis is true?
Selected-source overview · Independent review pending. Read the evidence context.
No. The ASA distinguishes model compatibility from hypothesis probability, effect size and practical importance. Context and the full analysis remain necessary.[2]
Evidence behind this answer
- American Statistical Association Releases Statement on Statistical Significance and P-Values
official original statement release · 2016
Original official release and its reproduced six principles inspected; full journal statement not independently inspected · Six principles on model compatibility, hypothesis probability, thresholds, transparency and effect importance
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Does the randomized-trial label automatically complete an evidence-certainty assessment?
Selected-source overview · Independent review pending. Read the evidence context.
No. GRADE considers multiple domains in a body of evidence. Its explanation does not supply a completed Atlas appraisal or recommendation by itself.[3]
Evidence behind this answer
- GRADE guidelines: 3. Rating the quality of evidence
original evidence certainty framework · 2011
Original abstract and bibliographic record inspected; full study text not independently inspected · Four body-of-evidence categories, five downgrading domains, possible upgrading and certainty-versus-recommendation distinction
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Is crossing the no-effect value the only imprecision question?
Selected-source overview · Independent review pending. Read the evidence context.
No. The GRADE update considers intervals relative to relevant thresholds and information size. Effects of different practical importance can remain compatible with the interval.[4]
Evidence behind this answer
- GRADE Guidance 34: update on rating imprecision using a minimally contextualized approach
original updated grade method guidance · 2022
Original abstract and bibliographic record inspected; full study text not independently inspected · Absolute-effect intervals, threshold of interest, information size and contextualized appraisal limits
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Does a nonsignificant difference establish equivalence?
Selected-source overview · Independent review pending. Read the evidence context.
No. Lakens distinguishes equivalence testing using justified bounds from an inconclusive conventional comparison. Failure to detect a difference can reflect inadequate precision.[5]
Evidence behind this answer
- Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
original equivalence methods primer · 2017
Original indexed abstract and selected original indexed passages inspected; direct original page unavailable or challenge blocked; full text not independently inspected · Prespecified justified bounds, conventional nonsignificance versus equivalence and no exact-zero claim
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Did acceptable GRADE wording prove better health outcomes?
Selected-source overview · Independent review pending. Read the evidence context.
No. The project assessed wording with review producers and users. Acceptability is distinct from demonstrated patient understanding, decisions or clinical outcomes.[6]
Evidence behind this answer
- GRADE guidelines 26: informative statements to communicate the findings of systematic reviews of interventions
original guidance development with workshops and survey · 2020
Original abstract and bibliographic record inspected; full study text not independently inspected · Magnitude and certainty language, workshops 80 / survey 110, acceptability versus understanding/clinical outcomes
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Does stating uncertainty inevitably cause people to distrust the source?
Selected-source overview · Independent review pending. Read the evidence context.
No. The five experiments found generally small trust effects, mostly with verbal uncertainty. Their particular messages and audiences do not establish a universal rule.[7]
Evidence behind this answer
- The effects of communicating uncertainty on public trust in facts and numbers
original five experiment communication study · 2020
Original abstract and bibliographic record inspected; full study text not independently inspected · 5780 total, formats/topics/magnitudes, national replication/news field study and generally small trust effects
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
Are trust in a statistic and trust in its source identical outcomes?
Selected-source overview · Independent review pending. Read the evidence context.
No. Kerr’s experiments found different patterns for numeric ranges and verbal uncertainty across these measures. Neither measure alone establishes later behavior or health benefit.[8]
Evidence behind this answer
- The effects of communicating uncertainty around statistics, on public trust
original two survey experiment communication study · 2023
Original abstract and bibliographic record inspected; full study text not independently inspected · 10519 current/2309 projected COVID statistics; number-versus-source trust and numeric-versus-verbal formats
This answer summarizes selected sources. An independent appraisal of the complete evidence has not been completed.
The size of an effect and confidence in it need separate language
The GRADE communication project developed wording that combines effect magnitude with certainty rather than using one adjective for both. Workshops involved 80 attendees, and a survey involved 110 producers and users of systematic reviews. Many proposed statements were acceptable to the surveyed participants. This is evidence about acceptability and wording, not a trial proving improved patient understanding or health outcomes. Such phrases work best when linked to an explicit outcome, comparison and actual appraisal, rather than used to imply a certainty rating that was never performed.[6]
Communicating limits did not cause a large universal loss of trust
Van der Bles and colleagues conducted five experiments involving 5,780 people, including a preregistered national-sample replication and a news-website field experiment. They varied uncertainty around facts across topics, formats and magnitudes. Communicated uncertainty was noticed, while trust reductions were generally small and occurred mostly with verbal uncertainty. Messages, audiences and measured short-term trust differ from individual medical decisions or every real-world dispute. The results challenge a blanket assumption that honesty about uncertainty inevitably destroys trust, without establishing that any uncertainty message is equally effective.[7]
A later study distinguished trust in a number from trust in its source
Kerr and colleagues tested COVID-19 statistics in a survey experiment with 10,519 participants and projected statistics in another with 2,309. Numeric ranges could slightly reduce the perceived trustworthiness of a number while leaving source trust largely unchanged. Unquantified verbal uncertainty could affect both. Those are separate measured outcomes. These context-specific experiments do not show that numeric ranges guarantee public agreement, behavioral change or clinical benefit. They instead support considering what uncertainty was communicated, to whom, and which response was actually measured.[8]
References
- Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Original statistical interpretation essay
Greenland and colleagues · Content/publication date: 2016
Accessed: 2026-10-14 UTC · Selected original full-text methods, results and limitations inspected; no independent systematic appraisal
Cited sections: Frequentist interval procedure, posterior distinction, model assumptions and endpoints not absolute exclusionsSelected original definitions and misconception passages inspected; PMC4877414 alternative challenge recorded. PMID 27209009/PMC4877414/DOI and author repositories are one publication. No universal probability interpretation is assigned to every kind of interval.
- American Statistical Association Releases Statement on Statistical Significance and P-Values
Official original statement release
American Statistical Association · Content/publication date: 2016
Accessed: 2026-10-14 UTC · Original official release and its reproduced six principles inspected; full journal statement not independently inspected
Cited sections: Six principles on model compatibility, hypothesis probability, thresholds, transparency and effect importanceThe official three-page release reproduces the principles; journal DOI 10.1080/00031305.2016.1154108 direct attempt failed. Release/statement aliases share one underlying statement allowance. Not a trial proving one inferential philosophy universally superior. Existing lowercase-PDF attribution and the differently cased accessed URL are the same official statement. New text is compacted within their single cumulative allowance.
- GRADE guidelines: 3. Rating the quality of evidence
Original evidence certainty framework
Balshem and colleagues · Content/publication date: 2011
Accessed: 2026-10-14 UTC · Original abstract and bibliographic record inspected; full study text not independently inspected
Cited sections: Four body-of-evidence categories, five downgrading domains, possible upgrading and certainty-versus-recommendation distinctionOriginal full abstract and bibliographic dates inspected. Online January 5 separate from April 2011 issue. DOI 10.1016/j.jclinepi.2010.07.015 and PMID 21208779 are one paper. No Cochrane Handbook chapter or What If book is cited or given a fresh allowance.
- GRADE Guidance 34: update on rating imprecision using a minimally contextualized approach
Original updated grade method guidance
Zeng and colleagues · Content/publication date: 2022
Accessed: 2026-10-14 UTC · Original abstract and bibliographic record inspected; full study text not independently inspected
Cited sections: Absolute-effect intervals, threshold of interest, information size and contextualized appraisal limitsOriginal full abstract in author-Mayo record inspected; PubMed shell and publisher attempt blocked. October 2022 issue distinct from earlier online release; year precision retained. DOI 10.1016/j.jclinepi.2022.07.014/PMID 35934265 are one paper. Updated guidance replaces reliance on older 2011 imprecision guidance for this discussion; no margin or formal downgrade prescribed.
- Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Original equivalence methods primer
Lakens · Content/publication date: 2017
Accessed: 2026-10-14 UTC · Original indexed abstract and selected original indexed passages inspected; direct original page unavailable or challenge blocked; full text not independently inspected
Cited sections: Prespecified justified bounds, conventional nonsignificance versus equivalence and no exact-zero claimOriginal indexed abstract and methodological passages inspected; publisher original online date verified. PMC5502906 direct challenge, full paper not independently inspected. DOI 10.1177/1948550617697177 and all author routes are one paper. Psychological methods primer, not a clinical equivalence-margin guideline.
- GRADE guidelines 26: informative statements to communicate the findings of systematic reviews of interventions
Original guidance development with workshops and survey
Santesso and colleagues; GRADE Working Group · Content/publication date: 2020
Accessed: 2026-10-14 UTC · Original abstract and bibliographic record inspected; full study text not independently inspected
Cited sections: Magnitude and certainty language, workshops 80 / survey 110, acceptability versus understanding/clinical outcomesOriginal complete abstract in author-university record inspected. March 2020 issue distinct from 2019 online/copyright history; year recorded for cited journal publication without inventing an online day. DOI 10.1016/j.jclinepi.2019.10.014 and PMID 31711912 share one allowance. No verbatim wording template reproduced or Atlas GRADE rating implied.
- The effects of communicating uncertainty on public trust in facts and numbers
Original five experiment communication study
van der Bles, van der Linden, Freeman and Spiegelhalter · Content/publication date: 2020
Accessed: 2026-10-14 UTC · Original abstract and bibliographic record inspected; full study text not independently inspected
Cited sections: 5780 total, formats/topics/magnitudes, national replication/news field study and generally small trust effectsOriginal full abstract and online/issue dates in author-university record inspected; full paper not independently appraised after PMC7149229 challenge. Online March 23 distinct from April 7 issue. DOI 10.1073/pnas.1913678117/PMID 32205438 and author record are one paper; national replication is within this paper, not another cited study.
- The effects of communicating uncertainty around statistics, on public trust
Original two survey experiment communication study
Kerr and colleagues · Content/publication date: 2023
Accessed: 2026-10-14 UTC · Original abstract and bibliographic record inspected; full study text not independently inspected
Cited sections: 10519 current/2309 projected COVID statistics; number-versus-source trust and numeric-versus-verbal formatsOriginal full abstract in author-university record inspected; direct PubMed shell and PMC10663791 challenge. DOI 10.1098/rsos.230604 and PMID 38026007 are one paper. Short-term perceived trust outcomes, not clinical decisions or guaranteed persuasion.
Scope & review
- Population
- Readers interpreting statistical estimates and bodies of health evidence; selected public-information experiment audiences
- Exposure or intervention
- Confidence intervals, P values, effect importance, certainty frameworks, equivalence and communication format
- Comparison
- Specified statistical models/thresholds; verbal or numeric uncertainty versus messages without uncertainty
- Outcomes
- Estimation compatibility, precision, practical importance, evidence confidence, number trust and source trust as distinct outcomes
- Timeframe
- Foundational methods 2011–2017, updated GRADE imprecision 2022 and communication studies 2020/2023; publication and retrieval dates separate
- Jurisdiction
- General methods explanation; no personal prognosis, clinical equivalence margin, formal Atlas GRADE rating or universal trust/persuasion claim
Selected-source educational explanation. Official guidance, original study findings and editorial interpretation are distinguished. No exhaustive search, independent systematic synthesis or qualified medical review has been completed.
Sources checked on their individual access dates. Selected sources were checked; ongoing surveillance and qualified review have separate statuses on the coverage page.
Revision history
- 2026-10-14 UTC: Added a bounded evidence-interpretation explainer; independent clinical review and comprehensive synthesis remain pending.
What this article still leaves open
- Which uncertainty comes from sampling, model assumptions, study bias, indirectness or missing evidence?
- What range of effects remains compatible with the data relative to a justified practical threshold?
- Which communication methods improve understanding and decisions, as distinct from short-term trust measures?
Suggest a correction
Submit a content correction to the editorial queue, or download a local note. Leave out personal medical details. Submitted text is stored privately with a browser receipt session and is visible to this browser and the editor. Clearing cookies removes access to your submission history. This is not an account or a medical service.
Your submitted corrections for this article
Explore the topic map
Browse the health domains connected to this article. The map also identifies topics whose articles remain unwritten.