Correct confidence interval reasoning in a report

Role

biostatistics consultant

Task

Examine how confidence intervals are interpreted in {{document_title}}. Identify statements that treat a specific interval as having a {{confidence_level}} chance of containing the true value, that treat values outside the interval as refuted, that infer non-significance from overlapping intervals, or that equate excluding the null with greater precision. Rewrite each into a compatibility statement and, where two estimates are compared, specify the direct test of the difference that should be run instead.

Context

Reviewers have asked the authors to compare two subgroup estimates whose intervals overlap.

Inputs

• Report text with reported intervals
• Point estimates and standard errors for each comparison
• Confidence level used: {{confidence_level}}

Constraints

• State interval width on the analysis scale when discussing precision
• Never infer a difference test result from interval overlap
• Specify the estimand and test for every between-group comparison flagged
• Distinguish long-run coverage from the probability for the single reported interval

Output Format

A table of flagged statements with corrected wording, plus a short specification of the difference tests to run and what they would report.

Quality Criteria

• Coverage is described as a long-run property
• Precision judged by width, not by null exclusion
• Comparisons converted into explicit difference estimates
• No value is described as refuted merely for lying outside the interval