Compare results across studies without comparing p-values
Assess agreement between studies by estimating the difference in effects rather than by sorting p-values around a threshold.
A cross-study comparison with an explicit difference estimate, heterogeneity assessment, and a conflict-versus-imprecision verdict.
Role
meta-analytic reviewer
Task
Two or more studies of {{research_question}} report {{results_summary}}. Assess whether their results actually conflict: compute or specify the estimate and interval for the difference between effects, explain why comparing the individual p-values cannot answer the question, and state what combined evidence the set of studies provides.
Context
A press summary claims the studies contradict each other because one crossed the significance threshold and the other did not.
Inputs
- Per-study point estimates, standard errors, and sample sizes
- Outcome and exposure definitions per study
- Population and design differences: {{design_differences}}
Constraints
- Report the difference estimate with its interval, not a verdict from two p-values
- Note when identical p-values accompany clearly different effects, and the reverse
- Flag design or population differences that make pooling inappropriate
- State whether individually non-significant studies could be jointly informative
Output Format
A comparison table of study estimates, an explicit difference/heterogeneity analysis, and a plain-language verdict on conflict versus agreement.
Quality Criteria
- Agreement judged on the effect scale, not the p-value scale
- Heterogeneity analysis specified concretely
- Pooling caveats explicit
- Verdict distinguishes 'conflicting' from 'imprecise'