Multi-Hop Scientific Research Synthesis Agent Quality Gate

Role: Lead Research Synthesis Auditor specializing in bibliometrics, epistemic conflict resolution, and evidence-based meta-analysis validation.

Context

• Target research domain: {{target_research_domain}}
• Contradiction resolution protocol: {{contradiction_resolution_protocol}}
• Source qualification tiering: {{source_tier_criteria}}
• Synthesis depth threshold: {{synthesis_depth_threshold}}
• Claim attribution metric: {{claim_attribution_metric}}
• Ingestion and retrieval pipeline: {{retrieval_agent_pipeline}}

Task

Produce an operational evaluation checklist that measures the evidence ground truth, cross-study synthesis coherence, and bibliographic integrity of an agent conducting multi-hop research analysis in {{target_research_domain}}.

Method

1. Map out validation gates for raw source parsing from {{retrieval_agent_pipeline}} against {{source_tier_criteria}}.
2. Define checks to detect source attribution degradation across multi-hop reasoning leaps.
3. Construct evaluation criteria for statistical interpretation (e.g., p-values, sample sizes, confidence intervals, effect sizes).
4. Specify deterministic tests for how the agent identifies and handles conflicting study findings via {{contradiction_resolution_protocol}}.
5. Design rigorous checks to uncover extrapolation bias, over-generalization, and unstated causal claims.
6. Formulate precise metrics based on {{claim_attribution_metric}} to catch hallucinated references and misattributed quotes.
7. Establish validation checks for synthesis depth up to {{synthesis_depth_threshold}} across diverse empirical methodologies.
8. Build summary integrity checks to guarantee non-omission of negative results or dissenting literature.

Constraints

• MUST verify that every synthesized claim contains a verifiable pointer to an accepted source document.
• MUST NOT accept vague summaries; each criterion must enforce explicit quantification of empirical findings.
• Checklist items MUST evaluate both micro-level claim accuracy and macro-level thematic synthesis.
• Must enforce strict disqualification for fabricated DOIs, authors, or statistical outputs.

Output format

A comprehensive quality assurance checklist divided into:

• Part I: Source Provenance and Ingestion Integrity (4-5 checklist items)
• Part II: Statistical Extraction & Multi-Hop Attribution (5-6 checklist items)
• Part III: Contradiction Reconciliation & Epistemic Balance (4-5 checklist items)
• Part IV: Structural Coherence & Over-Claim Prevention (4-5 checklist items)
• Quality threshold summary specifying mandatory blockers vs. warnings.

Self-review

• Verify that the checklist tests multi-source synthesis, not just single-document extraction.
• Ensure all variables, including {{contradiction_resolution_protocol}}, are directly applied.
• Check that citation fidelity checks explicitly prevent subtle author-year mismatches.