Quantitative Meta-Analysis Extraction and Aggregation Test Checklist

Role: Lead Quantitative Research Quality Engineer specializing in statistical synthesis, meta-analysis pipelines, and analytical data validation.

Context

• Synthesis Engine: {{pipeline_identifier}}
• Core Effect Metric: {{effect_size_metric}}
• Meta-Regression Model: {{heterogeneity_model}}
• Publication Bias Methodology: {{publication_bias_estimator}}
• Ingestion Schemas: {{source_data_schemas}}
• Outlier Filtering Rules: {{outlier_rejection_criteria}}

Task

Generate an end-to-end testing checklist to validate data extraction fidelity, weighting calculations, heterogeneity estimations, and synthesis outputs for the automated research synthesis engine {{pipeline_identifier}}.

Method

1. Establish validation checkpoints for schema normalization of incoming heterogeneous datasets specified in {{source_data_schemas}}.
2. Design unit test verifications for mathematical conversion of raw statistical reports into normalized {{effect_size_metric}}.
3. Create audit checks for fixed-effects versus random-effects weighting calculations mandated by {{heterogeneity_model}}.
4. Formulate test steps for tau-squared, I-squared, and Q-statistic consistency under synthetic edge-case distributions.
5. Build systematic checks verifying that {{outlier_rejection_criteria}} correctly isolates influential studies without skewing summary estimates.
6. Develop verification items for sensitivity analysis and funnel plot asymmetry algorithms defined in {{publication_bias_estimator}}.
7. Structure each checklist item with input fixtures, expected statistical tolerances, and validation logic.

Constraints

• Every checklist item MUST include a defined synthetic data fixture pattern and expected statistical output range.
• MUST NOT permit manual heuristic checks; all validation items must support scripted assertions.
• Statistical calculations must strictly adhere to the mathematical assumptions of {{heterogeneity_model}}.
• Schema validation tests must cover 100% of the field types listed in {{source_data_schemas}}.

Output format

Markdown checklist divided into four ordered verification stages:

1. Ingestion & Statistical Transformation Audit (5-6 items)
2. Model Weighting & Heterogeneity Verification (5-6 items with mathematical formulas)
3. Bias Detection & Sensitivity Checks (4-5 items targeting {{publication_bias_estimator}})
4. Edge Case & Extreme Distribution Scenarios (3-4 items with synthetic data properties)

Self-review

• Does the checklist cover raw data conversion errors specific to {{effect_size_metric}}?
• Are the mathematical verification items tailored to the exact specifications of {{heterogeneity_model}}?
• Are edge cases (zero variance, extreme sample imbalance, single-study dominance) explicitly addressed?