Nonprofit Quasi-Experimental Program Evaluation Statistical Readiness Checklist

Role: Senior Econometrician and Nonprofit Impact Evaluation Lead

Context

• Evaluation Target: {{target_intervention_name}}
• Control Baseline: {{control_group_source}}
• Primary Metrics: {{key_outcome_indicators}}
• Sample Dimensions: {{sample_size_per_arm}}
• Identification Strategy: {{statistical_matching_technique}}
• Pre-treatment Controls: {{baseline_confounders}}

Task

Produce an exhaustive, pre-flight statistical readiness checklist to validate identification strategy assumptions, data balance, and statistical power for evaluating {{target_intervention_name}} against {{control_group_source}}.

Method

1. Formulate exact null and alternative hypotheses for {{key_outcome_indicators}}.
2. Review minimum detectable effect size calculations given {{sample_size_per_arm}}.
3. Audit common support regions and covariate balance thresholds across {{baseline_confounders}}.
4. Define diagnostic checks for {{statistical_matching_technique}} (e.g., standardized mean differences < 0.10).
5. Specify unobserved confounding sensitivity parameters (e.g., Rosenbaum bounds or Oster ratio tests).
6. Establish cluster and robust standard error calculation rules for administrative units.
7. Detail falsification tests, placebo outcomes, and pre-trend parallel trajectory checks.
8. Outline clear pass/fail operational criteria for moving from exploratory modeling to confirmatory reporting.

Constraints

• Checklists MUST group items into phase-ordered categories: Pre-estimation, Balance Diagnostics, Sensitivity & Falsification, and Reporting.
• You MUST assign an explicit verification method and failure trigger to every single item.
• MUST NOT recommend randomized trial steps when validating {{statistical_matching_technique}}.
• Keep instructions actionable for public sector and nonprofit research teams.

Output format

• Markdown checklist structure.
• Section 1: Pre-Analysis and Power Validation (4-6 actionable check items)
• Section 2: Identification and Balance Diagnostics (5-7 actionable check items)
• Section 3: Robustness, Sensitivity, and Falsification (4-6 actionable check items)
• Section 4: Sign-off Protocol (summary matrix with Status, Responsible Role, and Remediation Strategy)

Self-review

1. Are all listed {{baseline_confounders}} explicitly addressed in the balance diagnostics section?
2. Does every checklist item contain a testable quantitative or qualitative threshold?
3. Are sensitivity tests suitable specifically for {{statistical_matching_technique}}?