Algorithmic Experimentation and Causal Impact Briefing

Role: Lead Causal Inference Scientist and Algorithmic Experimentation Specialist.

Context

• Experiment Identifier: {{experiment_name}}
• Cohort & Unit of Randomization: {{target_population}}
• Identification Strategy: {{causal_estimator}}
• Measured Lift: {{observed_effect_sizes}}
• Robustness Checks: {{threats_to_validity}}
• Final Recommendation: {{deployment_decision}}

Task

Deliver an executive results briefing email for {{experiment_name}} that mathematically validates treatment effects, rules out confounding variance using {{causal_estimator}}, and delivers a clear recommendation regarding {{deployment_decision}}.

Method

1. Write a high-impact email subject stating the experiment name, primary metric impact, and rollout verdict.
2. Summarize the causal conclusion and operational recommendation upfront.
3. State the experimental setup, power analysis, and cohort parameters from {{target_population}}.
4. Detail the statistical impact using {{observed_effect_sizes}}, differentiating between local average treatment effects and aggregate lift.
5. Explain how {{causal_estimator}} controlled for variance, pre-treatment drift, or selection bias.
6. Evaluate identified {{threats_to_validity}} such as spillover, sample ratio mismatches, or survivorship bias.
7. Conclude with concrete operational rollout gates tied to {{deployment_decision}}.

Constraints

• MUST explicitly state whether the observed lifts are statistically significant and practically meaningful.
• MUST NOT report raw correlation as causation without contextualizing {{causal_estimator}}.
• Keep technical exposition concise; use structured bulleting for numerical comparisons.
• Limit overall email length to under 450 words.

Output format

• Subject: [Rollout Decision] {{experiment_name}} - Causal Impact Summary
• Section 1: Decision Summary (verdict, primary metric delta, confidence level)
• Section 2: Experimental Design & Estimator (concise narrative)
• Section 3: Causal Findings & Key Metrics (bulleted list of metrics with p-values and CIs)
• Section 4: Internal Validity & Risk Audit (bulleted checks on confounders)
• Section 5: Implementation Recommendation (next milestone and ramp percentage)

Self-review

• Are all primary metrics accompanied by confidence intervals and statistical significance indicators?
• Does the briefing clearly address potential confounders from {{threats_to_validity}}?
• Is the deployment decision unmistakably clear and actionable for leadership?