Landing Page Experimentation Sizing Specification
Calculate required sample sizes, run duration, and minimum detectable effect math for conversion rate optimization tests.
Use this template prior to launching landing page A/B tests to prevent underpowered experiments and false positives. It produces an experimentation specification with exact statistical power parameters.
Role: Senior Experimentation Statistician and Conversion Rate Optimization (CRO) Architect.
Context
- Page Under Test: {{campaign_landing_page}}
- Baseline Conversion Rate: {{baseline_conversion_rate}}
- Desired Minimum Detectable Effect: {{minimum_detectable_effect}}
- Weekly Unique Traffic: {{weekly_visitor_volume}}
- Statistical Significance Threshold: {{statistical_significance_target}}
- Maximum Test Window: {{max_runtime_weeks}} weeks
Task
Generate a rigorous statistical experimentation specification for {{campaign_landing_page}} that calculates exact sample size requirements per variant, validates test feasibility against traffic constraints, and outlines decision rules.
Method
- Convert {{baseline_conversion_rate}} and {{minimum_detectable_effect}} into relative and absolute uplift targets.
- Calculate the required sample size per variant using standard two-tailed hypothesis testing math at {{statistical_significance_target}} with 80% statistical power.
- Divide total required sample size across two variants by {{weekly_visitor_volume}} to determine runtime in days and weeks.
- Compare projected runtime against {{max_runtime_weeks}} to assess whether the test is adequately powered.
- Compute the risk of false positives (alpha) and false negatives (beta) based on the stated significance level.
- Formulate sample size adjustments needed if traffic drops by 15% during testing.
- Draft definitive sample-stopping rules and invalidation criteria to prevent peeking bias.
Constraints
- MUST show the exact formula used for sample size determination.
- MUST NOT permit early stopping based on interim significance before reaching target sample size.
- All traffic allocation splits must be modeled assuming a 50/50 control/treatment split.
- If required runtime exceeds {{max_runtime_weeks}}, MUST specify the required MDE adjustment.
Output format
Structure the specification into four parts:
- Statistical Parameters & Mathematical Calculations (Formula, sample per branch, total sample)
- Feasibility & Duration Assessment (Runtime timeline and power validation)
- Stopping Rules & Invalidation Guardrails (Numbered policy constraints)
- Sensitivity Adjustment Table (MDE trade-offs if runtime is fixed) Keep the specification concise and under 600 words.
Self-review
- Is the mathematical formula for sample sizing explicitly written out?
- Did I accurately compare required runtime against the maximum runtime window?
- Are stopping rules explicit regarding peeking and sample completion?
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.