Public Health Survey Sampling Frame and Calibration Weighting Verification Checklist

Role: Principal Survey Statistician for Municipal and Public Health Agencies

Context

• Population Universe: {{target_population_definition}}
• Registry Frame: {{sampling_frame_source}}
• Strata & PSU Definitions: {{stratification_variables}}
• Non-Response Profile: {{expected_non_response_rate}}
• Calibration Targets: {{post_stratification_benchmarks}}
• Variance Engine: {{variance_estimation_method}}

Task

Deliver an end-to-end technical audit checklist to verify sample design integrity, non-response bias adjustments, and post-stratification weighting before producing official public health statistics.

Method

1. Inspect coverage errors and under-coverage bias between {{sampling_frame_source}} and {{target_population_definition}}.
2. Verify selection probabilities and base design weights across {{stratification_variables}}.
3. Audit unit non-response adjustments against expected patterns in {{expected_non_response_rate}}.
4. Check convergence tolerances and extreme weight trimming rules during raking against {{post_stratification_benchmarks}}.
5. Evaluate Design Effect (DEFF) and Effective Sample Size (n_eff) to detect excessive weight variance.
6. Validate replicate weights or linearization formulas configured for {{variance_estimation_method}}.
7. Establish cell suppression and coefficient of variation (CV) flagging thresholds for reporting.
8. Verify compliance with public sector statistical standards for data privacy and disclosure control.

Constraints

• Checklist MUST contain specific threshold guidance (e.g., max weight ratios, CV cutoff limits).
• MUST NOT approve final survey weights without checking trimming thresholds and raking convergence.
• All items MUST reflect complex survey design principles rather than simple random sampling assumptions.
• Use unambiguous pass/warning/fail criteria for each verification step.

Output format

• Markdown checklist categorized into distinct audit stages.
• Part 1: Frame Coverage & PSU Verification (4-5 checklist items)
• Part 2: Non-Response & Attrition Adjustments (4-5 checklist items)
• Part 3: Calibration, Raking & Weight Trimming (5-6 checklist items)
• Part 4: Variance Estimation & Reporting Safety (4-5 checklist items)
• Part 5: Sign-Off Governance Table (Task, Metric Checked, Passing Criteria, Analyst Sign-off)

Self-review

1. Does the checklist specifically reference {{variance_estimation_method}} in the standard error validation steps?
2. Are post-stratification adjustments against {{post_stratification_benchmarks}} explicit?
3. Are design effect (DEFF) formulas and weight dispersion checks included?