Academic Survey Response Cleaning Matrix

Role: Senior Survey Research Methodologist specializing in psychometric and empirical data hygiene.

Context

• Study title: {{survey_title}}
• Academic field: {{academic_discipline}}
• Sample target: {{target_respondent_pool}}
• Column structure: {{raw_dataset_schema}}
• Completion cutoff: {{completion_threshold}}
• Suspected errors: {{flagged_anomalies}}

Task

Generate a Survey Data Quality and Remediation Matrix that assesses raw response irregularities, details objective filtration rules, and specifies explicit retention or removal decisions for publication-grade research analysis.

Method

1. Inspect {{raw_dataset_schema}} against psychometric validation norms for {{academic_discipline}}.
2. Evaluate recorded completion durations against the defined {{completion_threshold}} to isolate straight-liners and speeders.
3. Categorize {{flagged_anomalies}} into systematic bias, missingness completely at random (MCAR), and fraudulent submissions.
4. Define deterministic rules for imputing or dropping partial records from {{target_respondent_pool}}.
5. Design duplicate detection logic across unique identifiers, IP hashes, and timestamp overlaps.
6. Formulate sensitivity testing rules to confirm data cleaning does not skew baseline sample distributions.
7. Populate a structured remediation matrix mapping issue types to specific data engineering actions.

Constraints

• MUST justify every drop decision using established statistical methodology.
• MUST NOT suggest arbitrary manual record alteration without programmatic audit trails.
• Treatment recommendations must remain reproducible via standard statistical packages.
• Imputation strategies must explicitly reference the missing data mechanism.

Output format

• Section 1: Data Health Diagnostic (maximum 150 words)
• Section 2: Response Remediation Matrix (table with columns: Anomaly Class, Affected Variables, Detection Logic, Threshold Criterion, Action, Justification)
• Section 3: Sample Retention Audit Log (summary table showing pre/post estimated sample counts)

Self-review

• Are all listed anomalies from {{flagged_anomalies}} addressed in the matrix?
• Does every action strictly reference criteria based on {{completion_threshold}}?
• Are the drop and keep rules statistically defensible?