Scientific Experimental Assay Data Scrubbing Architecture

Role: Lead Biomedical Research Data Scientist and Scientific Informatics Specialist

Context

• Research Consortium: {{research_consortium_name}}
• Assay & Instrument Types: {{clinical_assay_types}}
• Missing Value Thresholds: {{missing_value_imputation_thresholds}}
• Outlier Detection Standard: {{outlier_detection_standards}}
• Reproducibility Baseline: {{reproducibility_target_standards}}
• Data Dictionary Version: {{data_dictionary_version}}

Task

Construct an end-to-end laboratory data cleansing framework that standardizes multi-site raw outputs, manages measurement noise, and produces an analysis-ready dataset compliant with {{reproducibility_target_standards}} for {{research_consortium_name}}.

Method

1. Standardize measurement units, instrument timestamps, and sensor precision across {{clinical_assay_types}}.
2. Execute batch effect diagnostics to distinguish biological variance from technical instrument artifacts.
3. Apply {{outlier_detection_standards}} to flag instrument saturation, optical density clipping, and assay run failures.
4. Implement missing-data triage classifying values as Missing Completely at Random (MCAR) or Missing Not at Random (MNAR).
5. Execute imputation workflows according to {{missing_value_imputation_thresholds}} while tagging all imputed cells with indicator flags.
6. Align raw phenotypic and biochemical variables with standard ontologies defined in {{data_dictionary_version}}.
7. Establish reproducible version-controlled pipeline artifacts matching {{reproducibility_target_standards}}.

Constraints

• MUST explicitly separate raw measurement data from derived, transformed, and imputed variables.
• MUST NOT drop experimental replicates without documenting an explicit instrument error code.
• Outlier handling logic must preserve extreme biological observations while filtering mechanical noise.
• All cleaning scripts must maintain zero dependency on proprietary non-open data formats.

Output format

Deliver the sanitization architecture in the following sequence:

1. Instrument Harmonization & Unit Normalization Plan (tabular protocol).
2. Outlier & Batch Effect Treatment Guidelines (criteria and action mapping).
3. Missingness & Imputation Strategy Matrix (thresholds and statistical methods).
4. Data Cleansing Verification Checklist (step-by-step quality gate before publishing).

Self-review

• Confirm explicit references to {{clinical_assay_types}} and {{missing_value_imputation_thresholds}} are embedded in the method.
• Ensure the format adheres to the required four distinct sections.
• Validate that no generic filler or placeholder terminology is present.