Diffusion Benchmark Prompt Cleaning Specification Brief

Role: Staff Machine Learning Evaluation Engineer specializing in text-to-image alignment metrics and benchmark decontamination.

Context

• Benchmark Suite: {{benchmark_suite_name}}
• Raw Evaluation Corpus: {{raw_prompt_corpus}}
• Deduplication Threshold: {{semantic_duplication_threshold}}
• Data Leakage Patterns: {{leakage_token_patterns}}
• Target Evaluation Metrics: {{target_eval_metrics}}
• Token Window Ceiling: {{multimodal_token_limit}}

Task

Author a comprehensive prompt decontamination and data cleaning specification brief to remove training leakage, eliminate semantic redundancy, and normalize token complexity in our generative image benchmark corpus.

Method

1. Ingest {{raw_prompt_corpus}} and audit token length distributions against {{multimodal_token_limit}}.
2. Filter out non-semantic noise tokens, quality buzzwords (e.g., 'photorealistic 8k octane render'), and broken unicode sequences.
3. Compute embedding centroids using a frozen text encoder and prune duplicate prompts exceeding {{semantic_duplication_threshold}}.
4. Scan prompt strings against {{leakage_token_patterns}} to eliminate training set memorization traps and artist-specific copy strings.
5. Categorize cleaned prompts into capability buckets (spatial reasoning, compositional color, text rendering, style transfer) relevant to {{target_eval_metrics}}.
6. Normalize syntax formatting, grammatical structure, and capitalization while maintaining exact semantic intent.
7. Validate prompt balance across complexity tiers (short phrase, paragraph description, dense attribute lists).

Constraints

• MUST strip quality-booster filler tokens without modifying core relational noun-adjective phrases.
• MUST NOT retain prompts containing exact training set sequences defined in {{leakage_token_patterns}}.
• Prompt length post-cleaning must not exceed {{multimodal_token_limit}} tokens.
• Each evaluation category must maintain a balanced sample representation across difficulty strata.

Output format

Produce a technical decontamination brief organized as follows:

1. Benchmark Cleaning Objectives (concise summary under 120 words)
2. Decontamination & Filtering Rules (bulleted list of specific regexes and token exclusion filters)
3. Prompt Category Distribution Matrix (table: Capability Category, Clean Count, Complexity Tier, Target Metric)
4. Quality Assurance Protocol (step-by-step verification process to ensure zero leakage into {{target_eval_metrics}})

Self-review

• Confirm all 6 variables ({{benchmark_suite_name}}, {{raw_prompt_corpus}}, {{semantic_duplication_threshold}}, {{leakage_token_patterns}}, {{target_eval_metrics}}, {{multimodal_token_limit}}) are correctly incorporated.
• Verify that token cleaning logic directly addresses multimodal alignment evaluation requirements.
• Check that the output format enforces strict structural separation between filtering logic and distribution reporting.