Wind Farm Sensor Historian ETL and Analytics Database Migration Brief

Role: Senior Data Infrastructure Engineer specializing in renewable generation asset telemetry.

Context

• Generation asset name: {{generation_asset_name}}
• Legacy operational historian: {{source_historian_system}}
• Target analytical warehouse: {{target_warehouse_engine}}
• SCADA tag volume: {{scada_tag_count}}
• Downsampling interval: {{downsampling_interval}}
• Maximum allowable sync lag: {{max_sync_lag_seconds}}

Task

Draft an ETL integration and data modeling brief to stream SCADA operational records from legacy turbine historians into an analytical database optimized for predictive maintenance models.

Method

1. Profile the extraction protocol capabilities and egress limitations of {{source_historian_system}}.
2. Define a change-data-capture (CDC) pipeline to stream tag changes within {{max_sync_lag_seconds}}.
3. Design a wide columnar schema versus entity-attribute-value (EAV) schema for {{scada_tag_count}} active tags in {{target_warehouse_engine}}.
4. Formulate the edge-to-cloud downsampling pipeline applying {{downsampling_interval}} aggregation rules for high-noise vibration sensors.
5. Establish deadband filtering rules at the ingestion edge to drop non-significant sensor fluctuations before network transfer.
6. Outline schema evolution and dynamic metadata tagging for newly commissioned turbines.
7. Detail data validation checks comparing source historian tag counters with warehouse ingestion rows.

Constraints

• MUST specify the ingestion schema format (columnar, flat, or nested) with explicit justification.
• MUST NOT introduce network saturation over low-bandwidth wind farm backhauls.
• MUST maintain idempotency to avoid duplicate sensor timestamps during network re-connections.
• The output must follow standard technical brief conventions and stay under 650 words.

Output format

• Section 1: SCADA Egress & CDC Ingestion Architecture
• Section 2: Data Pipeline Transformation & Deadband Filtering
• Section 3: Warehouse Physical Schema & Partitioning Strategy
• Section 4: Data Quality Validation & Drift Mitigation

Self-review

• Verify that bandwidth constraints common to remote wind generation sites are addressed.
• Confirm that tag count scaling ({{scada_tag_count}}) is accommodated in the physical schema.
• Ensure all numbered steps in the method are represented.