Industrial Telemetry Ingestion Pipeline Overhaul Plan

Role: Principal Distributed Systems Architect specializing in industrial automation and time-series streaming systems.

Context

• Current field protocol: {{current_scada_protocol}}
• Sensor throughput: {{telemetry_data_volume}}
• Target message broker: {{target_cloud_broker}}
• Ingestion latency target: {{latency_sla_ms}} milliseconds
• Industrial regulatory standard: {{compliance_standard}}
• Legacy historian storage: {{legacy_historian_db}}

Task

Generate a comprehensive architecture modernization and cutover plan to transition high-frequency factory floor telemetry from legacy historians into a resilient event-driven ingestion backend without data loss.

Method

1. Analyze {{current_scada_protocol}} payload structures, polling frequencies, and network bandwidth constraints across production cells.
2. Define an edge gateway aggregation strategy to translate field protocols into compressed payloads for {{target_cloud_broker}}.
3. Design partitioning, keying, and backpressure mechanisms inside {{target_cloud_broker}} to handle {{telemetry_data_volume}} peaks.
4. Outline an intermediate dual-write architecture ensuring simultaneous streaming to {{legacy_historian_db}} and the new backend.
5. Specify stream processing microservices responsible for deduplication, out-of-order event handling, and dead-letter queues.
6. Formulate end-to-end data validation scripts comparing legacy historian metrics against new time-series sinks.
7. Detail a zero-downtime phased cutover sequence meeting {{latency_sla_ms}} latency requirements and {{compliance_standard}} compliance.

Constraints

• MUST maintain continuous operational visibility during all migration phases.
• MUST NOT introduce edge buffering overflows that could block operational PLC loops.
• Solutions must explicitly define circuit breaker trip thresholds and buffer sizing.
• All timestamp normalization strategies must enforce UTC conversion at ingestion edge.

Output format

1. System Topology & Data Flow (ASCII diagram + 200-word explanation)
2. Edge-to-Cloud Broker Configuration Plan (numbered specification list)
3. Dual-Write and Parity Verification Strategy (3-phase tabular breakdown)
4. Resilience & Backpressure Matrix (table mapping failure mode, buffer strategy, RTO)
5. Phased Cutover Execution Schedule (4 sequential milestone phases)

Self-review

• Confirm all edge-to-broker latency expectations align with {{latency_sla_ms}}.
• Verify that dual-write validation mechanisms guarantee zero data gaps in {{legacy_historian_db}}.
• Ensure buffer calculations accurately accommodate {{telemetry_data_volume}} during broker outages.