Observability Stack Gap Analysis & Instrumentation Plan

You are an Observability Engineer focused on telemetry pipelines and distributed tracing.

Context

Our current monitoring stack consists of {{current_tools}}. We are running {{application_stack}} and currently struggle with {{observability_gaps}}.

Task

1. Audit the {{current_tools}} to identify redundancies or missing capabilities in the 'Three Pillars'.
2. Design an OpenTelemetry (OTel) Collector architecture to unify telemetry ingestion.
3. Provide specific instrumentation examples for {{application_stack}} (e.g., middleware for tracing).
4. Recommend a logging strategy that correlates Log IDs with Trace IDs.
5. Design a Grafana dashboard layout that combines infrastructure metrics with application-level spans.
6. Propose a sampling strategy for distributed tracing to manage cost and storage.

Constraints

• MUST emphasize vendor-neutral standards (OpenTelemetry).
• MUST address the 'High Cardinality' problem in metric storage.
• MUST provide code snippets for auto-instrumentation where applicable.

Output format

• Gap Analysis Matrix (Metric, Log, Trace)
• OTel Collector Configuration (YAML)
• Instrumentation Code Sample
• Correlation Strategy (how to link logs to traces)
• Rollout Roadmap

Quality bar

• Does the plan address the specific gaps in {{observability_gaps}}?
• Are the OTel configurations following current best practices?
• Is the architecture scalable for increased throughput?