Cloud Infrastructure Incident Runbook Triage and Coverage Matrix

Role: Lead Site Reliability Documentation Engineer and Systems Architect with deep expertise in cloud resilience and operational playbooks.

Context

• Distributed System Architecture: {{microservice_topology}}
• Service Reliability Baselines: {{slo_availability_targets}}
• Incident Classification Standard: {{severity_tier_definitions}}
• On-Call Engineering Cohorts: {{oncall_engineer_profiles}}
• Upstream and Downstream Links: {{infrastructure_dependencies}}
• Regulatory and Compliance Rules: {{audit_compliance_framework}}

Task

Construct an Incident Runbook Coverage and Verification Matrix that evaluates operational procedures across distributed services, diagnosing missing triage steps, verification commands, and recovery action documentation.

Method

1. Deconstruct {{microservice_topology}} into critical failure domains, network boundaries, and stateful storage dependencies.
2. Cross-reference failure domains against dependencies outlined in {{infrastructure_dependencies}}.
3. Map incident severity triggers defined in {{severity_tier_definitions}} to measurable breaches of {{slo_availability_targets}}.
4. Audit existing runbook procedures for each service against the operational skillsets of {{oncall_engineer_profiles}}.
5. Evaluate verification commands (CLI snippets, telemetry queries, healthcheck endpoints) required to validate triage hypotheses.
6. Inspect rollback, failover, and data reconciliation instructions to ensure compliance with {{audit_compliance_framework}}.
7. Identify non-executable, out-of-date, or ambiguous playbook steps that increase Mean Time to Resolution (MTTR).
8. Standardize escalation paths, stakeholder communication templates, and post-incident review requirements into matrix entries.

Constraints

• MUST validate that every critical component in {{microservice_topology}} possesses at least one severity-mapped runbook entry.
• MUST NOT accept vague mitigation steps like "restart service"; exact CLI commands or orchestrator API calls must be referenced.
• Telemetry query syntaxes (PromQL, Datadog syntax, CloudWatch logs) MUST be specified in every diagnostic row.
• Compliance requirements from {{audit_compliance_framework}} MUST be explicitly mapped to evidence collection steps.
• Matrix columns MUST remain consistent across all infrastructure components.

Output format

1. Operational Documentation Health Assessment

A 2-paragraph analysis evaluating system runbook maturity and on-call operational risks.

2. Service Failure Mode & Runbook Coverage Matrix

A Markdown matrix formatted with the following columns: | Microservice / Domain | Failure Mode | Severity Tier (Breached SLO) | Telemetry Diagnostic Query | Mitigation Playbook Link / Command | Runbook Coverage Status | (Minimum 8 detailed operational failure scenarios).

3. Incident Escalation & Compliance Audit Matrix

A Markdown table addressing operational governance: | Service Component | Escalation Path | Compliance Evidence Requirement | Recovery Time Objective (RTO) | Audit Sign-off Status |

4. Operator Tooling & Command Validation Matrix

A table outlining operational commands: | Operational Step | Tool / CLI Dependency | Execution Command Template | Fallback Action |

Self-review

• Ensure every failure mode correlates directly with a threshold in {{slo_availability_targets}} and {{severity_tier_definitions}}.
• Verify all dependency risks in {{infrastructure_dependencies}} are addressed in mitigation steps.
• Confirm compliance verification steps strictly align with {{audit_compliance_framework}}.
• Confirm every command template contains valid, executable parameter flags without placeholder syntax.