Automated Canary Release Rollback Orchestrator Generator
Generate an automated shell and API orchestration script for instant canary rollback and incident notification based on telemetry drift.
Use this template when canary deployments exhibit anomalous latency or error spikes and platform operations teams require a hardened, idempotent rollback script. It coordinates cluster state reversion, traffic draining, and on-call paging.
Role: Principal Site Reliability Engineer & Platform Operations Architect
Context
- Target Microservice: {{service_name}}
- Runtime Environment: {{deployment_environment}}
- Health Metric Boundaries: {{telemetry_thresholds}}
- Deployment Infrastructure: {{orchestration_engine}}
- Alerting Channel Integration: {{pagerduty_service_key}}
- Rollback Execution Protocol: {{rollback_strategy}}
Task
Generate an executable, production-grade Bash and API orchestration script that monitors canary health against defined metric boundaries, executes an immediate zero-downtime rollback if degradation occurs, and logs execution telemetry to incident responders.
Method
- Initialize safe execution defaults (set -euo pipefail) and parse environment runtime credentials.
- Ingest {{telemetry_thresholds}} and initialize active polling against Prometheus or Datadog APIs.
- Implement a bounded polling loop evaluating sliding window metrics against baseline criteria.
- Define an evaluation branch assessing canary HTTP 5xx rates, P99 latency, and host resource utilization.
- Upon anomaly detection, invoke {{orchestration_engine}} API endpoints to instantly divert ingress traffic back to stable baseline pods.
- Execute the {{rollback_strategy}} sequence to terminate degraded canary instances safely without connection drops.
- Dispatch an automated high-urgency incident payload to {{pagerduty_service_key}} containing trigger metrics, blast radius stats, and pod execution logs.
- Emit structured JSON execution logs detailing script start time, metric snapshots, mitigation timestamps, and final cluster state.
Constraints
- MUST include rigorous error trapping and exit traps to ensure no orphaned canary containers remain active on failure.
- MUST NOT expose raw API credentials or tokens; extract them strictly from injected environment secrets.
- Script logic MUST be fully idempotent so repeated executions do not corrupt production deployment state.
- Do not use non-standard shell dependencies beyond curl, jq, and core system utilities.
Output format
Provide the deliverable in two strict sections:
- SCRIPT_HEADER: Configuration block defining environment variables, threshold constants, and dependency validation.
- SCRIPT_BODY: Complete, comment-annotated POSIX-compliant Bash script (under 250 lines) formatted within a single code block.
Self-review
- Ensure every shell parameter expansion is quoted to prevent injection vulnerabilities.
- Verify API query syntax against the declared {{orchestration_engine}} spec.
- Confirm all exit codes map distinctly to successful rollback, telemetry timeout, or API communication errors.
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.