Last-Mile Dispatch Engine API Latency Diagnostic
Diagnose latency spikes, rate-limiting bottlenecks, and circuit breaker patterns in dynamic last-mile dispatch APIs.
Use this template when route optimization, carrier assignment, or vehicle dispatch APIs exceed SLA thresholds during peak delivery windows. It produces a detailed microservice diagnostic and resilience plan.
Role: Staff Integration Engineer specializing in dynamic route optimization microservices and API gateways.
Context
- Target Microservice: {{dispatch_service_name}}.
- Operating Load: {{carrier_count_peak}} concurrent couriers under peak dispatch windows.
- Routing Engine: Core calculations executed via {{routing_engine_vendor}}.
- Current Performance: P99 latency is {{current_p99_response_ms}} against an SLA target of {{sla_target_ms}}.
- External System Graph: Microservice orchestrates {{downstream_dependency_list}}.
Task
Conduct an in-depth API latency diagnostic on the dispatch engine, identifying synchronous blocking paths, connection pooling issues, routing matrix cache misses, and upstream rate limiting to restore performance within the SLA.
Method
- Map the synchronous versus asynchronous execution paths across {{downstream_dependency_list}} during a dispatch lifecycle event.
- Analyze how {{routing_engine_vendor}} integration impacts API response times under high concurrency.
- Evaluate HTTP connection pool sizing, keep-alive configurations, and circuit breaker trip thresholds in {{dispatch_service_name}}.
- Design a multi-tier caching strategy (L1 in-memory, L2 Redis) for distance-matrix lookups and geocoded addresses.
- Audit database lock contention and transaction isolation levels when assigning deliveries across {{carrier_count_peak}} drivers.
- Formulate an API gateway throttling and priority-queuing scheme to safeguard the solver from traffic bursts.
- Establish asynchronous event-driven dispatch patterns (e.g., transactional outbox pattern) to decouple non-critical tasks from the synchronous API path.
Constraints
- MUST address specific architectural characteristics of {{routing_engine_vendor}}.
- MUST NOT compromise transaction consistency (double-assignment of packages MUST remain impossible).
- MUST distinguish clearly between network latency, compute time in the solver, and downstream I/O blocks.
- Recommendations must be implementable at the API Gateway, service logic, and database layers.
Output format
Organize the analysis into the following structured sections:
- Latency Breakdown & Critical Path Map (tracing time spent across {{downstream_dependency_list}}).
- Bottleneck Root-Cause Matrix (categorized by Network I/O, Solver Compute, and Database Locking).
- Caching & Concurrency Optimization Design (exact cache keys, TTL policies, and connection pool configs).
- Resilient Fallback Architecture (circuit breakers, degraded-mode routing, and outbox patterns). Limit length to 550–850 words.
Self-review
- Does the analysis explain how to drop latency from {{current_p99_response_ms}} down to {{sla_target_ms}}?
- Are race conditions and double-dispatch vulnerabilities explicitly addressed?
- Is the routing matrix cache invalidation strategy operationally sound for dynamic traffic conditions?
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.