Distributed Consensus Protocol Evaluation and System Specification

Role: Staff Distributed Systems Researcher and Infrastructure Architect

Context

• Network synchrony assumptions: {{network_synchrony_assumptions}}
• Fault tolerance requirements: {{fault_tolerance_thresholds}}
• Target throughput and latency: {{throughput_target_tps}}
• Academic literature corpus: {{corpus_focus_papers}}
• State storage and log model: {{state_storage_model}}
• Cluster topology and node scale: {{deployment_topology}}

Task

Synthesize consensus and state-machine replication literature from {{corpus_focus_papers}} to author a formal distributed protocol specification that delivers {{throughput_target_tps}} under {{network_synchrony_assumptions}} while satisfying {{fault_tolerance_thresholds}} across {{deployment_topology}}.

Method

1. Review the theoretical properties, safety proofs, and liveness arguments documented in {{corpus_focus_papers}}.
2. Analyze leader election, log replication, and view-change mechanics under partial synchrony and asynchronous partitions.
3. Evaluate quorum intersection properties and message complexity against {{deployment_topology}}.
4. Reconcile theoretical message-pass overheads with real-world disk I/O bottlenecks in {{state_storage_model}}.
5. Synthesize the optimal algorithmic variant into an explicit state-machine transition model.
6. Formulate formal invariants for safety (e.g., linearizability, agreement) and liveness under {{fault_tolerance_thresholds}}.
7. Detail message wire formats, RPC lifecycles, and commit step protocols.
8. Define edge-case handling for partitioned leaders, Byzantine nodes (if applicable), and crash-recovery loops.

Constraints

• MUST define explicit safety invariants and formal proofs-of-correctness sketches.
• MUST NOT leave message complexity or quorum size unquantified for {{deployment_topology}}.
• Spec MUST document exact protocol behaviour during network partition and recovery phases.
• Protocol invariants must strictly conform to bounds defined in {{corpus_focus_papers}}.

Output format

Deliver an engineering specification divided into:

1. Theoretical Literature Comparison (Markdown matrix evaluating 3-5 consensus protocols on quorum size, rounds to commit, message complexity, and recovery cost)
2. Protocol State Machine Specification (formal states, transitions, message schemas)
3. Safety and Liveness Proof Sketches (rigorous logic statements and invariant bounds)
4. Storage Engine and Log Compaction Interface for {{state_storage_model}}
5. Failure Mode Matrix (table mapping failure scenarios to mitigation logic)

Self-review

• Check that message complexity scales realistically with {{deployment_topology}}.
• Verify all quorum overlap conditions satisfy {{fault_tolerance_thresholds}} mathematically.
• Ensure recovery protocol guarantees zero data loss under specified synchrony assumptions.