Once a Validator, Not Always a Validator

Many consensus protocols, from Tendermint and HotStuff to our own Minimmit, are defined under a "permissioned" model where the validator set is fixed and known ahead of time by all participants. Safely transitioning from one committee to the next, where validators can be added and removed, is left as an exercise to the implementer.

Tendermint-style protocols make the transition straightforward: replicas only start voting on height h + 1 once height h is finalized, so modifying the validator set between heights is safe. New faster-than-finality protocols (i.e. commonware-consensus::simplex), however, begin casting votes for the next height immediately after the previous height is notarized (when 2f+1 PREVOTEs exist but before 2f+1 PRECOMMITs exist, in Tendermint parlance). While this approach minimizes block time (and thus transaction confirmation latency), it complicates reconfiguration because there is no point at which validators have a synchronized view of the finalized tip. If validators use their "local view" of the finalized tip to determine who is eligible to notarize a block, the network risks a halt or a safety failure (as participants may consider votes from different validators to be useless).

What do? From Permissioned to Proof-of-Stake Consensus—a new paper by Jovan Komatovic, Andrew Lewis-Pye, Joachim Neu, Tim Roughgarden, and Ertem Nusret Tas—offers an elegant, consensus-agnostic solution. Treat the replicated log (i.e. the blockchain) as a sequence of segments broken into epochs where we simply "ignore" blocks that exceed some configured epoch length, whether finalized or not. Practically speaking, the validator enters a new epoch as soon as they process a finalized block at the epoch boundary (entering the next epoch with a synchronized view of the finalized state where they can update the validator set safely).

Figure 1: After some configurable number of blocks is finalized, a new epoch begins. If more blocks than desired are finalized, they are dropped. Epochs require direct finalization of the last block to advance. If this hasn't happened yet, we re-propose the same block. The last block of some epoch is treated as the genesis block of the next epoch.

To avoid reinventing the wheel each time a new consensus protocol is added to the Commonware Library, we opted to implement this reusable technique. This means you can now implement any permissioned consensus protocol, as described, and not worry about how to uniquely extend it for reconfiguration (including not having to implement mechanisms required to synchronize missing blocks across epochs, update peer sets, etc.). Check out commonware-reshare for a concrete example of how to incorporate reconfiguration into your application.

Resharing: Piggybacking on Consensus

When maintaining a threshold secret, changing the validator set requires more than updating a list. Whenever participants are added or removed, a new dealing of the secret must be performed to ensure new participants receive a share to participate in consensus and existing participants can't use multiple shares from old epochs to induce a safety failure (i.e. each player must not be able to use shares across dealings or else it is trivial for a single malicious participant to acquire f shares over f epochs). Our original commonware-vrf demonstrated how to perform proactive resharing but relied on a trusted Arbiter to ensure all participants received the same set of commitments. Removing that trusted coordinator requires some form of reliable broadcast such that all participants can each run their own arbiter (and all arrive at the same output). Fortunately, we get that out-of-the-box with a replicated log.

Our new commonware-reshare example marries the DKG implementation from commonware-cryptography::bls12381::dkg with threshold commonware-consensus::simplex's (implicit) replicated log. Validators proactively refresh their key material every epoch, a configurable interval of blocks, and derive new shares for incoming participants.

Each epoch begins with a set of dealers, the validators currently signing blocks, and players, the validators who will hold shares in the next epoch. Dealers generate shares for all new players (based on their share from the current epoch to maintain the same shared secret across epochs) and send them to players over commonware-p2p::authenticated's encrypted, point-to-point channels. Players validate the received shares and, if correct, send acknowledgements back to dealers. After some number of blocks, dealers construct an outcome containing their commitment, acknowledgements from players who received shares against that commitment, and any revealed secret shares for players that didn't acknowledge receipt. They publish that outcome to the log.

Figure 2: Dealers and Players interact with each other and the replicated log to derive the new shares and group polynomial.

At the epoch boundary, all participants derive a new group polynomial from the submitted commitments and a share they can use to participate in the next epoch from the selected dealings. Because commonware-consensus::simplex already ensures all participants see the same log, we know all participants will derive the same group polynomial and a correct share that can be used to participate.

Generating a Threshold Secret

The example ships with two bootstrap strategies. A trusted participant can derive all shares for the initial set of participants, or the validators can run a one-time DKG ceremony to generate the initial shares collectively (where no participant ever has full knowledge of the secret).

Figure 3: The DKG ceremony is performed once, and the resulting shares are used to bootstrap the initial epoch.

The trusted path is great for local testing and CI: the setup tool deterministically samples the group key, splits it into shares, and writes configs for each validator. The DKG ceremony is more complex but should be used for any production deployment (recall whoever has the shared secret can trivially undermine threshold commonware-consensus::simplex). A bootstrap committee uses the non-threshold dialect of commonware-consensus::simplex (powered by commonware-cryptography::ed25519) to agree on the transcript produced by commonware-cryptography::bls12381::dkg, and the generated shares are used by the initial participants in the first epoch. From there, the built-in resharing mechanism handles all future key material updates.

With an end-to-end solution for bootstrapping and maintaining a threshold secret over a dynamic validator set, the road is now paved for integrating Seeds, Links, and Views into applications building on the Commonware Library.

Trying It Locally

You can run the example by generating participant configurations and launching the validators with the command emitted by the setup step:

$ git clone [email protected]:commonwarexyz/monorepo.git && cd monorepo
# Trusted Setup
$ cargo run --bin commonware-reshare setup
# DKG Setup
$ cargo run --bin commonware-reshare setup --with-dkg
# Execute the `mprocs` command(s) emitted by the previous step
        

The setup step writes per-validator configuration files under the chosen --datadir (default ./data) and prints one or two mprocs invocations. For the DKG flow, run the first command to complete the ceremony, shut everything down, and then run the second command to launch the threshold committee that was just minted.

Next up, commonware-constantinople!