Scaling Throughput with Linked Broadcast

Blockchains require replicating data, like transactions, across nodes in the network—but doing it efficiently and reliably is a challenge. For the last decade, best-effort broadcast has been the norm, relying on consensus to guarantee delivery. The result? Wasted bandwidth by sending the same data multiple times, bottlenecked throughput by routing messages through a single node (the leader in consensus), and degraded performance whenever that node is unresponsive.

Enter commonware-broadcast. Our latest primitive provides a common interface for data dissemination, with multiple “dialects” tailored around different tradeoffs and types of data. The first dialect, broadcast::linked, uses a concurrent, ordered, and fault-tolerant approach to replication that efficiently integrates with consensus mechanisms using threshold cryptography.

Let's break it down.

This dialect specifies two roles in the network: sequencers who initiate and order messages, and validators who verify and acknowledge messages. Membership in the sets is arbitrary; the set of sequencers and validators may overlap or be completely distinct. When used in a blockchain setting, the set of broadcast validators may or may not match the set of block validators.

Validators acknowledge receipt and validate messages by emitting partial signatures. These partial signatures are aggregated into threshold signatures once a quorum is reached.

For sequencers, each new message must also carry the threshold signature of the previous message in the sequencer's history. By induction, a single threshold signature provides cryptographic proof of the existence of the entire history of linked messages. That is, each previous message is guaranteed to have been validated by a quorum of validators. Credit goes to Autobahn, which introduces the idea that certified messages—when linked—imply a complete, valid, and well-replicated history of messages.

Figure 1: A diagram of a sequencer broadcasting messages, linking each to the previous one using threshold signatures. Multiple validators are contributing partial signatures.

The formation of threshold signatures over each message has powerful implications.

1. With just the latest message (i.e. the tip) of each sequencer, participants know that all previous messages have been broadcast and validated. This lets them begin syntactic verification of any new messages, while backfilling all missing messages in parallel. This enables efficient recovery from node failures or temporary network disruption, or efficient startup for nodes coming online for the first time.
2. Broadcast is decoupled from consensus. Consensus can finalize—in “one shot”—the entirety of all unfinalized messages by including the certified tip of each sequencer. This results in massive throughput. Broadcast can be parallelized with multiple sequencers without being blocked by consensus—even by a single-leader mechanism. Finalization via consensus then requires only a tiny amount of data per sequencer.
3. The threshold signature serves as a pre-consensus receipt of inclusion. Partial signatures are broadcast to all other validators as well as the sequencer, allowing any of these parties to construct the threshold signature. The threshold signature guarantees that the message becomes an immutable part of the sequencer's history, and thus that it will eventually be included in consensus.

Figure 2: A diagram showing how broadcast messages may be used in consensus. Every block references a set of sequencer tips, each with a threshold signature.

broadcast::linked also handles set reconfiguration. It is robust in the face of changing participants across epochs—distinct periods when the set of validators and sequencers may change. During changes, the aggregate set of validators must still be able to produce a threshold signature using the same threshold public key. Meanwhile, the set of sequencers may change arbitrarily. For example, the system might modify the number of active sequencers or remove and later reinstate them.

Sequencer tips are stored robustly in a write-ahead log, ensuring consistency even in the face of unexpected node shutdown. This prevents honest sequencers from broadcasting conflicting messages and also prevents honest validators from signing conflicting messages from dishonest sequencers. Conflicting messages are messages with the same sequencer and height that contain different data.

As with other primitives, this dialect is tested for resilience using Commonware's deterministic simulator. It reproducibly simulates complex scenarios such as packet loss, complete node failure, and temporary network partitions.

For developers building decentralized applications, broadcast::linked unlocks high-throughput replication using multiple sequencers and a simple, easily verifiable structure over message dissemination. Its use of threshold signatures proves the integrity of an entire history of messages, allowing for fast, succinct validation by external nodes ingesting network activity and can serve as early confirmation for users (and a slashable commitment for the sequencer).

Scalability is no longer a differentiator—it's a requirement. broadcast::linked is your shortcut.