New open-source consensus protocol Mir-BFT goes Hyperledger Labs

blog

With the recently increasing media attention on cryptocurrencies, the technology behind it, blockchain, has become more and more well known to the general public. At the same time, blockchains are in flux as the technology continues to evolve. An area of current interest is that of the consensus protocols that constitute the backbone of these blockchains. Away from the media buzz, progress has been also made in the realm of permissioned blockchains.

Working to improve the performance of these permissioned blockchains, our team at the IBM Research Europe - Zurich lab has recently open-sourced a new Byzantine Fault Tolerant (BFT) consensus protocol called Mir-BFT, or simply Mir, that offers a number of advantages over the previous state of the art. Mir is now a Hyperledger Lab, available at https://github.com/hyperledger-labs/mirbft, which means that it is open to the broad community of developers to contribute to it. Hyperledger is a Linux Foundation open-source umbrella project for blockchain for business. In 2016, IBM open-sourced Hyperledger Fabric and made it a Hyperledger community project.  

Mir is designed primarily with the idea of powering Fabric ordering service, but it can be also used by other Hyperledger projects as it is developed as a library. Mir is partly funded by the PRIViLEDGE H2020 project.  

The key role of consensus protocols

Consensus protocols lie at the very heart of any blockchain. They establish the mechanisms by which all network participants agree on transactions and on the overall state of the blockchain. Different blockchains use different consensus protocols with different advantages and disadvantages in terms of throughput, efficiency, security and level of decentralization. For example, Bitcoin uses a consensus protocol known as “Proof of Work”, which involves participants using considerable resources in terms of CPU capacity and energy. Other cryptocurrency blockchains have therefore moved to implementing other consensus algorithms that trade off some desirable features such as decentralization for more efficiency and security.  

In permissioned enterprise blockchains with partially trusted members, the Practical Byzantine Fault Tolerant (PBFT) protocol has established itself over the years as an excellent consensus algorithm. Byzantine fault tolerance refers to the robustness of a distributed network to continue to operate even when a certain number of nodes are faulty or malicious (Byzantine).  

Traditional BFT protocols such as PBFT are leader-based. That means that one node, the leader, is tasked with putting together requests into blocks of transactions and communicating them to all other nodes. Consequently, the leader quickly becomes a bottleneck because the number of messages involved in reaching consensus increases exponentially when the number of nodes in the network grows.

Scaling up by removing the leader-associated bottleneck

In some use cases, especially for enterprise consortia working together in common supply chains, this prevents leader-based BFT protocols from being used beyond a certain number of nodes. Therefore, to allow scaling up to hundreds of nodes, this bottleneck needs to be addressed. One way to do it is by distributing the workload of the leader among several nodes. This parallel leader approach eases the strain on the single nodes in terms of CPU and bandwidth. But it introduces new issues like the possible occurrence of duplicate requests. None of the parallel leader consensus protocols currently available tackles this issue. However, duplicate requests are not just a waste of resources; they also can be used by malicious actors to mount certain types of attack.   

Removing the scalability limitation while retaining the advantages of BFT protocols has been the motivation behind our efforts for some time. This work now culminates with the public release of Mir-BFT, the world’s first parallel-leader BFT protocol capable of scaling up to an order of magnitude of 100 nodes even on wide area networks (WAN) while at the same time offering robustness against common attacks such as request duplication performance attacks and the Byzantine leader straggler attack.  One important feature of Mir is its robustness against arbitrarily long (albeit finite) periods of asynchrony. In terms of performance, our measurements confirm that Mir achieves the best throughput to data on public WAN networks with up to 100 nodes.  

30 times faster than the average VISA transaction rate

We implemented Mir in Go and ran it with up to 100 nodes on a multi-datacenter WAN, as well as in clusters and under different faults. Our results show that Mir is capable of ordering more than 60’000 Bitcoin-sized (500-Byte) requests per second. That represents a throughput one order of magnitude higher than other state-of-the-art single-leader BFT protocols. For the sake of comparison, Mir’s request throughput on 100 nodes on WAN is 2,5 times higher than the alleged peak capacity of VISA (24k requests per second) and 30 times faster than the actual average VISA transaction rate (2k requests per second). The research prototype code with which we achieved this performance is also open source available at https://github.com/hyperledger-labs/mirbft/tree/research

With Mir now becoming an open-sourced algorithm and a Hyperledger Fabric lab, we’d like to call on the community to contribute to its further development. This is a great opportunity to shape the future of enterprise blockchains which could have potential real-world impact in areas such as more transparent supply chains outfitted with better track and trace capabilities or in privacy-preserving digital health records such as the IBM Digital Health Pass.

Written by Marko Vukolic, IBM Research

Photo by Pixabay.