Harmony, an optimized sharding blockchain network which scales at multiple layers to achieve full scalability

Introduction

Fundamentally the blockchain technology is decentralized and it is a critical aspect of the blockchain. Using blockchain technology it is possible to send monetary transaction, data, etc in a peer to peer way. Bitcoin blockchain was the first ever blockchain technology which was invented in 2009. Then the next major blockchain technology was Etherum which introduced smart contract function in blockchain technology and explored the applicability potential of blockchain.

But blockchain in its application and implementation has failed to make a visible impact. It has failed to support gaming industries, high-throughput applications due to its scalability bottleneck. BTC processes at a rate of 7 tps, ETH processes roughly at a rate of 15 tps. The centralized peer like VISA processes at a rate of 2000 tps.

Since 2014 many blockchain projects proposed various solutions to achieve higher scalability. Some of the proposals include changing the chain-based consensus like PoW to PoS or to DPoS. The blockchain project like EOS use DPoS, IOTA use DAG. Many other existing projects proposed second layer scaling solution to increase the transaction throughput of the network.

However the various proposals fail to give a composite solution. The scalability in most of the proposal is achieved by sacrificing the critical aspect like security and decentralization.

There is one scaling solution which is a first layer scaling solution which does not sacrifice security and decentralization and increase the transaction throughput of the network considerably. The one I am talking about is sharding technology. In sharding, the network is divided into a number of multiple groups, these groups are called shards which process the transactions concurrently and achieve high transaction throughput.

Harmony optimizes this sharding technology and apply it at multiple layers of blockchains and form shards. These shards process the transaction concurrently and make the blockchain fully scalable.

Harmony

Harmony as a blockchain project has been designed with vast research to apply sharding at multiple layers of blockchain such as network layer, transaction layer, consensus layer, etc. It use FBFT algorithm to reach consensus which is linear and scalable. It does not sacrifice security and decentralization. This is secure with the use of protocol like DRG, adaptive threshold PoS, multiple layers of sharding, etc. The decentralization is also upheld with effective stake. The communication between the shards is faster with the use of Kademlia protocol.

With such a deep structure and a detailed one, Harmony offers its network as a perfect solution for the large enterprises, high volume exchanges, gaming platform and many other types of applications.

It has achieved a scalability of 118000 tps with 44000 nodes which is much bigger than the centralized payment network like VISA. So it has scaled a new height in terms of scalability.

Main components of Harmony

Beacon chain, Shard chain

The Harmony network being a sharded network consists of a number of shards. The shards further consists of a number of nodes. The different shards are known as shard chain in Harmony. They have their own state, they have their own account balance and they process the transaction that are relevant to them. So each shard chain differs from other shard chain in terms of state, transactions, account balance etc.

Theoretically Beacon chain is also same as that of shard chain. But the Beacon chain is specified with certain additional functions-- to accept the tokens deposited by the participant in the process to become validators & to generate random number for managing the sharding network.

Apart from the above, Beacon chain is also important for a synchronous network as well as from security point of view. When a new block is committed in the shard chain, its block header is sent to the beacon chain using Kademlia protocol. The beacon chain checks the validity of the block header and then broadcast it in the network.

Kademlia router in general is a fast routing protocol which travel across O(log N) nodes in the network. Harmony network is optimized with Kademlia protocol, so the communication between the shard chain and beacon chain is quick as compared to other network which uses gossip protocol.

Distributed Randomness Generation- DRG

The state shard network of Harmony consists of a number shards and each shard consist of hundreds of nodes. There is a time interval known as "epoch". The nodes are movable in Harmony and they move from one shard to another shard after each epoch. In order to make it unpredictable, a random number is generated and that random number further decides which node will go to which shard.

The random number is generated by using a protocol called DRG in Harmony. It utilizes VRF and VDF to enhance security, as VRF makes the random number unpredictable for the attacker and VDF delays it so that the network will not suffer from last-revealer attack.

The other properties of the random number are-- it is scalable to a number of nodes, verifiable by anyone and unbiased.

VRF- Verifiable Random Function-- It uses cryptographic sortition to select the group of consensus validators.

VDF- Verifiable Delay Function-- It delays the actual randomness.

Randomness generation involves following steps--

1. A leader sends an 'init' message with the hash of the last block H(Bn-1 ) to all the validators.

2. For each validator, after receiving the init message, a VRF is computed to create a random number and a proof for all validator nodes. Then, each validator sends back {random number , proof } to the leader.

3. The leader waits until it receives at least f + 1 valid random numbers and combines them
   with an XOR operation to get the preimage of the final randomness pRnd . (f- number of malicious validators)

4. The leader runs BFT among all the validators to reach consensus on the pRnd and commit it in block Bn .

5. After pRnd is committed, the leader starts computing the actual randomness Rnd. Actual randomness is delayed by VDF with a level of difficulty & can only be computed after certain predefined blocks.

6. Once Rnd is computed, the leader initiates a BFT among all validators to agree on the validity of Rnd and finally commit the randomness into the blockchain.

Note--

• Any dishonest actor does not get a chance as VDF delays it.

• The leader also can not corrupt it. If it tries to, then the leader will be switched and the randomness generation process is restarted again.

Consensus

In the shard network of Harmony, the consensus is reached by running an improved version of BFT algorithm, known as FBFT. But before that it is needed to understand that the Harmony network scales with the size of the network. So more shards means more nodes, means more transaction through-put also. But the key point is the connection latency, which should be linear and efficient. The objective to use FBFT is to effectively handle the connection latency and to linearly scale across the network.

In FBFT, the message is broadcasted only once as opposed to PBFT where it is broadcasted twice. So the network load in FBFT is O(n) as opposed to O(n * n) in PBFT.

In FBFT, the votes are collected using multi-sgnature signing which has only round trip as opposed to two round trips of schnorr signature.

Both of the above characteristics of FBFT are key to effectively handle the connection latency.

Harmony's consensus using FBFT is reached using the following steps--

1. The leader constructs the new block and broadcasts the block header to all validators.

2. The validators check the validity of the block header, sign the block header with a BLS
   signature, and send the signature back to the leader.

3. The leader waits for signatures from the validators who collectively possess at least 2f + 1 voting shares and aggregates them into a BLS multi-signature. Then the leader broadcasts the aggregated multi-signature along with a bitmap indicating which validators have signed.

4. The validators then verify the transactions in the block content broadcasted from the leader in Step 1, sign the received message from Step 3, and send it back to the leader.

5. The leader waits for signatures from the validators who collectively possess at least 2f + 1 voting shares
   from Step 4, aggregates them together into a BLS multi-signature, and creates a bitmap logging all the signers. Finally, the leader commits the new block with all the multi-signatures and bitmaps attached, and broadcasts the new block for all validators to commit.

Note--
Steps (1 to 3) constitute "prepare phase" and steps (4 & 5) constitute "commit phase".

State shard network

Sharding means partitioning or dividing the network horizontally and form small groups which are called as shards. But the sharding again is of different types-- transaction layer sharding, state sharding.

In transaction layer sharding, the shard will have different transaction but same state of blockchain.

In state sharding, the shards will have different transaction and different state.

Harmony uses state sharding technique to bring full scalability potential of blockchain. It applies 10x innovation at every layer to make the blockchain network fully scalable. Nevertheless sharding does not sacrifice decentralization.

The detailed process of stake based sharding

The sharding network consists of number of shards and each shard consist of hundreds of nodes. The nodes remain with a shard during an epoch interval and then changes and the change is governed by randomness.

The process of stake based sharding involves the following steps--

1. The participants stake their tokens in Beacon chain. Then the stakers obtain voting shares which is in proportion to the staked tokens. These voting shares are then randomly assigned to the shards. Here the sharding is done on the basis of voting shares, not on the basis of validators. The stakers become a validators once they are assigned to shard corresponding to their voting shares.the security requirement is to keep the malicious validators below one third. Adapative threshold PoS ensures this security requirement and adjust the price of voting share and assigns individual voting shares to shards rather than validators.

2. Once an epoch end, resharding is done. resharding is basically aimed at mainatining the balance & security in the network. Just imagine a huge network having thousand of nodes. In every epoch, there will be some nodes who will unstake and some new ones will come and join as nodes. So to allocate them to a particular shard and to keep a proper balance, to make it secure, resharding is done. Resharding follows cuckoo-rule to assign a new node and evict an existing one.

Resharding is done in the following steps--

• Those who unstake are evicted from the network.
• The new validators who come and join get new voting shares.
• These voting shares are then randomly assigned to the shards who have more than the median of the total voting shares.
• A constant number of the voting shares from all shards are randomly re-distributed to the other half of the shards who have less than the median of total voting shares.

Security

1. Both Beacon chain & shard chain improves the security of the network. Both the chain remain synchronous to the network. For an attacker to gain access and to include an invalid block it has to corrupt both beacon chain and shard chain, so that is not easy for the attacker.

2. Adaptive threshold PoS assigns the vote shares to shards rather than validators, which prevents large-stake attack.

3. DRG makes the assigning of nodes randomly using VDF and VRF which is unpredictable and unbiased .

4. As PoS is used for validator selection, it does not suffer from 1% shard attack, which otherwise requires hashing power.

Potential applications

1. High volume decentralized exchanges with high scalability and low latency can be a reality now with Harmony.

2. Gaming platfroms.

3. Large enterprises, industries which handle heavy traffic.

4. Data sharing

Comparison of Harmony with other sharding based blockchains

credit

Conclusion

It is a unique project, which has innovations at every layer. The potential is that-- as the number of nodes and shards grow, the scalability will also increase. So unlike other blockchain its application potential is more-- in other blockchains more number of applications result in congestion, where as in Harmony it will not be the case. In future when it will be bigger, the scalability potential will be more than what its now.

Its decentralization remain intact, security is robust, the scalability increases in proportion to the nodes. So it is a new dawn in the blockchain sphere, where the decentralized market place can grow with the assurance of all the critical aspects of blockchain--security, decentralization, scalability.

References

• Harmony Website
• Harmony WhitePaper
• Harmony Onepager
• Harmony Telegram
• Harmony Medium
• Harmony LinkedIn
• Harmony Instagram
• Harmony Twitter

Images Courtesy- Harmony.One website