Harmony- Enhancing scalability by optimizing across multiple layers

Background

The blockchain technology is well known as a disruptive technology which introduces a cryptographic mechanism for the transaction between two parties in such a way that it does not require the involvement of a third party. The business can happen in a p2p way and on the top of that, it is immutable, auditable by anyone. Simply put, the business is fair & transparent than ever with blockchain technology.

Since the invention of Bitcoin & blockchain technology, many other blockchain projects have evolved and the usability, applications further expanded. The smart contract concept introduced first by Etherum takes the usability & application of blockchain technology beyond mere p2p transactions.

The blockchain ecosystem by now has really become huge and definitely, it is in a growing trend. But we still have a long way to go as participation is in hundreds or thousands in number. This technology can really push its application and mass adoption when millions or billions of people join it and apply it in their daily life to serve various purposes, just like the people do with centralized systems and technology.

So what is needed is that the blockchain itself has to be made scalable, without scalability it will not be able to disrupt the traditional model of centralized business. Scalability is that driver of blockchain technology if achieved, then it can further market its decentralized model. So the priority is scalability from an application point of view, but not at the cost of decentralization.

Harmony is a network which aims to offer a scaling potential of blockchain technology with its unique innovation which also extends decentralization and highly secure.

In this article, Harmony will be discussed in details with its sharding technology, how it reaches consensus using BFT algorithm, how security and efficiency enhance its possibility for use in large institutions, gaming industry, etc.

Harmony

Harmony is a fully scalable(to an extent which is not achieved yet), provably secure and energy-efficient blockchain which uses linear BFT to reach consensus and it leverages on state sharding technology It constructs & design its sharding structure where the nodes of the shards keep on moving from one shard to other randomly in each epoch.

It has already achieved scalability of 118000 in the testnet with 44000 nodes. It also optimizes the cross-shard communication and resharding is done at every epoch to make the network efficient and evenly distributed without compromising with security aspect and decentralization.

It offers a highly scalable blockchain infrastructure where the big projects, decentralized marketplace can utilize the scaling capacity and can take the decentralized economy to a new level to create many possibilities for billions of people. It is the most decentralized form of staking powered by optimized sharding technology.

State sharding Architecture of Harmony

Sharding fundamentally is a scalable technology. Using this technology, the network or the database is partitioned horizontally and the small groups which are formed as a result of partitioning are known as shards. Each shard process different transactions and when all such shards process different transactions, the overall transaction throughput of the network increases and hence the network becomes scalable.

Security is an important consideration in sharding technology. The shards have to be made secure so that the network remains in a good state and reach consensus.

Depending upon the type sharding applied to a network, the properties of the shards varies.

For example, in Zilliqa the sharding used is of "transaction layer sharding", so the nodes of a particular shard process a set of transaction which is a smaller part of the net transaction throughput. Many such shards process a different set of transactions. But in transaction layer sharding, the state of the blockchain remain unchanged for all the nodes. No matter a node belongs which shards, the state remains the same. This type of sharding is simple but may not be efficient, as the nodes keep the whole state of blockchain and for ever-growing blockchain, it is a perennial problem, as the nodes take the burden and the burden increases progressively.

Analyzing this problem of "transaction layer sharding", Harmony apply state sharding instead. In state-sharding, the state and transaction of shards of the network vary and they collectively process and scale the blockchain network and reach consensus.

State sharding is a tricky concept because the shards keep different state. The consensus is an important security requirement. Keeping a chain where it has multiple shards each with a different state requires a robust design to tackle consensus and security.

So Harmony further takes the design of the sharding to a new level and establish a time interval known as epoch. During an epoch the nodes of a specific shard remain fixed but as soon as the epoch changes, the nodes also change and assigned to different shards.

But again, if the assigning of the shards to different shards after an epoch is over, is predetermined, then it will be easy prey for the malicious actors. So Harmony uses a random number and on the basis of the random number generated in its protocol, the nodes are assigned to the different shards in the network.

Harmony uses a protocol DRG which ensures that the random number generated in its protocol is unpredictable, unbiaseable, sclabale to a number of participants, verifiable by any observer. In order to make it unbiaseable & unpredictable, it uses verifiable random function(VRF) which uses cryptography to compute random number. In order to make it secure against any possible malicious behavior (for example malicious leader), it uses verifiable delay function(VDF), VDF is the latest technology which delays the reveal of random number, so it prevents last-revealer attack.

In its network, one node is a leader node and all others are validator nodes. The leader is responsible for broadcasting the message to validators, collecting multi-signature signing, initiating the consensus and committing the block.

The random number is committed in the block, by using DRG and running BFT among validators.

The leader sends the message with the hash of the previous block, say Bn-1.
VRF then creates random numbers for all the validators along with a proof.
The random numbers & the proof created in the second step send back to the leader.
The leader then runs XOR operation to get the preimage of the final randomness pRnd.
The leader then runs BFT among validators to reach consensus on pRnd. If the consensus is reached then it is committed in the block Bn.
VDF delays the actual randomness Rnd with a certain difficulty which is set algorithmically.
The leader begins to compute the actual randomness and once it is computed the leader again runs BFT consensus among the validators to agree on the validity of actual randomness Rnd.
Once the consensus is reached, the actual randomness Rnd is committed in the blockchain.

Any malicious actor or leader can not harm this protocol and the timeout mechanism can also restart the protocol, in case it detects any malicious act.

The sharding structure consists of two types of chains in Harmony. One is a Beacon chain and the other is a Shard chain. Fundamentally both are same but in function, they vary slightly. The state sharding in Harmony, make partition of the network and make small groups know as shards, these are also known as shard chains. The Beacon chain performs two additional responsibilities.

It collects the stake from the participants who want to become validators.
It generates a random number using DRG.

Beacon chain also includes the block header from each shard chain, that makes the chain consistent and secure. It will also be difficult for the attacker to crack both shard chain and beacon chain and include a fake block.

The communication between the different shard chains and between beacon chain & shard chain is accomplished through cross-shard communication which uses kademlia protocol. In effect, Kademlia protocol is much faster than gossip protocol. Kademlia protocol broadcast the message by traveling log(n) distance, so it can travel 1,10, 100.......as against 1,2,3....in gossip protocol. Put simply, it will achieve faster communication in the sharding structure of Harmony and that will improve the efficiency of the whole network.

Before the validators are elected to take part in the network and in the consensus process, they have to stake tokens in the beacon chain and become stakers first. Based on the staking amount, the voting share is determined. The price of each voting share is determined by adaptive threshold PoS, which uses a security parameter "λ" and adjusts the price of each voting share. Then the voting shares are randomly assigned to different shards. The stakers can then become validators and join the particular shard where their voting share is assigned randomly.

Once an epoch is over, resharding is done. In the next epoch the staker who unstake are evicted from the network. Those who stake can continue and the new stakers who join the network are given new voting shares. Then these voting shares are randomly distributed among shards which have more than median of the total voting shares. A constant number of shares from all the shards are taken out and then re-distributed to those shards which have less than median of the total voting shares. It uses cuckoo-rule to evict a node and assign a new node to a shard and the evicted node joins the shards which have less than the median of the total voting shares.

From the above sharding architecture, it is clear that the Harmony network does not shard based on validators. It first assigns the voting shares and then distributes it randomly to the shard and based on the that the nodes join a specific shard. Put simply, it gives very little to no room for the large-stake attacker to predict which nodes will be assigned to which shard.

Consensus

The consensus in Harmony is run though FBFT algorithm in Harmony. The leader runs the FBFT algorithm among the validators and reaches consensus and then the block is committed.

FBFT has certain specific merits over PBFT.

The first advantage is that in PBFT, the validators re-broadcast the message to all other nodes to count the votes after they receive the message from the leader. So this configuration yields a total network complexity of O(n * n). As the message is being routed two times.

FBFT makes the network complexity O(n), as the message is routed only once and the validators dont need to re-broadcast. The leader collects the vote by multi-signature signing.

The second advantage is that it uses BLS signature in place of schnorr signature. BLS multi-signature signing incurs only one-round trip as against two-round trips of schnorr signature.

Put simply, the BFT becomes fast and linear and the consensus is reached quickly with simplified network complexity. Harmony adopts RaptorQ fountain code to speed up the block broadcasting process.

The arrangement consists of a leader node, validator nodes, prepare phase, commit phase and FBFT is run to reach consensus in each phase and finally the block it committed and broadcasted in the network.

Prepare Phase

(1) The leader constructs the new block and broadcasts the block header to all validators. It is also known as an announcement by the leader.

(2) The validators check the validity of the block header, sign the block header with a BLS signature, and send the signature back to the leader.

(3) As it is FBFT and as the sharding in Harmony is done on the basis of voting shares, the leader does not wait for at least 2f + 1 valid signatures from validators, rather it waits for signature from the validators who possess at least 2f+1 voting shares and aggregates them into a BLS multi-signature. Then the leader broadcasts the aggregated multi-signature along with a bitmap indicating which validators have signed.

Commit Phase

(4) The validators check that again, verify the transactions in the block content broadcasted from the leader in Step 1, sign the received message from Step 3, and send it back to the leader.

(5) The leader waits for signatures from the validators who collectively possess at least 2f + 1 voting shares from Step 4, aggregates them together into a BLS multi-signature, and creates a bitmap logging all the signers. Finally, the leader commits the new block with all the multi-signatures and bitmaps attached and broadcasts the new block for all validators to commit.

Security & Efficiency

Harmony network is an efficient network because of the following reasons:-

The network complexity at any point is O(n), where n is the number of nodes.
The consensus uses BLS signature which makes the FBFT faster and linear.
The beacon chain maintains the consistency of the network.
Optimal cross-shard communication uses Kademila protocol which can travel O(log(n)) distance.
The nodes can easily sync with the network as there is no overburden to download the full state of the network, using state sharding technology, Harmony has simplified this problem and the simple machines can also join the network of Harmony.

Harmony network is secure because of the following reasons:-

The nodes keep on moving randomly from one shard to the other in each epoch and it is impossible to predict.
The security parameter is dynamically adjusted in adaptive threshold PoS which adjusts the price of voting share.
The network shard by "voting shares" not by "validators", this prevents large-stake attack.
VDF delays the randomness, which prevents last-revealer attack.
It is full-proof against 1% shard attack, as it uses PoS and the sharding structure is not fixed and keeps on moving.
It does not have "rich getting richer" in its PoS and it uses the "effective stake" mechanism to prevent stake centralization.

Applications & Use-case

For the enterprises who align their business goal with the essence of decentralization, high scalability & security, can find Harmony as the most ideal network infrastructure.
The gaming industries can issue non-fungible tokens and can offer a highly scalable interface to their players with Harmony as the underlying infrastructure.
High volume exchanges with low latency can now use Harmony network and there will not any complain about scalability.
Data sharing can be done all over the globe with Harmony network. This is probably the next best use-case of Harmony.

Comparison of Harmony with other similar blockchains

Summary

Driving on robust security, decentralization & scalability Harmony network can be the platform of multiple purposes; a platform for one and all, a platform for billions; a platform where decentralized market place can scale new heights. The optimization at every layer makes this blockchain distinct and stand out among all. Resource efficiency will lower the barrier to entry and expand the adoption rate of blockchain technology.