The biggest problem with blockchain and how to solve it, illustrated

The biggest problem with blockchain platforms right now can be illustrated by a simple analogy.

At the beginning of the 21st century, the internet was incredibly slow. If you had a blog back then, it would have been both time-consuming and expensive for users to access it.

People noticed the problem, but they also recognized the usefulness of the internet. To resolve this, they worked on improving the internet’s infrastructure and design. At the same time, they built better hardware to run important elements of the internet as we know it today.

Because of these changes, the internet became faster, and the costs of accessing websites became much lower. Before, developers were only able to create a single web page. But after the changes, developers could now build sites like Facebook, Google, and Twitter on the improved infrastructure. All this is possible because the internet is now very fast and scalable.

At present, blockchain transactions face a similar hurdle: they’re slow and expensive.

Blockchain companies are working to provide scalable infrastructure so that blockchain platforms can become mainstream in future. Once this happens, not only can people use blockchain for payment purposes, but they can also build applications similar to Google and Facebook on top of them.

Zilliqa is one of the companies seeking a solution to this problem.

Tech in Asia sat down with Yaoqi Jia, head of technology at Zilliqa, to find out more.

Blockchain and the problem of scalability

The problem of scalability cannot be explained without first explaining the idea of throughput.

Imagine you are at a train station. Trains here come in one-hour intervals, and only 10 people can fit inside. Once the train fills up, the eleventh passenger will have to wait for the next one.

Using this example, passengers represent transactions. Popular blockchain platforms like bitcoin and ethereum can process around 10 transactions per second on average.

In contrast, payment companies like Visa currently process around 5,000 to 8,000 transactions per second on average.

The rate at which transactions are processed is called its throughput. Blockchain platforms currently have a drastically lower throughput compared to payment platforms like Visa.

Now imagine you are at the same train station, but this time with a hundred other people who want to take the same train. The train arrives. The conductor looks at the crowd and says, “The 10 people who can pay the most for the ticket will get to board the train.” You’re in a rush, so you pay a huge sum of money for your ticket and board the train.

Due to the low throughput of blockchain platforms, transaction fees rise. For instance, a $10 transaction can easily have an added transaction fee of $2 on the more congested blockchain platforms today.

As such, the slow speeds and high costs for transactions on blockchain platforms are huge barriers for mainstream transactions to take place on these platforms today.

Increasing the scalability

For mainstream adoption of blockchain platforms, these platforms need to scale. Blockchain companies have come up with several solutions for this.

1. Increasing block size

The first and most straightforward solution is to increase block size.

Going back to the train analogy, this means increasing the train’s seating capacity from 10 to, say, 20 people. This way, the train can transport more passengers at a go and reduce congestion at the station.

The current block size for bitcoin is 1 MB per block. Bumping that up to 2 MB per block can double the number of transactions processed per second. The bigger the block size, the higher the number of transactions processed per second.

But this is problematic in practice.

As Jia of Zilliqa explains, “Say you increase the block size from 1 MB to 1 GB. The problem is that when you create the block, you have to send it to other people in the network.”

He continues, “If you use Gmail, you’ll know that when you upload a file that big, Google starts complaining. The network just can’t process that huge a file that quickly. That’s the problem with increasing the block size. You can’t go beyond a certain block size, so there’s a limit to how this solution can scale.”

2. Off-chain solutions

The second possibility is through off-chain solutions, like the Lightning network for Bitcoin, Raiden network for Ethereum, and Trinity for Neo.

With current methods, every time you make a transaction, it has to be approved by every node in the network. (A node is a computer that runs the algorithms to process the transactions.)

For example, you’d like to buy a cup of coffee from a particular cafe every day for a month. The coffee costs $1, but you don’t want to burden the entire network for a $1 transaction. Making a daily $1 transaction on the network is very costly.

So you sign a contract with the cafe owner. The contract states that every day, you’ll pay for your cup of coffee with a token worth $1. After 30 days, the owner can exchange these 30 tokens for $30 on the network. You send one transaction to the network to mark the start of this contract. To mark the end of the contract, you send another after 30 days. The owner then exchanges his tokens for $30 in cash from the network.

Because these are private agreements that happen outside the blockchain network, they’re called off-chain solutions.

The drawback of this system is that the process will be centralized around the servers hosting these off-chain transactions. These off-chain networks are also black boxes, and the transparency and security of of this solution is questionable.

3. Delegated consensus protocol

Processing transactions on the blockchain network is currently slow because every node in the network has to process every transaction. So if there are a thousand nodes in the network, then all of them would have to process every transaction made in that network.

It’s as if you have a group of 1,000 people, and each person has to agree on something. Not only will the process be lengthy, but it will also be very messy.

Instead, what if you pick 10 members from the community, appoint them as governing members of the group, and let these 10 people make a vote? It’ll be much easier and faster for them to reach a consensus.

Similarly, you can delegate a group of nodes to run the protocol and agree on transactions. Their agreement will be taken as representative of the consensus of the group.

Most scalable blockchain projects are using this protocol. To get into the “governing body” or consortium, members of the group will have to register their interest and undergo certain tests (called proof of stake, or proof of authority). If they succeed, they will be given a certificate to join the consortium and run the consensus protocol.

The trouble with this, however, is that the protocol now becomes centralized around the delegated group of nodes that run it.

4. Sharding

This is where sharding – an on-chain solution that divides the network into “shards” – comes in.

Sharding is not a new concept in the domain of databases. Sharding separates large databases into smaller, faster, more easily manageable parts called data shards.

The concept had never been applied to blockchain until Loi Luu, Prateek Saxena, and their team explored it in their research paper Elastico. An assistant professor at the School of Computing at the National University of Singapore, Saxena is also Zilliqa’s chief scientific advisor. Luu, who works for the company as an advisor, was a PhD student at the same university when the paper was written.

Zilliqa’s solution is called network sharding.

To illustrate how it works, imagine that you have a five-step problem.

Now you are in a classroom of 10 students. You have this problem, and the steps can be solved in parallel. You divide the class into five groups of two students, and get each group to solve each of the subtasks. The first group will solve the first subtask, the second group will solve the second subtask, and so on.

Once a group is done with its subtasks, it passes the information over to the teacher. The teacher will aggregate the answers from the five groups and pass the full set of information to all the groups. This way, all the participants get to know what each group was working on, even if they had no idea what the others were doing while they were focused on their subtasks.

The students in this example are nodes on the Zilliqa platform. The “class” consists of all the nodes in the network, and the “groups” are called shards. The “teacher” represents the Directory Service (DS) Committee, which is a shard in itself.

Compare this to a classroom on the bitcoin network, where students have to solve every step of the problem on their own. This approach takes much longer and is a lot more taxing on each participant.

Using Zilliqa’s networking sharding, if you had 10,000 nodes, they will be randomly divided into 10 groups via a process called proof of work. Each group is called a shard.

Every shard will process a different set of transactions and come up with a set of transactions they all agree upon. They then report a digest of these transactions to another shard called the DS Committee, which will aggregate such digests from the different groups, and combine them to form a larger entity called the final block. Information from the final block will be sent back to all groups.

This process is called network sharding, and Zilliqa is the only company known to be working on it at the moment. Network sharding is a decentralized, secure on-chain solution that gives you linear scalability — that is, the more nodes you have, the higher the throughput you get. Zilliqa can currently process 2,488 transactions per second, and is aiming to get to 10,000 transactions per second in future.

Ethereum is working on a similar but different process called state sharding. In network sharding, not every node has to process every transaction, but every node will have to store information about what’s going on in other shards in the network. But with state sharding, every node will only store the subset of the information they’ve processed. While this reduces the burden of each node, communication between shards gets complicated.

However, the downside of sharding is the complexity of the design and implementation.

“We started our research in 2015, and are still running experiments for sharding,” observes Jia. “It’s simple to implement a naive version of sharding, but difficult to do it correctly and securely. This is why so few teams are working on it.”

“Compare this with the idea of increasing the block size,” he says. “That’s the simplest solution. You only have to tune one parameter. Increase the block size, and you increase the throughput. When you introduce sharding, there are so many other things we need to think about. How do you properly select each group member? How do you split the tasks? All these bring complexity to the system. But if done correctly, your system will be decentralized, secure, and give a linear throughput.”

Bandwidth and storage will also be common downsides – not just for sharding, but for all scalable blockchain solutions.

This is not yet a problem for popular blockchain platforms like bitcoin and ethereum because of their low throughput.

“Bitcoin and ethereum process 10 transactions per second,” says Jia. “At the current rate, all transactions in a year will amount to a few GB. But when we tackle tens or hundreds of millions of transactions a day, there will be an explosion of data, and in order to store all those transactions, we’ll have to find a better storage solution. This is something that network sharding does not solve directly.”

Running public tests

Network sharding has been successfully implemented internally at Zilliqa.

The team would now like the public to try it out on their testnet, which will be out at the end of March.

A testnet is basically a duplicate system for the team to run and test the platform before deploying it as a production system, called the mainnet.

Trial of Zilliqa on the testnet is targeted at supporters of Zilliqa – including developers and community members interested in the protocol – as well as blockchain developers interested in the concept of sharding.

In the testnet, users can observe the status of the blockchain system, such as the number of translations being processed and the current epoch number. In the meantime, users can use wallets to create valid accounts to send and receive transactions in test tokens.

Goals

In the near future, Zilliqa is planning to launch its blockchain system and smart contract platform, which will enable the running of applications on the blockchain. Next in line is the release of anchor applications – called decentralized applications, or dApps for short – on the platform. Think of these as applications like Google and Facebook, but released on a decentralized blockchain platform. These apps can push Zilliqa into mainstream usage.

“If a blockchain is an operating system, we want Zilliqa to become the Android or iOS of the future,” says Jia. “We hope that ordinary people will be able to use and develop dApps easily, and someday notice that their favorite apps are built on Zilliqa.”

In future, Zilliqa is also looking to solve open problems in blockchain, such as scalable solutions, privacy, and storage.

“These are problems not just for the industry, but for academia,” says Jia. “In the long term, we would like to provide a scalable blockchain platform. At the same time, we also want to do research, publish papers, and support the blockchain industry so that we can resolve these open problems and build the internet of the future.”

Here’s all our sponsored coverage about Zilliqa.