Hive-Engine Light Nodes

Image Source: https://hive-engine.com

Hive Engine Light Nodes

The Hive Engine sidechain has been running for quite some time now and since its start a lot of new smart contracts were added. This and the increasing usage has led to the sidechain becoming more heavy over time. Currently running a full node for the sidechain requires 4-8GB of ram and a minimum of 100GB of storage. This doesn't sound like a lot when compared to HIVE itself, but it's increasing a few hundred MB per day and even witnesses have to run full nodes currently, even though they don't really require all data for verifying blocks.

In the last week I have been working on making changes to the Hive Engine node software to enable running light nodes, which store sufficiently less data, including the data required by witnesses to verify blocks.

As a result it will be much cheaper to run Hive Engine nodes, if not the full history of blocks / transactions is needed and witnesses won't be required to run full nodes anymore. This will hopefully make the Hive Engine network more stable in the long run, encourage people to run more nodes and make the process of restoring in case of an error much faster.

The changes are still experimental, but if you want to test out a light node, feel free to do so and report back to me if having any issues. You will find more info on how to do so further down.

Pull Request

I have created a pull request 2 days ago, which I put up for discussion and review in the witness channel. In this post I will be outlining the most important parts of it and describe some of the benefits / downsides of light nodes and what exactly I changed.
Here is a link to the pull request, if you would like to have a look at the code yourself: https://github.com/hive-engine/steemsmartcontracts/pull/144

Description

The pull requests adds light node functionality to hive-engine. Instead of storing all data of the hive-engine blockchain, light nodes only store smart contract data and data required for block verification, but not the full chain (blocks, transactions). This will allow for faster restoring / synchronizing and make the network / the witness block signing process more stable.

How to enable

If you don't know how to run a Hive Engine node, you will find information on how to do so here: https://primersion.com/he-links

If you already have a node you just need to:

• Stop the node
• Set lightNode: true in the config.json
• Start the node and it will initially delete all transactions from the database and all blocks < lastVerifiedBlockNumber - 1 (ATTENTION: this can't be undone and you can't switch back to a full node)

This process may take a few minutes - if you want to be faster you may want to use a lightnode snapshot and restore the database first. Ask me on discord for a recent snapshot or check out the following url:
https://snap.primersion.com/light/

Restoring process is the same as for full nodes:

• Download snapshot
• Drop database
• Restore snapshot using mongorestore
• Set startHiveBlock in config.json
• Set flag lightNode: true in config.json
• Start node
• If switching from a full node you may need to run db.runCommand({compact: 'chain', force: true}) to free up storage.

Differences of light nodes

Light nodes do not store transactions and do not store blocks, which means that the transactions collection is empty and the chain collection will only contain the genesis block and blocks >= lastVerifiedBlockNumber - 1. As a result the following changes were made to the rpc interface:

• getStatus request will return an additional parameter lightNode: true
• getBlockInfo rpc call disabled (throws an error)
• getTransactionInfo rpc call disabled (throws an error)

I already made a change to the witness monitoring site https://primersion.com/he-witnesses, which will now show a small green circle for witnesses only running a light node:

Cons

• Transactions are not stored in the database (transactions collection is empty)
• Blocks are only stored as long as >= lastVerifiedBlockNumber - 1
• Can't switch back from light node to full node (need to restore from full node snapshot)

Pros

• Fast restore and dump (~10 minutes, compared to > 2 hours for full node)
• Small dump size (400MB currently, compared to 40GB for full node)
• Nodes get less load
• 2GB of storage compared to 90GB of a full node
• Less RAM usage

The long term goal will be to create a good mix of full nodes and light nodes, which will make the network more stable and still provide enough full nodes for services requiring the data.
In any case I will continue to run a full node with my witness account @primersion.

That's it - if you have any questions about the code or something else, feel free to ask below.