The Problem
Changes to Hive consensus rules are often based on our intuition of what we expect to be valuable - it is very difficult to base changes to our consensus rules on empirical evidence.
How Hive forks work now
Every change to the consensus rules comes with great risk. Even if the change does not work out positively, there is a further cost in returning to old consensus rules, so often changes are taken forward regardless of whether or not they brought about the intended value - which itself is difficult if not impossible to measure.
Historically there have been debates about Author/Curation reward ratios, power down time periods, etc. We have made changes, and we have stuck with them, but we can't say empirically if they were beneficial or not. We can compare to differences in Steem, eg. Steem reduced power down time to 4 weeks. Price has gone up recently, but Steem vesting is substantially lower than Hive vesting as a proportion of available supply. Due to other differences, drawing a causal analysis is again difficult if not impossible.
As Hive gets more established, making changes to the consensus rules will be more and more risky, and the need to be based on data will become even higher.
A/B Testing on the Web
A/B testing is common on major websites. Typically, small changes are applied that a subset of users will see, for a limited period of time. The website will measure analytical data such as time spent on site, how many people clicked a certain button etc. The performance of the variations can be understood, and a decision can be made as to extend the change out to all users, or to not proceed with the change.
With a blockchain, we can't quite take this simple A/B testing method. If a chain splits into two, only one will retain trading pairs by default. The same can be said of the websites and all the infrastructure around the blockchain. Such infrastructure will have to make a decision about which chain to follow, and usually the default would be to stay with the unchanged ruleset. This greatly undermines our ability to learn lessons from the performance of the experimental chain.
Concept: How to A/B test consensus rule changes on Hive.
A better way would be to do fork two new branches at once, while the original chain continues as the established Hive network with existing trading pairs, website infrastructure etc.
The two branches would have a set end date, after which blocks would stop. This would allow us to compare the performance of the new ruleset (Green, ruleset A) against a control (Blue, ruleset B) which has the minimum changes required to form a new chain (ie. chain ID and anything else minimally required).
At this point there should be some obvious problems that can be seen with this system.
- Why would anyone buy a coin where blocks are set to stop, and coins would no longer be transferable.
- There would likely be a bias to support Ruleset A over the control, because people who favour the existing ruleset would already have an established chain they can use.
- What is to stop either of the new branches from removing the block end date, and just continuing as a competitor to the red chain.
Turning the experiment into an economic 'game'.
The way to make the experiment economically valid is to put money at stake in predicting which fork will be preferred, and to reward that back on the Hive blockchain. This will provide a greater financial motivation for both sides to support the fork they expect to have higher value.
Ahead of an A/B fork experiment, a fund would be set up on Hive using the Hive Proposal System. This 'Fork Fund' could run for perhaps a month before the fork actually happens. The Fork Fund would put financial value into the success of either new branch. At the end date, a snapshot will be taken of both chains - in order to know ownership of the tokens at that time as well as to have a price feed value for each token.
Instead of the new branches simply stopping at the end date, new blocks after that point will hyperinflate the branch tokens. All new tokens will be split among witnesses and a special account at some ratio. The special account will sell its ever increasing tokens, buy Hive and donate the Hive to the fork fund. This will collect the value from those who continue to speculate on the experimental branches contrary to their purpose.
At another fixed date, perhaps 1 month after the experiment end date, the entire Fork Fund will be split among accounts on the Hive blockchain (ie. an airdrop) - according to the ratio of prices on the two branches, and the account balances on those chains.
At another later date - a full fork of the Hive blockchain can be done based on the lessons learned from the A/B test - being more informed on the economic performances of the changes.
Limitations and Drawbacks
- This makes forking more complicated, to the benefit of more information and less risk to the system
- The Fork Fund and the special accounts would likely need to be managed by trusted individuals. Doing it fully trustless would likely be complex, although some kind of multi-sig or escrow solution could reduce the trust involved.
- New accounts on the experimental branches may need to be mapped to the main chain for airdrop (or account creation disabled on the branches, this is an added complexity to the experiment).
- Running the experimental branches may be expensive for witnesses. Perhaps the experimental branches should abandon old Hive data in order to save on running costs.
- It may be that branches would need to run for a year or more to show worthwhile information. At this stage we cannot say how much information will be attained from an experiment, but the principle of experiment, trial and error should apply to crypto as it does to other complex domains.
If you've read this far, thank you, I would love to hear your thoughts. I hope we can move towards making our decisions on Hive consensus rules based on data and experiment, to improve our ability to adapt and change going forward.