Consensus is a weird thing.
What is it?
The core meaning of the word revolves around something along the lines of a sovereign community agreeing on a set of rules to follow. Of course then that community needs to actually follow those rules. There may be incentives to cheat in this regard. It's very possible that the community in question is large and members of it don't necessarily trust each other. The best example of this is government, or even more relevant perhaps is our monetary system in general.
How do we force people to follow the rules?
Well governments use things like fines. And then after fines comes jail time. Then perhaps after that capital punishment, or if you resist or directly fight a "peace officer" you may be killed immediately. There seems to be a pattern here. Governments rule by punishment and fear of punishment. Very rarely is positive reinforcement actually employed, and this is likely a big mistake, as negative reinforcement has wildly diminishing returns, and can even act as a casino of sorts for thrill-seekers willing to break the rules for personal gain (while getting away with it 99% of the time).
But also a community could be something as trivial and insignificant as a book club. How does one decide which book they are going to read next? Such a decision requires consensus among the group. How is consensus achieved? Is there such a thing as book club drama? I assume so, as ridiculous as it sounds to say out loud.
The nice thing about crypto is that rules can be hardcoded into the system so that it is nearly impossible for any one person to break them. Even given the cases in which it is possible to break the rules of consensus on blockchain: the system remains transparent and everyone can often see that fuckery is going on. That alone makes it difficult to continue cheating, not to mention avoiding punishment.
The other nice thing about being able to hardcode the rules within a digital environment is that it is much much easier to employ scalable and healthy practices of positive reinforcement. By rewarding users to do what we want them to: control of the system can be much easier maintained. Consensus will always be stronger when good behavior gets rewarded rather than bad behavior being punished. Those are simply just easier and more agreeable rulesets to greenlight. They are exponentially more effective if set up correctly.
Looking at Hive:
I've had to do a lot of theory-crafting when considering building out my own token (Magitek). I've flip flopped on a couple issues back and forth a couple times now as I learn more and make revelations about how this all works. The biggest issue here is consensus itself. Which rules need to be confirmed by consensus and which ones don't? And how will that be achieved on the "second layer".
Of course by "second layer" I think it's more fair to say "sidechain" at this point. In my opinion second-layer very much implies a connection to the first layer. For example, on Bitcoin's Lightning Network several problems can be resolved by issuing certain operations on the main-chain layer-one. The first-layer supersedes the second-layer and overrules it.
Hive will likely never have such a protocol. The thing that we call a "second layer" is completely self sovereign and can't be manipulated by the first layer logic. This is by design so witness nodes have a lighter workload and don't have to confirm anything on these "second layers"... which are actually more accurately sidechains and totally separated from the main chain in many ways on the consensus side of things.
Of course maybe that's all just bullshit semantics & nitpicking.
Even I will continue to say second-layer even if another term is technically more correct. It just sounds better and makes more logical sense. Layer-0. Layer-1. Layer-2. It's a logical progression. Why break the pattern?
So what would layer-3 be?
Ah well in the context of Hive if layer-2s are on-chain custom-jsons that get verified by third-party nodes, then layer three might just be off-chain (centralized) operations. Of course perhaps another 5 years goes by and the definition of such things changes entirely given more infrastructure, better understanding, and new context. Crypto evolves quickly like that. We are still in the Wild West phase. Governments are trying to reign it in but only self-sovereign crypto can reign in itself. Such is the nature of decentralization.
Frontends aren't in consensus
The real reason I wrote this post is to point out something I've understood for a while but never quite appreciated the gravity of. Frontends are not in consensus with the blockchain they are attached to. At least not implicitly. The operators of Peakd can serve us any information they choose to. Same goes with LEOfinance, Splinterlands, Hive-Engine.com, or any other centralized frontend.
If say Peakd was inclined to write a fake post in my name making me look like a fool and plastered it all over the trending tab... nothing is stopping them from doing so. Of course it would not be that difficult to prove that said body of work was never actually signed by my private posting key. Peakd.com would then lose a ton of credibility as a valid source of information, which is why they'd never do such a thing in the first place. Still, we need to remind ourselves of what is possible and what is not within the ecosystem if we truly want to understand it and manipulate it in helpful and productive way.
Why do I bring this up?
When considering my own token I had to ask myself how consensus would be achieved. At first I thought I could just use a shortcut and piggyback on Hive consensus and that would be fine. Then I overthought it some more and realized that might not work. How can Magitek nodes/databases be in consensus if they aren't communicating with each other and able to change the state of the database? This adds a layer of complexity to the system that I'm quite frankly not equipped to handle, which was depressing.
But now I've flip-flopped the other way into believing that the shortcut piggybacking will indeed actually work. This belief is based on the knowledge that indeed frontends are not in consensus. Magitek nodes would not need to talk to each other because all of the operations have already been confirmed on the Hive blockchain itself.
Virtual Operations on Hive
This implies that every Magitek operation would simply be a derivative virtual operation pulled from the Hive blockchain, which I think will work just fine. Derivative consensus is an interesting simplified solution. If you don't know what a virtual_operation is I highly recommend that post I wrote in August 2020: that's how long it took me to truly understand virtual operations (from my humble beginnings in late 2017).
Or if you just want the quick rundown I can give it to you straight right here. A virtual operation is any operation that doesn't require a signature from a Hive user. They are implied operations, and they are quite powerful.
Think about it this way:
Every Hive node knows EXACTLY how inflation and the reward pool works. Every Hive node knows exactly what everyone is owed, so there's absolutely no reason to bloat the chain with data that every node already knows the answer to.
As we can see there are other virtual ops as well
- Powering down (
fill_vesting_withdraw) - Conversions to/from Hive/HBD (
fill_convert_request) - Trading on the internal market (
fill_order) - Unlocking money from the savings account.
- Allocating savings account yield.
- Reclaiming HP from a delegation.
- ETC
The only reason to put data on-chain is if an individual user needs to confirm that operation. Like transferring money or writing a post. So while the user must confirm that they want to start a powerdown, they don't need to confirm every week for 13 weeks that they want to unlock the money; this is implied using virtual ops and exists off-chain but still within the full-node API and other databases that maintain consensus. Thus these operations still take up space in a database and within indexes and whatnot, but they never have to be implicitly shared and confirmed constantly after every block between all nodes.
Again, this is quite powerful, and all second-layer Hive apps can leverage custom json tech to create consensus nodes that are 100% virtual ops and never need to be shared between other nodes. That's because the user has already confirmed the operation on the first layer. So perhaps it is indeed appropriate to call it a second layer instead of a sidechain. Who knows.
But what about database corruption?
Through no fault of anyone's own, databases can bug out. Numbers can get zeroed and transistors can glitch. This is obviously bad when it comes to building a financial system from the ground up. Luckily crypto is the definition of robust backup.
My idea for this problem is for second layer nodes to ping the Hive blockchain with a hash every once and a while to ensure that all second layer nodes are still in consensus. For example, my database would serialize all the data and then hash it with an algorithm (even Bitcoin's SHA-256 would work), and then post that hash to the chain. Other nodes in consensus should be able to serialize their own data and get the same hash. If the hash was different there would be a problem that needed to be resolved.
But what about financial incentives?
Another thing that really threw me off was the idea that second-layer nodes need to be financially incentivized otherwise nobody is going to run them (just like Hive witnesses). However, there are plenty of Bitcoin nodes running worldwide that get paid nothing for their troubles, so there are ways around this problem as well without needing a DPOS system for incentives (at least to start).
Conclusion
We put a lot of trust in frontends, to the point of many wrongfully assuming they are infallible and will never serve us corrupted or intentionally misleading data. This blind trust may be exploited one day down the line, but also the simplicity of this trust can be leveraged to create networks that aren't perfect, but are still good enough to operate as intended.
Consensus among groups of people is never a trivial thing, even if it's something as basic as a small private club. Humans have a way of trying to manipulating the situation to get what they want even if it's at the expense of others; Especially if it is at the expense of others. The problem of governance within this reality continues, but clearly crypto is a fantastic upgrade that hasn't even come close to full fruition. Soon™