Yesterday I started setting up a test environment for Hive things. I have done this before but I tend to get busy with other things and forget.

This time the intent was a bit different though. The main focus will be custom_json and using it to drive apps without too much else.

I spent 3 hours tweaking the frontend-only setup....

You know what, how the hell do I get all custom_json with a specific ID in one call?

As I mentioned I forget about all the little tests I do. I also forget about problems I have already solved. For Keychain login I have a script which I reference each time, due to not really writing Hive direct code often these little things slip right out of my brain.

That gets me back to the last question.
_{You know what, how the hell do I get all custom_json with a specific ID in one call?}

Custom_Json's place in the app workflow

It is one of the biggies I have at least tried to find the answer to more than once now. Yes, I am bloody ancient so go giggle somewhere else.

I always forget "custom_json is linear and not indexed".

It is just something I say too myself everytime I conclude the answer again. 3 times. many bloody times I have done this... give or take ;)

So no, you cannot say create a custom_json transaction with the ID current_balance for example and then receive all the current_balance transactions in one go.

How you would do it starting with 0 stored data is to replay the chain. Yes, the whole chain up until say the genesis transactions block.¹

Then I do hope you store that data in a way that works for your application because if you don't then guess what?

See 1 above.

Realtime

My first thought regarding custom_json is that it is a great store of data. It kinda isn't.

It is a good store of transactional data.

My thinking always breaks down because I try to make custom_json into relational data and it just isn't.

That is why something like HAF is being built. That is also why Hive SQL exists.

Although obviously HAF and HiveSQL index the whole chain I presume then we get to query it in a relational fashion.

So technically yes, you can do the call in my previous question all in one call. Given you have indexed the data.

This is where my gripe is with one of the chain's superpowers.

I want it to be bloody indexed from the get-go. Alas it is not so that leaves me to keep forgetting it.

Until I work with it more often that is.

So by default, custom_json is realtime data, and if you don't process it as such then you will be hard-pressed to manage any sort of state.

That is where it really falls into the app workflow. Immutable and replayable transaction data.

I bet you are saying, "Well howdy Captain Obvious," and fair enough. I didn't say this is the trap you fall into, did I?

No, it is the trap I constantly fall into when musing on different utilities for custom_json.

I always come up short when it gets to the aggregation. Yes I have googled sharding databases and you know what I still can't picture using custom_json as direct access for historic data.

It just does not work. Cough cough, HAF, HiveSQL

Exactly.

The flow for any app as I see it relying on custom_json is thus:

As you can see, the amount of Hive interaction has been scientifically noted in the red space.

So through this peer-approved analysis... We can see that aggregation of the data is the biggest pain point really. It really depends on the type of application also. If it is specifically say a single users note taking system, then sure you can replay their account history each time it loads up, you could even make this a bit quicker by submitting an aggregate custom_json that possibly stores hashes to the transactions directly.

For simple things, I am sure there will always be a simple(ish) solution. When I say simple I just mean a way to not have to constantly stream the chain and or reduce time-to-display.

In a way that is what things like HiveStats and them do, they just load your history and loop like there is no tomorrow.

For a single-user application that is okay.

Even so, Web3 has a habit of thinking it is the shit just because it somehow makes people a penny at the cost of user experience. Like saying poor people should be happy for the free clothes even if they hardly fit and need to use a coathanger to tie their pants.

Honestly, my only reason for harping on about wanting custom_json a lot more usable right out of the gate is specifically because of the cost to entry when a person realises well dammit I can't build this without a solid DB and high ingress traffic.

To be fair those people should also stop using these useless edge and cloud services that are designed to deliver nothing of use at double the cost.

Anyway all that to say I realised I need to change my Hive test app over to a full node.js at least so I can stream or replay the data as required and store it relationally for quicker access.

In short custom_json is rightly as HAF describes it in their docs:

An understanding of Hive's custom_json operations is critical to developing an interactive Hive app. A custom_json operation allows a user to embed one or more pieces of arbitrary json data into a Hive transaction. Interactive hive apps can utilize this feature to create a set of "commands" that their app recognizes and will process when a user publishes a transaction containing those commands.

Images are screenshots or made by me.