Image by @dozeOne of Hive blockchain's powerful features is its custom_json operation. Normally, users don't even see and hear about this operations. However, this operation enables developers to store their game or web application data on the Hive blockchain. If you would like to know more about what is stored in Hive blocks and and curious about various Hive operations, feel free to read my Exploring Hive Blocks post.
In this post I would like to share how to broadcast custom_json operations to Hive blockchain, how to retrieve them afterwards using Beem module in Python. Beem is a python module created by @holger80, and makes is easy to connect to Hive blockchain and interact with it using python programming language.
Custom_json is one of many operations of Hive. It allow developers to store game or applications data on Hive blockchain. Data is stored as text in Json format. Sine Json is a native JavaScript format and widely used in website and web applications, Hive makes it super easy to create Apps and Games that would like to use decentralized solutions.
Every Hive block has a size limit of 65536 bytes. Which means we can only store 65536 characters including all the properties in one block. I believe this is a parameters that witnesses can change if they wish to based on demand, infrastructure, scaling, etc. For now and foreseeable future this is the limitation every developer of apps and games should consider.
Another limitation is each custom_json operation is limited to 8192 bytes or characters. However, it is possible to broadcast multiple operations within a transaction. We face another limitation there. We can only broadcast 5 operations within a transaction by the broadcasting account.
Before we get to the code, another thing to keep in mind is, once transaction is built and we broadcast it, the return value doesn't not give the block number where the transaction is stored. That is because when we broadcast with python we don't wait until the block is produced. A couple minutes after broadcasting the transaction the block is produced and available for us to retrieve the data stored. When we broadcast we get back values like expiration, ref_block_num and ref_block_prefix. Using these three values we can find our transaction and the block it is stored in. So that we don't search the entire blockchain form the beginning, I also decided to store the current block number before broadcasting. That way I can start searching for the transaction starting from that block number.
Following is the code to broadcast custom_json operations:
from beem.transactionbuilder import TransactionBuilder
from beembase.operations import Custom_json
from beem import Hive
from beem.blockchain import Blockchain
from beem.block import Block
from helper import wif, name
from beem.nodelist import NodeList
from pprint import pprint
nodelist = NodeList()
nodelist.update_nodes()
nodes = nodelist.get_hive_nodes()
hive = Hive(node=nodes, nobroadcast=False, keys={'posting': wif})
blockchain = Blockchain(hive)
current_block_num = blockchain.get_current_block_num()
tx = TransactionBuilder(blockchain_instance=hive)
for i in range(1,6):
txt = str(i) * 8100
data = {}
data['data'] = txt
cj = {
"required_auths": [],
"required_posting_auths": [name],
"id": "librarian",
"json": data
}
tx.appendOps(Custom_json(cj))
tx.appendWif(wif)
signed_tx = tx.sign()
broadcast_tx = tx.broadcast()
print(current_block_num)
print(broadcast_tx)
Main thing to pay attention to is the TransactionBuilder. It allows us to build a transaction before broadcasting then broadcast it. I stored Hive account name and keys in a separate python file named helper.py, and imported variables name and wif to sign the transaction. To broadcast a this transaction posting key was enough.
We can add upto 5 operations in our transaction using .appendOps() method. Everything else should be self-explanatory. Feel free to ask in the comments if something doesn't make sense. I will be happy to explain each line.
Now the last two print statements show the block number before the transaction was broadcasted, and return values after the broadcast happened.
Now we need to retrieve the data from Hive. We will use the printed or stored block number to start the search from and values returned when we broadcasted the transactions such as expiration, ref_block_num, and ref_block_prefix.
from beem import Hive
from beem.block import Block
from pprint import pprint
from beem.blockchain import Blockchain
from beem.nodelist import NodeList
nodelist = NodeList()
nodelist.update_nodes()
nodes = nodelist.get_hive_nodes()
hive = Hive(node=nodes)
start_block_num = 52374262
blockchain = Blockchain(hive)
end_block_num = blockchain.get_current_block_num()
exp = '2021-03-23T02:08:41'
rbn = 10998
rbp = 3231133325
def get_data(start, end, exp, rbn, rbp):
ref = []
for num in range(start, end):
block = Block(block=num, blockchain_instance=hive)
b = block.json()
for i, tr in enumerate(b['transactions']):
if tr['expiration'] == exp and tr['ref_block_num'] == rbn and tr['ref_block_prefix'] == rbp:
print(i)
print(b['id'])
print(b['timestamp'])
print(tr['operations'])
print(b['transaction_ids'][i])
ref.append(b['id'])
ref.append(i)
return ref
ref = get_data(start_block_num, end_block_num, exp, rbn, rbp)
print('-----')
print(ref)
bb = Block(block=ref[0])
bb = bb.json()
bb = bb['transactions'][ref[1]]
bb = bb['operations'][0]['value']['json']
print(bb)
What the code above does is, it searches for the stored transaction with custom_json operations starting with the nearest known block numbers. Once found it displays the data we are searching for. It also provides the block number the transaction is stored at.
Now, we want to store this block number as reference for future use. Next time we need the same data, we can just get the block from the blockchain instead of iterating through multiple blocks and save us time.
If we have the block number, then we can easily get everything stored in that block as following:
block_num = 52374284
block = Block(block=block_num, blockchain_instance=hive)
b = block.json()
pprint(b)
One last thing I would like to share has to do with resource credits. I used an account with 1006 HP to test the codes above. The transaction above had 5 operations with 8100 characters each in size. It only took less than 1% of resource credits for this account to broadcast the transaction.
I hope you find this post useful. Let me know your thoughts. Also, let me know if there are better ways of broadcasting transactions and retrieving data from Hive blockchain using python.