HeyStack!
It's been a while since I made an update post about HeyStack. What with the festive season and then alt-season everything's been a little hectic. But never fear, I have been pressing on, even if it's taking longer than I would have liked!
For those who haven't seen one of these posts before, HeyStack is a Hive-orientated crypto portfolio tracker project. It was my entry to the STEMGeeks Hackathon that took place back in the spring. I am still working on it! Still!
Progress - Loading and storage of pre-Airdrop (Hive=Steem) transactions and post-Airdrop Steem transactions
Before Christmas I'd sorted out the coin price storage and added a new profit and loss chart. This left the organisation of "Steem" transactions as the last major area of work to put in place before the launch of the basic project.
Essentially the issue is one of overlap:
- Transactions/operations prior to the launch of Hive will appear on both the Hive and Steem blockchains. And you don't want to load and store them twice.
- Transactions/operations after the launch of Hive will be different on the Hive and Steem blockchains. If you want a full portfolio history you need to load both sets of post-airdrop transactions, otherwise your balances for the unloaded blockchain will remain unchanged from the date of the airdrop onwards. So if you sold all your Steem you would end up double counting the sale proceeds (which would be captured from exchange trades) and the "unchanged" balances from the date of the airdrop.
This situation is made more complex by:
- Hive's account-history loading for transactions is much improved compared to Steem's thanks due to the work carried out by the @blocktrades team. Hive can now filter account transaction loading by type so that you only receive the types that you want. Steem still loads everything.
- Account history loading calls are based on the transaction number (your first transactions is 1, your second transaction is 2 etc). However as I found out after my first implementation attempt, Hive and Steem have different transaction history numbers for an account - even BEFORE the airdrop when the transactions are identical!
Anyway, the solution is a little dull visually - no pretty charts this time - but it looks like this:
Broadly it works like this:
- Start by loading the Hive wallet transactions.
- Loading of transactions starts with the last transaction and goes back in time in chunks of 1000-2000 or so. Only financial transactions are obtained.
- Hive transactions made after the creation of the Hive blockchain go into the Hive wallet storage.
- Hive=Steem transactions made before the creation of the Hive blockchain go into the Steem wallet storage.
- A new Steem wallet appears in the table to reflect this.
- When all transactions are loaded airdrop amounts are calculated based on a roll-up of the Hive=Steem transactions. These are stored in the Hive wallet storage.
- Then the Steem wallet can be loaded.
- Again, loading of transactions starts with the last transaction and goes back in time - now in chunks of 1000 as all transactions are obtained rather than just the financial ones.
- Steem transactions made after the creation of the Hive blockchain go into the Steem wallet storage (along with the Hive=Steem transactions).
- The loading checks the block number of the earliest loaded transaction in the chunk and loading stops once the latest Hive=Steem block number is reached.
- Any overlap in the last chunk is excluded.
As you can see in the image this saves loading tens of thousands of transactions even on these fairly-quiet accounts.
Loading time
So for these two accounts, for both Hive and Steem blockchains, the whole loading took around 5 minutes. And most of that time will probably be my slow internet.
Not too shabby!
State of HeyStack
These are the four tabs of HeyStack:
- WALLETS: For loading of blockchain wallet transactions - currently Hive / Steem accounts.
- JOURNEY: For charts of Hive / Steem transactions.
- IMPORT: For loading exchange trades, deposits etc
- STACKS: Portfolio overview - combining the blockchain wallets and exchange trades data. (In this particular example it's only the Hive / Steem wallets shown above).
What next?
- Some testing - my account balances currently do not tie up perfectly - although they used to before I changed the account loading. I need to work out why and test some other accounts.
- Tying it all together. Currently everything works well individually but it's not fully integrated. For example if you load new trades or include a new blockchain wallet it doesn't recalculate the portfolio until you relaunch. Similarly with things like refreshing prices.
- Refactoring and tidying. Lots of this.
- Income / gains. Because gains!
Support my projects!
That's it for today! My project is free and open-source (FOSS). It will be on github (once I've tidied it because embarrassing). If you like my project upvote to support it!