The Problem
For the last few years I have been relying on CoinTracking as my crypto portfolio tracker. For those of you who have not used it, or perhaps have not used a portfolio tracker at all, the basic functionality is:
- Import your trades, deposits and withdrawals from exchanges like Bittrex.
- CoinTracking aggregates this information to provide a summary of your current crypto portfolio held on those exchanges.
- There is also historic information on your trades and the value of your crypto over time.
I have been broadly happy with CoinTracking as (a) It's free (b) I'm not a hugely active trader so something simple is fine and (c) It works by import so I can avoid linking to the exchanges through the site.
However this year Cointracking has changed its business model. Now you can only import 200 trades on the free plan, after which you have to purchase a pro account for $10+ per month. A 200 trade limit essentially makes the free version useless. Time for something new!
The other main irk that I always had with CoinTracking is that it does not do Hive (or FWIW Steem). There are links for many crypto wallets but Hive is not one of them.
This means that:
- My portfolio overview has always been incomplete.
- Any transfers of Hive to and from exchanges result in my portfolio value appearing to jump up and down over time, masking any trade performance and gains/losses.
- Trades that I make on the Hive market, or 3-day conversions between liquid and HBD, are not tracked.
- There is no possibility to monitor the split of my Hive earnings between income / gains / trading which would be useful for budgeting and tax.
The Solution - HeyStack!
So the solution I am proposing is HeyStack. A Hive-orientated crypto portfolio tracker which does everything you would expect from a typical portfolio tracker but additionally integrates automatically with the Hive blockchain.
Initial aims are to provide:
- An overview of your current crypto portfolio across exchanges and wallets. This will start with Bittrex and Binance for exchanges and, of course, Hive as the first crypto wallet.
- Historical tracking of your portfolio, including performance against Bitcoin, fiat and market indices.
- Trades analysis, including Hive trades and conversions.
- Separation of income and realised / unrealised gains for budgeting and tax purposes.
Your will be able to populate your portfolio through:
- Automatic integration with Hive at the click of a button.
- Imports from exchanges through drag-and-drop.
- Imports of all you old transactions from CoinTracking, if you use this.
- Manual adjustments for corrections and refining transfers / income sources.
So what is HiveJourney?
There are no historical account balances available from the Hive blockchain or API. So no way to simply grab the value of your Hive, HBD, or Hive Power at a point in the past and illustrate how these have moved over time.
In fact, the only two points in time where the account balances are known are (a) Day one of an account since you start with zero and (b) Now, i.e the current balances.
In addition, I want transfer information, particularly transfers to and from exchanges, conversion transactions, Hive market trades as well as ideally information on author and curation rewards to separate income from portfolio gains and growth.
It quickly became apparent that I would need a complete extraction and roll-up of all financial Hive transactions for an account. This would allow me to recreate the account balances over time as well as providing all the other items I was looking for.
In order for this to be robust I would need to analyse some accounts over their complete history, from account inception to now, to check that the balances I was creating eventually tallied with the current account values.
Later, once I am happy that the process works, accounts can work backwards from their current balances if they are only interested in more recent history.
The process allows me to build up a complete history for an account over time, showing the growth of their account through rewards, their powering up and down history, trades, transfers, conversions, account fees, the works.
It's an interesting project in its own right.
This is HiveJourney.
Progress - HiveJourney
I started with HiveJourney. If this didn't work I was going to be pretty stuck, or at least unable to provide the full functionality for HeyStack that I am looking for.
This is the account loading page that I built for HiveJourney which is now incorporated as the 'Wallets' page in HeyStack for your Hive wallet(s).
Add an account, load the transactions from Hive and then you are ready to see your journey!
Currently it loads all transactions rather than working backwards for a fixed period of time. This is something I will change once I am happy with the robustness.
The number of transactions is the most significant downside of the whole project so far. I am not particularly active compared to many but I still have around 100k transactions on each of my main and alt accounts. This takes a while to load; obviously more so for larger accounts.
This is a little frustrating as I only need a fraction of these transactions for HeyStack but there is no way to filter out the ones I want through the API.
The good news is that you will only need to do this once. The transactions will be aggregated and stored in a database on your computer, with non-financial transactions discarded. Then you just need to top up every now and again, which should be pretty quick.
Just a warning that, until I have the project fully operational, changes will most likely be required to the database structure and you would need to load the data again. So don't invest too much time on this at the moment!
The other piece of good news is that a few (simple?) changes to the Hive API would make this download process much more efficient. Maybe these could happen in future.
My Hive Journey
So this is me!
Powering up and then powering down again (in dark red; top chart) making little ears like a monster cat come to terrorise New York. Currently sitting on Bitcoin seeing if the Hive price will fall before coming back in. Red is liquid Hive. Grey is HBD.
OK, technically for most of the chart it is Steem, SBD and SP but it's the same transactions that are reflected with the Airdrop to produce Hive.
The lower chart is author rewards. I used to post more!
So the graphics are still terrible! But that's the part which will take a bit of time. Pretty much all the work is complete under the hood.
Hive Journey - did it work?
I'm going to say yes. There were lots of issues to solve and there are still a few more. But I have managed to reconcile accounts across their whole history to their current balances.
The process is perfect for most accounts for liquid Hive and HBD, both on the rewarded and claimed side of things.
For Vests (Hive Power) the reconciliation has around a 1% error. This is due to the power up transactions being one-sided; they are missing the amount of Vests obtained. So I have to estimate this using Hive * Vests Per Hive which requires some trickery. But 1% is pretty good and I'm sure I can improve it with more time.
There is also an issue with duplicate transactions on some accounts. These (virtual) duplications are ignored by the blockchain but I will have to filter them out.
Aside from that, I'll work on the graphics for HiveJourney.
Progress - HeyStack
So the HiveJourney side of things actually took most of the majority of the two weeks. But here's the progress I made on HeyStack:
This is the import page. Currently it supports:
- Binance - trade, withdrawal, and deposit files in xlsx format.
- Bittrex - trade files in csv format.
- Bittrex does not do withdrawal or deposit exports so I implemented a 'copy-and-paste' solution where you can, well, just copy and paste the table of deposits or withdrawals from the screen on Bittrex. You lose the withdrawal fees unfortunately but hopefully I can work around that.
- Cointracking - csv import of all their old accumulated trades for anyone, like me, moving on.
This all works well. I have a couple of issues on trade fees to resolve; a net or gross kind of thing.
HeyStack - did it work?
Again, under the hood, most of the work is done! I managed to import and reconcile my exchange account balances exactly for many currencies and very closely (aside from the potential fees issue discussed above) for others.
There was a lot of data processing and formatting work which is done. The data storage is currently an indexedDB database (as for HiveJourney above) on your own computer and this is done.
The aggregation work to produce totals and historic balances is done. The calculation of the Airdrop amount on exchanges for Hive is done (although I am likely to overall both these last parts as they were produced in a hurry).
The last major part under the hood is to bring the two projects together; integrating fully the Hive historic data from the Hive wallet with the Hive historic data from exchange imports and with the reconstructed airdrop. I have done this for the current balances but the history requires more work.
HeyStack - Graphics
As with the HiveJourney side of things, the graphics and trades analysis will take some time to complete, even if the work is done underneath.
The top chart provides the same info as a pie-chart but in the form of stacks of coins (imagination required here!). Stacks are the general theme I am thinking of going with. Each stack is 10% so the first coin is 45% of the portfolio etc.
I will play with this now I have the data coming through.
But in general the current charts are illustrative only and will definitely change.
Also that's not my crypto (actually the bottom one is - everything heading to zero!).
What comes next?
I think in order or importance:
- Getting this first proof-of-concept fully operational with the current import sources:
- Linking up all the Hive side of things between wallets and exchanges.
- Adding lots of cool graphics, trade analyses, .
- Firming the database so I am confident it won't have to be changed in future.
- Work on the efficiency of some of the database requests and calculations. I've done this is a hurry. There are undoubtedly lots of improvements.
Adding more exchanges (the Hive supporting ones first) and more wallets (e.g. Bitcoin).
Further into the future, potential for a forum (i.e. through to Hive) for people to discuss crypto and wallet related things, airdrops, coin launches etc.
A few other interesting related ideas, under wraps for now!
Is it useful?
For people already on Hive, with Hive being an important part of their crypto holdings or earnings, hopefully HeyStack will provide much more functionality than they can get from any other portfolio tracker.
For people outside Hive I would hope to build HeyStack into a tool with great functionality which would attract any crypto enthusiast, bringing people towards Hive from outside.
Conclusion!
Well this was my first hackathon. Much fun. I learned a huge amount. Thanks to the organisers and funders and other participants!
And I'm very happy with what I've produced. It's almost a fully operational Hive orientated portfolio tracker in just two weeks! And it should resolve my problem. Plus the HiveJourney side on top is interesting.
Apart from that I haven't slept now for 27 hours. So I'm going to tidy my code and go to sleep. Or go jogging. Potentially both at the same time.
Code will be here:
https://github.com/miniature-tiger/HeyStack
Questions in the comments.
Oh and love to @coingecko. For all your daily price history needs.