Maintaining Full History API nodes has become a burdensome and expensive task for any BP who has committed to doing so. However, their disappearance would seriously complicate things for dApp developers as this is the only way to efficiently query blockchain data. Attic Lab have been following the situation closely while also working on our own solution that would be scalable, efficient and cheap. So today, after 2 months of work we are happy to present our open-source Full EOS History API running on Elasticsearch cluster.
When designing a full history solution, our main goal was to keep the cost of building and maintaining the cluster at a reasonable level. Therefore, we preferred using several items of commodity level hardware over a single item of a top-notch gear. Applying such an approach gives us more scalability and higher speed of parallel processing.
Setup details
Out current setup is: 7 servers, ~10Tb of disk space, 448Gb of RAM, 3 shards per rollover index, and replicas=1.
We have built and synchronized the elasticsearch cluster and developed API in Go for interaction with it. The API supports native requests as well. You can find more details here:
https://github.com/atticlab/eos-es-historyapi/edit/master/README.md
Our cluster is powered by the Elasticsearch plugin by EOSLaoMao. https://github.com/EOSLaoMao/elasticsearch_plugin
Although GO API is in the beta-testing stage now, we are already working with real requests. During the benchmark testing the cluster was able to seamlessly handle 2500 req/sec at the peak of activity.
Further plans
As of now there are two main areas that we plan to work on:
First of all, we are focusing on increasing the processing speed by means of optimizing traffic and caching requests. Secondly, a lot of work is done on expanding the cluster itself and increasing its fault-tolerance.
Our Full History API endpoint: https://eosbp.atticlab.net/v1/chain/get_info
Besides, we have already started development of our own solution, Cassandra plugin for Full History node storage and native API. More information on the progress will be released a little later, so stay tuned!
We would like to thank the guys from EosLaoMao https://eoslaomao.com/ and Pete from BlockMatrix https://blockmatrix.network/ for their support and suggestions. This achievement wouldn't have been possible without them.
And of course, any help and contributions from the community are greatly appreciated. We encourage everyone to test our solution and leave feedback.
Follow us!
Twitter: https://twitter.com/atticlab_it
Reddit: https://www.reddit.com/user/atticlab_it
Website: http://atticlab.net/eos/
Facebook: https://www.facebook.com/atticlab/
Steemit: https://steemit.com/eos/@attic-lab
Medium: https://medium.com/eosatticlab
Golos: https://golos.io/@atticlab
Telegram Chat: https://t.me/atticlabeosb