Introducing the Acela Core - Upgrading Existing Web2 Apps into True Web3 Dapps
Hello, Community!
Welcome to another 3Speak update! In this post will talk about Acela Core, the new video platform backend we are developing for 3Speak.tv and SPK Network apps. As we develop this backend, it will play a critical role in bridging the gap between web2 like platforms and web3 land. We intend for this blog post to be a rough overview as each topic mentioned can be expanded out to dozens of individual posts. Over time we will gradually release more updates and information about what this new backend would entail.
Think of it similar to a layered cake. At the very bottom, the first layer of the cake is HIVE and other web3 tech. Then, one level up is a barrage of indexers to make the base layer data useful (SPK Network union indexer is a perfect example). Then, finally, the icing on the cake, the Acela Core, will go the remaining extra mile to provide platforms will a high end web3 experience without needing to write a ton of backend code.
Creating a platform is hard, you need:
- Account management (authentication)
The authentication side of the backend combines various authentication and identity systems such as HIVE keychain, metamask, SSO, and more into a single unified backend. This doesn't just apply to 3Speak, but other apps as well wanting to utilize this backend. A portion of this is simply linking your HIVE account to a username/password login and using that as the primary logins means. But also being able to create a HIVE account without needing to know about HIVE at all (web2 like experience). On the more complicated side of the spectrum is HIVE proxy accounts. With this, we can bridge operations from non HIVE accounts onto chain via a set of dedicated proxy accounts maintained by 3Speak or others. While this is a later step in the development process, we are actively looking at building it into the backend.
The authentication backend would have a simple set of APIs for handling login, posting, voting, account creation, etc. All can be easily accessed via regular HTTP calls with proper authentication token created during the login process.
- Storage layer
Large scale projects involving thousands of files and hundreds of TBs of data tend to require a dedicated service for handling data storage, management of data storage and integrations with other services. To begin with, storage on IPFS isn't easy. You can't just put a file on IPFS and expect everything to work 100% of the time. Thus, we are building a dedicated service to handle uploads, communication with a dedicated IPFS-cluster to manage part of the IPFS storage and a other misc functions. Additionally this storage management service would handle SPK Network POA (Proof of Access) integration at a later date once available.
- Indexing
By default in HIVE, there are only a handful of indexing systems available. Many of which are focused around providing their own forms of social indexing, and all of them do not have support for offchain posting. You can't just execute an API call on any random normal HIVE node and to do a massive complex query. That is why we are building the SPK Network's union indexer to turn HIVE on chain data and most importantly offchain data into a web2 like database. By indexing all this data we can do things like full text search. And we can even do advanced recommendations to improve the trending or user feed. We can mix both on and off chain content into the same feed. As time goes on we will publish more detailed blogs on such topics.
https://github.com/spknetwork/union-indexer-node
- Write capabilities
Currently most write operations such as posting a video, commenting, voting and more are done by first granting posting authority to @threespeak then using an API to interact in a web2 like fashion. The API translates incoming requests into on chain actions on the behalf of the user. This gives us lots of flexibility to do things like scheduled posts, login via email/password etc. We will be keeping this largely the same in the new backend, but modernized to significantly usability and overall quality of the backend. Additionally, HIVE proxy accounts will come into play where instead of posting directly to an account on a 1:1 basis, we can directly post onto a shared proxy account for hundreds or even thousands of users. Utilizing the union indexer to differentiate between users.
We also acknowledge login options exist such as HAS (Hive Authentication Service)/Hive Keychain are available where the user can directly sign for actions. We will be working that into the Acela Core where it fits/is necessary.
- Video Encoding
Generally speaking, all videos on the 3Speak site need to go through some form of encoding. Whether that reason is to create multiple resolutions of the same video or to reduce the size of the original video. It's widely considered necessary. The new backend will provide an interface to communicate with a dedicated SPK video encoder cluster for video encoding needs. That cluster is either operated by the a platform, or a remote 3rd party of users who form their own encoding clusters. The video encoding part of the backend is heavily tied into the auth, and most importantly storage.
- Health checks
We will be creating a dedicated service for handling healthchecks of the backend. To start this covers mundane things like an API being down and notifying developers. But also more complicated tasks such as verifying integrity of all stored video content and post metadata. Healthcheck system status will be available on a webpage and through a discord bot for more frequent updates in the future. This will play a critical role during the development process and maturity phase of the backend. Even more so important for platforms that might not have any idea what is going on during the initial phases of setup.
This is why we are developing the Acela Core for platforms like 3Speak and others to leverage without spending significant time developing a customized backend. It's also entirely open source and able to easily fork and build your own copy. Why build your own web3 infrastructure when the Hive and SPK Network community already provides it for free? Not only will these changes be important to the advancement of the 3Speak.tv platform, but also the entire web3 content and video community.
Ending notes
If you'd like to know more about the Acela Core backend in depth and discuss your use case. Please reach out to us on our Discord Server or contact @vaultec directly at vaultec#0001 to get in touch
Stay tuned for our upcoming witness blog post that should be available in coming days!
The above graphic is an in progress and incomplete representation of the new backend architecture.
Legend for the above graphic:
Green = Available/Meets MVP requirements
Yellow = In progress/partial
Gray = Not started/unknown/TBD
