Repository
https://gitlab.com/vaultec/dtubepermanente
Weekly permanente dev update 1/12/19
In this series of future posts, i will post progress updates towards the development of permanente. This includes bugs, release dates, project changes, future improvements, new technologies that are being utilizes. I will also be developing all the modules permanente will be utilizing. Such as an IPFS distributed database, which any solution I have seen is not up to the requires that I need. OrbitDB comes close to something I can use but it is written in javascript/nodejs. These modules will be full developed to be used in external applications. An entire set of tools will be created for usage in permanente and external projects (more information in future posts).
Overview
- [Done] Database migration. Calling it permDB for now.
- [WIP] P2P networking.
- [WIP] Contract system.
- [WIP] PermDB (description TODO in future post)
- [Bug-fix] Database json serialization.
- [Not done] Logging support.
view roadmap here
P2P networking
This will serve as the main communication method for all permanente clients to utilize. Permanente (full instance) functions as an P2P server/client. If API is enabled clients can connect and utilize P2P functions of a full Permanente instance. Full instance meaning port forwarded + API + P2P. Contracts will be stored via DHT. In order to accept a contract, a node wanting to accept a contract, must send a message to the responsible node selling the contract. Once connection is established contract can be accepted. The node selling the contract will remove access for future peers to buy the contract. More details below on contracts. As mentioned above P2P capabilities are a work in progress. I would rather be using libp2p, however, they do not have a java implementation as of writing this. Using this off chain communication system reduces lag of contract agreement and prevents unneeded data to be stored on chain (contract hash can be referenced)
Specs
After researching a reasonable amount of P2P libraries for java. I have nominated TomP2P as the golden choice for P2P library. However, it lacks by default encryption support. Encryption can be added on application level, all P2P traffic is cryptographically signed. See security.
According to what is posted on TomP2P documentation. It can automatically manage port forwarding (UPnP), hole punching and firewall configure (if a peer is firewalled, has no control over actual firewall). Protocol is RPC like, TomP2P provides an interface into other peers via specifying raw bytes or utilizing object serialization. I do not know if sending objects through P2P works. I will have to experiment to figure out what the P2P system should look like.
Security
P2P encryption will be accomplished with secp256k1 over top TomP2P direct messaging capabilities. DHT records are signed with owner private key and can only be overwritten with that key. Allowing secure records to be published on DHT such as contracts, secp256k1 public keys, contract transaction memo/account name. PeerIDs are DSA public key. The key pair for DSA can be generated from a password or seed. Config.json will contain a a P2P seed. The hashing algorithm to improve security and reduce potential for attack is argon2. I have specifically choose Argon2 because it has won the Password Hashing competition. Argon2 settings are as follows; Iterations: 10, memory cost: 65536, parallelism: 4, hard coded salt. This brings the hash time to around 750ms (estimate). Which is well above the reasonable amount of time it would take for any GPU cracking device to break the key. That is not even including the time required to generate a DSA key pair out of the generated hash. Generated keys from P2P seed can be used in other areas of P2P related functions. Argon2 will be utilized in various areas related to security of permanente.
Contracts
See contract.java
Contracts should be built on top of IPLD objects. This is a major issue as IPLD is lacking java documentation and development. I will have to work around this and find other mechanisms to reference contracts, videos and other objects on the chain. Contracts are stored on DHT and signed by owner. They should also contain signature data to verify owner. These must be unspoof-able, strong data structure. Contracts should be verifiable. Nodes with active contracts will be checked by other nodes. Contract cost also must be verified.
Verification
It is required that nodes should check contract validity at all times. Contract rate per mb is in mSTM (0.001 Steem). Contract price is based off of total size. Size is calculated by adding the total size of every IPFS sub object that is in the contract root. Contracts are signed with DSA owner private key. Support for steem posting key for signature a possibility.
Final notes
Where are the IPFS Video formats in the last post at? Well due to lack of documentation and really my lack of understanding with IPLD and advanced sections of IPFS, I cannot create a working format (that is up to what i want) for Dtube videos and future formats. Keep in mind I am a single developer working on a massive undertaking. There will be more information in future updates. I may extend posting period to around 2 weeks. In the future permanente will not just be a video storage system. including new formats such as dImage, dAudio and D-Ads; More details in future posts!
Contact
You can reach me via partiko messenger or discord
See also:
dtube community improvement overview
#permanente #dtubearmy @dtubearmy #onelove #ipfs #WeAreDtube
