Computes DAG-based DLT and decentralized queuing technology

When we set out to build our decentralized peer-to-peer mesh computing platform, we obviously looked into blockchain technologies but decided against this architectural approach due to several major reasons including: slow transaction speeds, wasted computes on proof-of-work (PoW), and ledger storage overhead. We ultimately decided to build a DAG-based "blockless" distributed ledger technology (DLT) for computes transactions and a decentralized queuing technology built on top of IPFS. The benefits of this approach include: incredibly fast transaction speeds with no wasted proof-of-work mining and no ledger storage overhead.

Computes architecture

All Computes transactions are linked via an IPFS Merkle DAG connecting all parent and child transactions in the appropriate order to create a ledger that manages tasks, input/output files, and logs. In addition to the DLT, Lattice includes our decentralized queuing and messaging systems. Computations can be sent into the mesh computer from any node in the private/public mesh network. Likewise, any node on the mesh network can assist with computations.

Tasks are decomposed and orchestrated to support both serial (frequently used in machine learning) and massively parallel computations. In fact, tasks can create new tasks and feed the output of one task into the next task as input. The split/map/reduce steps are being driven by the conditions of the previous task. Everything is linked together using IPFS’ DAG and IPLD.

Computes Subsystems

Other Computes subsystems work in concert with Lattice to create a single decentralized peer-to-peer mesh computer.

Computes Network:
Computes supports both private and public mesh networks running over wired and wireless networks with or without Internet access. Our networking subsystem is powered by IPFS Swarm.

Computes FS (filesystem):
Computes FS consists of a peer-to-peer (P2P) file system (also powered by IPFS). Input, output, and log files can be read, written, and streamed securely between nodes on the Computes mesh computer.

Computes Messaging:
Computes uses IPFS pubsub for creating a secure P2P and group messaging system between nodes on the Computes mesh computer.

Computes DB (database):
When needed, we use a combination of IPFS PubSub and IPLD (InterPlanetary Linked Data) to create a CRDT (conflict-free replicated data type) database system. This is primarily used to keep track of hash pointers to current DAG entries and files but it could also be used as a decentralized key/value store.

Getting Started

If you were one of the fortunate first 600 developers in the beta queue, you have already received a license key along with our Getting Started documatation that walks you through submitting computations to Lattice and checking results. It also walks you through chaining computational tasks for serial machine learning computations.

We subsequently released a demo bash script to further automate developer interaction with Lattice.

We also provided additional instruction how to build and run your own Docker-powered AI algorithms on Lattice.

Finally, we released another important article entitled, “Decentralized APIs”. This article highlights the local REST APIs provided by the Computes Daemon. These APIs could be used for building your own decentralized apps (Dapps) on top of Computes (without the need for a blockchain)!