Hey @edicted. This seems a lot like what we are doing at @spknetwork for our offchain indexing approach. We are working on a system to divide up the entire dataset between multiple nodes (slightly different than splitting up the index). Meaning a node doesn't have to store the entire dataset of all offchain content. In the event the node does not have the desired content (for example newly created comments under a post), the node can send out a special query to other nodes asking for new information. We utilize IPFS pubsub channels for communication with bloom filters to improve exchange efficiency. For the actual data itself we use Ceramic to provide us data mutability which is built directly on top of IPFS. Exchange speed will naturally be slower than storing directly, but that can be improved by prioritizing low latency peers.
You are viewing a single comment's thread from: