Eyes on Growth

SPK Network aims to validate Exabytes of data and to do that there needs to be a lot of thought put into how to break down the work load. In this post I'll talk about the lottery system and how it will enable scaling.

There are a few classes of players in this ecosystem:

• Validators
• Node Operators
• Storage Nodes
• File Owners

Validators are all node operators, but not all node operators are validators. Additionally, each validator is responsible for checking different content. Let's find out how the lottery system works to ensure fair play.

Content is referenced in the consensus database littleendian. This allows us to use very good search algorithms to get a range of content.

"IPFS": {
         "BzgfhxutiMS6e1hQVUWRXqm3it7SQc3KCQhe7W4hQKoSmQ": "dlux-io,dlux-io:0:74039882",
         "CVX7tyQYEGY7akDDc2Psn16r8UPr3NyocNNFNChvYE1WmQ": "disregardfiat,disregardfiat:0:74019598",
         "GdxSrHGJASmSbT5umJZxgJ9S15MfhNmDxCvhv9mxfZKdmQ": "markegiles,markegiles:0:74019097",
         "J5yTAAGVpexWm3TtQoyueYFEGx8wjQpabv3fz3GHWtHZmQ": "markegiles,dlux-io:1:74019011",
         "LMMLk3Vp9G3NiMHRsTSrwwUxpjy4tGdXBAUThXbQjTGbmQ": "dlux-io,dlux-io:0:74038164",
         "TnrqrQPUfHb1cS6XauP5m7UHq56JWQSe8PDiwDp94PRemQ": "markegiles,markegiles:0:74019097",
         "VUUqK53Rw2vMb7yArTcV34EFjwS8EZNmM1cpXgGfYvPamQ": "dlux-io,dlux-io:0:74038164",
         "VqpCP3cFHWtJaNgXMnGLpAe6AzyQnd7TUVBW8KZEZ2FZmQ": "dlux-io,dlux-io:0:74338637",
         "ZChdBejVbm6ydhMTKzGHGuqetodkApfZE2kjRMrj7sTamQ": "markegiles,markegiles:0:74043647",
         "cVHLzAT8LMb4JmUYnhuoGhpoBZvsqABF8Azj5iMA6qEemQ": "dlux-io,dlux-io:0:74039882",
         "fEdpNxazRR8b5V8DGSZqN5kiqKWagQNymCwsqbknDiyTmQ": "dlux-io,dlux-io:0:74039882",
         "gPZKaJ5TUaJzq1LA9XXmgv9noBpxQE2DBhYrLqdV7prYmQ": "markegiles,markegiles:0:74019097",
         "gZHJ2ixK6w9gJTBGmpZmLu1wds4kZ5NyP1h6a61xREvQmQ": "dlux-io,dlux-io:0:74038164",
         "jgM9svTatRsLLzgn4m393B8hKxod1Q3EVnvJcFcMD4FYmQ": "dlux-io,dlux-io:0:74039882",
         "kUZWX83pKyJU5W96NssfE1vwy5Q6ZQKKn4mBR3tk7AwVmQ": "dlux-io,dlux-io:0:74038164",
         "kwLc3ZM6CaUaaWCYA6nbKVdjiZ9XbhXVhaLXBhrKBBKVmQ": "disregardfiat,disregardfiat:0:74019598"
...

First we define our lottery as 9 digits of Base58. For those familiar with PowerBall in the USA that's 69 choose 5 and 26 choose 1. Which means the lottery for files in this state would be roughly the odds of winning PowerBall twice. To get to the place where we are checking an adequate number of files... we just specify how many files we want to check per day.

We know we have 20 validators who are elected and each will get a 2x share of the validator election threshold. This means there is no good reason to over vote a candidate. There is also any other registered validator who will get a proportional share to their votes.

How does this work? A drawing happens every 3 seconds. 28800 times a day. We take all our raw votes, and validator calculated votes, then determine how many numbers have to be checked to reach our target... of say 10%

Math.pow(58,9)*10/100

This range is then divided by the total votes, and again by 28800 (blocks per day).
Now every 3 seconds, each lottery number starts a range depending on who drew the number. A validator picks the number, their range will be 4x the size of a validator outside the top 20 who has half as many votes as the 20th validator.

The best part is, all of this BigInt calculation, and all the database searches for matching files doesn't need to be done by everyone. Only when a file is found, do all the node operators need to check to see if the numbers for that account allowed a file to be validated and the broca to be cycled into each good operators payout account.

Additionally, since the payouts happen from a pool, there is no reason to keep additional tracking of contract balances. It all comes out in the wash.

When a file is found in the lottery, the validator will query the contract and find all the nodes who have claimed to be storing the content. They will send a salt to the storage node that will be used in the calculation of the verification hash. If all three nodes send the same hash back there is a pretty good good chance the file is stored by all three. The responses are sent, encrypted on a pubsub channel that can be monitored to verify timestamps are acceptably placed on chain. With the PoA algorithm developed by @nathansenn file verification only needs to include a psudorandomly selected collection of file chunks... so additional verification won't need to transfer gigabyte files to verify.

So based on additional factors, like if the storage nodes didn't agree on a hash, not enough storage nodes replied, or the distance from the lottery number is modulus 10 (certifies 10% of checks) we can have additional, cheap, and most importantly, scalable verification.

We've spent a long time mentally attacking and improving our system and we're pretty confident that we are at least close to a system that isn't gamable. If you have any questions, comments, or improvements... drop them below!