Greenshift 037: Developing A Network Cache

So, I've missed the last couple of days due to being stuck at work for consecutive twelve hour days, but given that I'm relatively free I thought I would share an idea I came up with while in the shower and refined during my breaks during work. Pardon not adding any new code today, I'm kind of burnt out on that end for today.

So, let's setup the issue we're going to tackle today:

When we're getting content for users to pick from, we are doing a lot of querying from the blockchain. Some of this querying is justified. But what if we query from the same starting point a second time a short time after the first? Well it is likely nothing has changed. So we waste a lot of time querying, especially if areas of the Steem topology are traversed very frequently in a short period of time.

Querying also takes a lot of time and some of the relationships do a lot of queries. So any solution needs to reduce the number of queries while also maintaining the fidelity of the product. What's the solution for this problem? Developing a cache for the entire network and updating the cache when updates are made.

Here's how it works:

We store nodes of the relevant network in our database. The relevant nodes in our case is a point where the relationships branch off of. In our case, a node would be a specific account since all relationships derive some information given a account. So, we would store seven things (given our current application):

1. Account Name (This acts as a Node identifier / reference )
2. Content derived from relationship 1
3. Content derived from relationship 2
4. Content derived from relationship 3
5. Content derived from relationship 4
6. Content derived from relationship 5
7. Timestamp of last operation

The first six are pretty obvious why we store them. We store the account name so if we search the same account in the future by anybody, we can simply give them the content rather than querying the blockchain a lot. The reason we need the timestamp is to perform a single check on the blockchain. If the author has performed an operation, then our information may be outdated and we should re-query the chain. If no operation has been performed, then we simply store the information in our network cache.

In heavily traversed paths, this prevents us from spamming the blockchain and overloading the server on the web application end. The cache also saves the end user tons of time and makes the overall experience more enjoyable while providing the same content they would receive otherwise. And that's an ideal solution. One with tons of benefits with no downside.

So we'll implement something like this in the future before launching the app. But for now, we'll probably shift our focus to developing a good entry portal into our traversal mechanism and developing a few backup relationships, so users have plenty of options to select from. Have a great night everyone!