You are viewing a single comment's thread from:

RE: Discontinuing my daily statistics posts

in #stats8 years ago

Thank you for replying here @ned.

The first thing I want to point out is the technical challenge of pulling these stats: payouts are part of what is called virtual ops in the blockchain these are not present in blocks, but rather in the accounts state part of the blockchain, this data is not available via the SteemSQL database implementation, and even if it was, parsing all accounts to generate a single daily report would put SteemSQL in DoS or get the client connection banned because it would trigger some DDoS protection mechanism, we are talking about millions of records that have to be parsed each time. I read some people here think it’s an easy task, I invite anyone who said that to give it a try. With about 95k accounts in the database, taking as little as 0.1 second per account could result in hours of treatment to get a single report. I use a custom program I wrote which is more than 3k lines of code, and I also use a modified steemd client that allows larger queries to the blockchain database, so even using a public API cluster will not work. I also use a top-shelf workstation to process the data, regular machines will just crash. I don’t want to go into more technical details as it’s not our main point here.

When you asked me to set up a website, you did not make it clear that it would mean I should stop posting the stats here. The above technical challenges would make it very hard and expensive to run a live website with this kind of stats, which leaves us with the choice of posting the same reports in a WordPress blog or something similar outside of Steemit? Which does not make any sense to me, because Steemit is a blogging platform in the first place, let alone the fact that it would be driving traffic out of Steemit to an external blog. So the feeling I get is that I’m being kicked out of Steemit because my posts are successful?

The next point you mentioned is that my posts’ reward was too high, this is certainly a very subjective point, which could result in some very long byzantine discussions. But my main point about it is that I have followers and people who really like my posts and upvote them as you can see in the numerous comments of this post, and my posts are not in any way abusive or obscene or anything similar that would deserve a flag especially one that nullifies votes of literally hundreds of people and one that is coming from one of the founders of this platform.

It seems many people here do not agree with the systematic flagging of my posts.

I’m an early member of this community, and I have the utmost respect for all its members including the normal users, the founders, the big stake holders, the investors and the staff. I have been contributing to it in ways I deem to be useful to community and I also have some ongoing projects for it. I certainly wish it big success as it would mean the success of everyone here! But the last thing anyone here wants is to see it turn into an Orwellian platform.

Sort:  

You can make it more efficient by parsing only last 24h active accounts, which is about 5k, instead of parsing every 95k accounts

I invite you to do that, because it will take longer as you will need to determine the active accounts first (by parsing the same 95k accounts and not blocks, as blocks do not have virtual ops information). It would be interesting to see how you will solve this problem. It will also allow you to get a better idea about the difficulty of this operation.

Identify the accounts by monitoring blocks then after midnight (utc) you can begin to parse them. I will try!

Not all posts get paid within exactly 24 hours some live more due to late big votes, also the second payout occurs after 30 days. Your method is not accurate and will probably miss some payouts but I invite you to try!

@masteryoda
Ah right. Instead you can make a 30+a day active account list which has about 20000 accounts. I am working on in.