Steem History: March 23, 2016

Originally, I wanted to start digging into the source code for Steem to begin to understand how the underlying mechanics work, but while doing some digging, I began to uncover the rich history behind the blockchain.

I couldn't resist taking a detailed look of a day that will live in Steem infamy. Yeah, we're going back to the early days of Spring 2016 and going through forum post by forum post to uncover Steem's dark secrets during the early days.

Image Source

The Day Before

So, if you go to the Steem Github and search for the first commit in the commit history, you find that this commit was published on March 22. A character that we'll call "Reverse Flash" published 149 files and over 80,000 lines of new code in a single commit. While many developers (including myself) consider this to not be very helpful Git usage, it is fascinating that an individual or group of people would wait until the day before launching before publishing their entire codebase to a public repository.

While I'll probably look deeper into this commit on my free time, I decided to take a quick look at the README and see what the outside world would be able to figure out about this mysterious new cryptocurrency.

Steem is an experimental Proof of Work blockchain with an unproven consensus
algorithm.

The most interesting thing about the first sentence of the README of the Steem blockchain was that Steem was advertised as a PoW blockchain with an unproven consensus algorithm. Interesting how quickly things can change in less than three years.

No Pre-mine, No ICO, No IPO

Another interesting nugget of information here. So, before Steem launches, the author(s) of this code intended there to be a fair launch open to the public. A little bit of ironic foreshadowing?

Code is Documentation

Rather than attempt to describe the rules of the blockchain, it is up to
each individual to inspect the code to understand the rules consensus rules.

Well then, fascinating. Not really the best approach to making a user-friendly system that maybe people will want to develop in the future. It is almost like this sentiment continues to haunt the blockchain to this day. From this statement, Reverse Flash (and some of his friends) is either being sneaky or lazy. I understand the lazy aspect to this. There are plenty of times I didn't want to write documentation, but again, these sorts of decisions catch up with you in the future by creating learning cliffs and mass confusion.

Reverse Flash proceeded to make eight much smaller commits to tidy things up before the big day. Steem was getting ready to launch and things needed to go smoothly in order for the blockchain to be perceived with a successful launched. Unfortunately, Murphy's Law reared it's ugly head at the worst time.

A Wild Steem Appears!

At 1:13 AM, a thread now named [ANN][STEEM][POW] - An experimental Proof of Work blockchain is spawned by Reverse Flash. The initial post has been edited multiple times, so we start moving through the story from the posts that show up afterwords. I'll cite some interesting posts, but I highly encourage you to read the thread to get the full experience.

One key thing to note is that Reverse Flash makes the statement that mining has started in his original post. So, before any time passes, Reverse Flash and any associates involved in the Project are currently mining blocks.

So, what's the first response to this experimental new PoW coin?

NextGenCrypto posts at 1:18 AM:

Checking this out. Could ya provide a little less info on how to get started?

Within the first five minutes of this announcement post going up, we already have criticism of how much information is given to users upfront. Nothing like code is documentation, right? This thread begins to heat as more users begin to critique Reverse Flash for not being user friendly. In less than an hour, Steem's perception isn't looking too good:

blackchopper posts at 1:43 AM:

It's called an instamine. Since people have to read the code to figure out how to use your coin and mine it, it's no different than someone releasing a coin without wallets where people have to compile from source and the developer gets to mine a ton of coins while everyone else has to either figure out how to compile the wallet or compile the wallet.

Less than a hour after starting the thread, Steem was already being called an "instamine". And the criticism is valid. If you want to decentralize a coin and give people a fair start, you should give people enough information to make that start fair. People can't simply jump into code and understand things within a matter of minutes. But good documentation takes time and effort and that simply wasn't given here.

We then move into discussion between Reverse Flash and other users on how to set up their miners and some information on the hashing algorithm used and how the PoW works. We then get this interesting nugget:

thereverseflash posts at 3:56 AM:

Most of what we are mining will be given away in the future, so what ever concerns there are about our early mining dominance will not matter in the long run.

What is most interesting about this post is the notion that the owners of this advantaged stake planned on giving it away in the future. So, maybe Steemit can start sharing some of it stake since it was to be given away in the first place... But maybe we can talk about those issues later.

Reverse Flash continues to talk for another hour about technical details with some of the other forum members, with complaining and confusion mixed into the equation. Reverse Flash then leaves the forum for awhile to catch up on some sleep. While the launch has been met with a rocky reception, there's still hope for things to turn around. Disaster hasn't struck yet.

Disaster Strikes

At some point, during the early morning hours, the mining node run by Reverse Flash crashes and the chain is unable to produce more blocks. Chaos ensues.

Eclipse Crypto posts at 8:45 AM:

This has been a monstrous shit show. The dev crushed the difficulty and turned off his miners. Now the chain won't move.

At this point in the morning, new blocks are not being produced which is never a good sign for any blockchain. What makes it worse is that this is occuring during the launch of the chain that many are already questioning. Unfortunately, Reverse Flash doesn't show up for another four hours.

thereverseflash posts at 12:44 PM:

Ran in to some technical difficulties while I slept. My mining node crashed, the first time that has happened in a really long time. Everything is up and running again.

So, the Steem blockchain was down for several hours. But of course given the lack of information to forum users and the sheer amount of confusion, users begin to suspect malicious intent around this suspicious behavior.

Eclipse Crypto posts at 2:29 PM:

Why was no one getting any blocks while your miner was down? You are screwing people over by not telling them how to mine. We should be able to mine without your miner. I have a feeling you are getting all the blocks and no one else can mine because of some technicality we are all missing. Why didn't you just premine all the coins and save us all a lot of time? I literally had 20 cores on this shitcoin for 6 hours and didn't get a single block, even while no blocks were coming in from your miner or anywhere else.

Nobody was able to mine a single block while Reverse Flash's miner was down. Which is obviously an issue. It effectively isn't working as a blockchain if other miners can't produce blocks if another miner is down. Any good intention intended by this launch has backfired and we can see why some people are so annoyed with how this coin started. Because lots of things are going wrong at this moment.

Reverse Flash attempts to explain this behavior:

thereverseflash posts at 3:30 PM:

At the start of the chain, "initminer" is the only node that can produce blocks, but it does not get paid. In fact, no one gets paid until 21 miners solve a POW.

thereverseflash posts at 3:58 PM:

At the time I controlled all 21 witnesses in the active round.

So, I far as I understand due to Reverse Flash controlling all of the witnesses and his node then failing, this prevents any new blocks from being produced. Nothing like an edge case showing up at a sensitive moment in the early moments of the blockchain.

It isn't until more than 12 hours after launch that another account is able to confirm that they have gotten Steem from block production:

STEEMD posts at 4:16 PM

im mining with 6 cores and i got around 70 STEEM in total, no problem here. thanks dev

But, even then, folks are skeptical. I mean just look at that username.

NextGenCrypto posts at 4:25 PM

Yea and just felt the need to create a new shill account rather than using your own?Good try "dev".

So, while the blockchain appears to now be back up. Massive damage has already been done. So far we have had a masterclass in how not to launch a blockchain. So has Steem's reputation been damaged to the point where it can't be repaired?

Damage Control

At this point several miners are unhappy, very few miners appear to have actual Steem, and there is still a lot of confusion over the situation. So, Reverse Flash attempts to give some more transparency about his motives:

thereverseflash posts at 5:08 PM:

let me explain a bit about what is going on with this launch:

1. We want to have a large amount of STEEM to give away via faucets
2. We don't want to spend a lot of money competing with miners just so we can turn around and give it away
3. So we have intentionally made the launch unappealing so that we could execute a longer-term strategy.

What this means is that anyone who reads the code and identifies the value within has a huge opportunity, you can get STEEM the cheapest you ever will be able to and therefore are rewarded for your efforts.

...

We have instead opted to go for public mining but low information, which means we are giving savvy miners and people willing to look at our code a HUGE advantage.

So, bad documentation was all part of the strategy. Basically a pre-mine with the appearance with a public mining without giving away any information. And to add insult to injury, an edge case stopped anyone from mining on it for several hours. And to sneak in a point I echoed above, they still have most of that stake (well not THAT specific stake, but we'll get to that).

Activity begins to die down a little bit, and outside of STEEMD, the other forum users are already calling for the end of Steem. One could consider this Steem's first obituary, less than 18 hours after the thread even started:

Eclipse Crypto posts at 6:35 PM

I'm officially reading the last rights. Dev you might as well turn off your miners and abandon ship. It will be courteous of you to say your farewells instead of just disappearing.

After some more arguing about the intent behind the launch, we then get this nugget from Reverse Flash:

thereverseflash posts at 7:12 PM

The purpose is so that we can mine this coin publicly and in a manner that gives everyone a chance to participate. If no one else chooses to mine because it is too difficult, too little information, or no interest then that is fine. So, if this approach is not for you, then please leave and let those who are curious explore the code and get involved without a dispute about "how" things should be done.

Basically, Reverse Flash doubles down on his position. Not the best PR move, but definitely bold. Now I see where Steemit gets it marketing skills from. On one hand you can understand them wanting to get funds for developers and to find developers willing to jump off that learning cliff, but with such a move there is always a risk that you'll alienate the public. And the scars of the decision have hung around Steem's reputation since.

For the rest of March 23, it appears that only two accounts interact on the forum (other posts may have been deleted). STEEMD (suspected by the public to be a sock account) and Reverse Flash. The two talk about Steem implementation details for the rest of the night. Steem's darkest day comes to an end.

The Aftermath

On March 24, an integer overflow bug is found and due to the anger over the initial launch, Reverse Flash decides to restart the entire chain. There is some dissent over this move, but the old chain dies and the Steem you all know today lives. Another bug is found soon after but the chain is not restarted. Things appear to have stabilized after this point.

But there are some key takeaways from this story. The first being that Steem has a history of not being user and developer friendly. The whole reason I went to the first commit was to see how this whole thing was started. But I then found that whole commit and that lovely README that did not give me any helpful information I was looking for. The next is that Steem has a history of cutting corners. To have multiple bugs show up after launch shows a lack of quality assurance. That culture has persisted in the number of hardforks and the rocky upgrade to HF20.

But those who don't learn from the past are doomed to repeat it, right? So, I think there are some lessons from this whole ordeal:

1. Documentation is important.
2. User Interfaces are important.
3. Reputation is hard to overcome.
4. Intentionally making things harder for others in a self interested way is a dumb idea.
5. Double check your code and stress test it while not rushing to push it to production.

Well then, I just wasted a Sunday night. Well, to stick with the culture displayed in this post, I'm not going to do the due diligence to proofread this post and live with the consequences. Such is the Steem way.

UPDATE: I ended up proofreading the post lightly prior to posting. Well, there goes me not following the Steem way.