[WITNESS] Missing blocks? Did you just get a "Core Dumped" error? Time to increase your shared memory!

in #witness-category7 years ago (edited)

Just a short post for all of the witnesses panicking right now.

IF YOUR WITNESS HAS NOT CRASHED, BUT YOU'RE STILL MISSING BLOCKS, IT'S NORMAL. Check out the witness-blocks channel on steemit.chat and you'll see tons of witnesses are now missing blocks. Seems HF19 is not a smooth upgrade. No known fix, but seems to be improving over time (possibly just a hiccup near launch time).

If you got something like this in your logs, and now your witness is dead:

terminate called after throwing an instance of 'boost::interprocess::bad_alloc'
  what():  boost::interprocess::bad_alloc
Aborted (core dumped)

You need to increase your shared memory file to 10GB instead of 8GB.

shared-file-size = 10G

Remember to make sure you have enough RAM for this!

Don't forget to increase your /dev/shm

If using steem-in-a-box

./run.sh shm_size 10G

NOTE: Steemit.Chat is down again. But you can join us on the Peace Abundance and Liberty Discord @ https://discord.gg/wzsaxvC

Do you like what I'm doing for STEEM/Steemit?

Vote for me to be a witness - every vote counts.

Don't forget to follow me for more like this.


Have you ever thought about being a witness yourself? Contact me on STEEMIT.CHAT, or just ask in the witness channel. I'm happy to guide you! Join in shaping the STEEM economy.

Are you looking for a new server provider? My company @privex offers highly-reliable and affordable dedicated and virtual servers for STEEM, LTC, and BTC! Contact me on STEEMIT.CHAT for more information.

Sort:  

Thanks for getting this important info out there! :)

Thanks for the notice @someguy123

When I started up my new witness nodes I didn't want to take anything on chance so I've allocated a fair amount of memory to the blockchain.

My two witness nodes are running with the setting
shared-file-size = 21G

I can currently see that they are using around 8.8G for the shared_memory file, and that's probably why some nodes crashed when the file grew larger than the allocated space.

Nice to see all the witnesses back to normal again :)

Chat has crashed

The blockchain seems to be up, but Steemit.chat has 502 bad gateway.

I'm not sure all of us had quite understood hat the new rules would be backdated to all existing votes not yet paid out. But that means the witness servers must be doing 7 days worth of calculations as fast as possible. Not surprised with the hiccups.
Good work in keeping us informed.

What is the problem. Are we putting smiles on people's faces? Say cheese 😊😊

The face of a wealthy man reacts in a slightly different way to that of a regular man. It's barely noticeable, however the time reactions of that smile reveal it all ;) Steemit's making a lot of people vurrrrry happy :D

Crash reason confirmed

The witness nodes are crashing on news that I am making money on Steem now! Noob @uruiamme for the win!

Thanks hard for the hard fork! I guess this is going to bring back the whales from hiding, because the newbies are seeing massive improvements in payouts.

Thank you witnesses for choosing to increase minnow voting power. I hope the other aspect of the voting (increase in recovery time) works for everyone.

Thanks for the critical information when it counts. I just noticed a simple comment upvote from me moving the needle eight cents instead of the usual 0-1 cents. I'm all for my minnow votes going further so long as the system doesn't have too turbulent a transition.

that's too technical for me....hehehehhe

Thank you and all the other witnesses for such a smooth transition to the new hardfork!
I hope everything is going well

thanx for keeping this transparent and for answering the witness on this issue. Also is steemit chat was down today? some posted up saying it is.

Yep, Steemit.Chat is having issues.

You can join us on the Peace Abundance and Liberty Discord @ https://discord.gg/wzsaxvC

one love to all the witnesses, congratulations on HF 19. hope this implementation takes the steemit community higher.

Your are simplifying the case for me here thanks for the educative write up

Good good dood steemit @someguy123

I like what you are doing... But just a query... What does it means to be a witness... And how can one help steemit community If one wants to? ... Thanks for the post though :)

U guys are really working hard.... Thanks for your efforts. Proud to have you all on steemit :)

The shared memory utilisation has massively jumped in the recent update. I have not seen any unintended missed blocks but I'm pretty way down on the schedule and get about 1 every 1/1.5 days.

I was out of the loop for 2 months and in that time the blockchain went from 9 to 14Gb in size, and before, I could run the plain witness in about 16gb, or maybe it was 24, but now really you need a 32Gb file and mine shows that 29gb is used, so I may have to increase it and replay in a month or two.

I am not sure how you get it running with only 10gb, I got an extra missed block because of a greater time doing replay than I expected. But when you see 9gb up to 14gb in 2 months, and based on what I saw then...

I have now experienced the joy of a computer with NVMe, and in 2 and a bit monnths time, I am going to deploy a witness on my own hardware on a business grade 100mbit connection. The good connection is the main thing people would stumble up against as to running their witness at home. The standart of internet here is so high, I even just saw that one of the big ISPs here has just announced they are rolling out 100Gbit GIGABIT! connections.

Being a witness is a hard job it would seem. Us users definetely tend to forget that. ^^

Looks very technical.. Could it be due to inproper gateway?

Thank you very much for this post. I have upvoted, please do the same thing for me to help each other

Great read. Follow me for the latest tech headlines from my paid subscription to the Wall Street Journal.

Congratulations @someguy123
You took 31 place in my Top 100 of posts

I had a core dumped today while using 12G RAM, increased it to 15G.