[EN]
This post is intended to cover the server costs for SteemWorld.org and to enable me to continue my development in the future.
In case you missed the first part:
SteemWorld ~ Weekly Support ~ #1
[DE]
Dieser Post ist dafür gedacht die Serverkosten für SteemWorld.org zu decken und mir in Zukunft weitere Entwicklungen zu ermöglichen.
Falls du den ersten Teil verpasst haben solltest:
SteemWorld ~ Weekly Support ~ #1
Decisions Made
I'm going to order a few new servers in the coming weeks, one of them in the next days already. The current server that is handling the requests from SteemWorld is definitely not sufficient for the things which I've planned in the near future.
The coming 'Post Finder' requires much more SSD memory than I currently have available because I need to index much more data and my SQLite databases would eat my server to 100%. I will order two new dedicated machines / Xeon E5 (4c @ 3.7GHz) with 2x480GB SSD (Raid), 64GB DDR4 ECC 2133 MHz.
The blockchain is growing fast and a full-text index on the body column for all posts in the chain comes with some crazy requirements. My local machine was dead after indexing only 20%. SSD full, HDD to slow (would take months). Even if a large part of the coding for the Post Finder is done already I can't really test it yet.
The good thing is: I've found a high quality hoster that is way cheaper than my current one. The bad thing is: I need to move my data and reconfigure the whole setup.
Step One: Witness
Many people asked me why I'm no witness yet and my answer has been something like "I won't run any software on my server that I don't fully trust/understand". Maybe I'm different than most witnesses who just started the job via kind of a 'start witness' button. There is no automated way when it comes to understanding the core and I want to be able to react to unforseeable misbehaviours by myself. Quality always needs time and experience.
I'm currently writing some handy shell scripts for my coming witness servers so that I can install, update and configure the dockerized steemd instances in a comfortable way. My goal is to have an own 'Steem-in-the-box' like solution with some more possibilities for setting up the node configuration (memory management for example). I'm going to make it open source and publish the scripts on GitHub, when all works fine and tests/optimizations have been made.
Main Menu -> Node Setup
Set node type -> Exchange
Set account name for tracking transfers -> 'exchange-name'
Main Menu -> Start Node
Soon ;)
Step Two: Post Finder
As the contract with my current hoster will still be active for a month from now, I'm going to move SteemWorld to the new server in about three weeks. After that I can begin implementing the Post Finder.
In Between: Improved Node Request Caching
The new caching feature for account operations will be activated in the coming days. It will improve loading times for repeated page visits by caching the operations data and storing the last received op index in a cookie.
In The Making
Post Finder
The following images show how the new feature could look like, but there are already a few changes planned that are not being shown in there. As mentioned, I'm going to index the body content of all posts on my new server, so that we finally have a way to search for posts by text/words very fast. This feature wasn't planned from the beginning and it increases the server requirements a lot but I have to think bigger here and I want to be prepared for future additions.
Default Image View
Zoomed Image View
Links
EN | What is SteemWorld?
-> Welcome to SteemWorld.org!
DE | Was ist SteemWorld?
-> Willkommen auf SteemWorld.org!
Weekly Tip
We have summer, dude!
The sun won't visit you in your basement... ^^
Thank you for supporting my work!
Positive thoughts create a positive world. It all begins with connecting our thoughts with emotions. We are the creators of our world.
Much Love,
Test
Good work, man! We are using #steemworld everyday and enjoy the tool a lot! :)
Indeed, well done @steemchiller! :)
It seems you're moving to a highload environment. On this road you should reconsider your tech stack choices. SQLite is deffinitely not an option. At least you will need Postgres.
Also decomposition to a set of microservices helps. E.G. your main current server doesn't need to run the same host or even environment that Post Finder. And on some point of time you will hit a wall by data volume and should decompose your data for hot and cold storage. For example Post Finder probably won't be used for old posts so we can move old texts to Sta HDD or even Hadoop storage.
You run a cool service, contact me if I could help - I will do it gladly. Here or at Telegram @br0nevik
Supported with full upvote.
lieber Chiller, @krnel ist ein sehr guter und bekannter steemian, der soeben einen tollen artikel über deine steemworld herausgebracht hat. schau mal vorbei. lg
https://steemit.com/steemworld/@krnel/steemworld-org-my-favorite-steemapp
Ich danke dir für die Info @pawos!
Dann haben wir ja bald alle 'Großen' mit an Bord... ^^
LG, Chiller
Thanks you for what is already the most useful support site available for Steemit. And it just keeps getting better!
Most useful stats site by far! You have my vote, and witness vote for that matter.
Now I have been wondering about running a witness, you mention one click witness lol that sounds like me. Most likely just be a backup witness all said and done. Wondering about that affordable host you mentioned, will it work for a witness? I have a lot to learn still but I am getting a name for myself as founder of #OneLoveDTube founder and would love to be the OneLove witness 😎
Posted using Partiko iOS
Thanks man!
That's a good question and we will know the answer soon... :) I've thought about all this for a while and maybe it's good when someone makes the test by trying alternatives.
I often read statements like 'keep in mind that even established hosters may not work for your witness node' and many Steem witnesses won't take the risk of trying something new. In the end it leads to a centralized bunch of witnesses all using the same service...
Soon we know ;)
I have been thinking the same thing about centralized witnesses, proof is in the effect on the network when dealing with an “attack”.
Definitely going to try keep up with your findings on this front.
Posted using Partiko iOS
Great work as always! Glad to hear you're actively looking into setting up a witness... and I like that you're going the independent route with it. That is quality.
Great work as always! You deserve all support there is!
I use SteemWorld every day, multiple times per day. I’d be lost without it. Looking forward to even more incredible and helpful features 🙌🏻
Posted using Partiko iOS
Glad you are considering becoming a witness. My vote is standing by.
Also glad to see you putting... something on GitHub ;) maybe someday.... anyway, as I said last week and the week before, thank you for SteemWorld.
Wow! I'm so happy to finally READ your post, @steemchiller. Opened it several times from the loading thingie when I visit SteemWorld, but never clicked over to read before this.
I enjoyed reading your plans for development and appreciate your steady short to long term vision and step by step method of building and testing your offerings until you get them the way you want to... then go again at evolving it all again.
Thank you.
ABSOLUTELY! This REALLY made me smile.
I LOVE SteemWorld. Didn't find out about it until about six or so weeks ago but I now use it every single day and couldn't imagine not having it. Mostly for mentions but also searches and VP. I know I'm not using it as much as I could but my understanding of what it offers will grow. ;)
I look forward to you declaring your Witness. You'll have my vote. And in the meantime, gave you 100% of the little juice I wield and will support these Weekly Support posts from now on. Bless you and all that you're doing. Resteeming.
I hope you will be a witness and I'll be voting for you then I will tell my friends to vote too. I know many friends use your SteemWorld. Keep SteemWorld work well, please. Thank you.🎉
Thanks again for all you do :) And I appreciate how conscientious you are and why you're not running as a witness...yet :) Shows just what kind of guy you are :)! Cheers!
Good Luck! 100% Upvoted. If you start a witness server you'd probably get to top 50 in under 2 weeks. I love https://steemworld.org although I'm not a heavy user. Just know that there is always a witness spot waiting for you for all the goo work you have done :-)
It sounds like a lot of work what you plan to do to get your witness server up and running, but if it turns out anything like the amount of work you've put into steemworld, and continue to put in, it will be well worth the effort. Can't wait to vote for you.
I do have something to report. Under the account operations, I've been noticing that outgoing comments have not been updating as quickly as they normally do. The post count comments amount in the stats window has always been a little tardy, but now it's updating before the comment will show up in the account operations window. Sometimes, it doesn't show at all. A refresh of the page resolves that.
I thought I'd let you know about it in case you weren't already aware. I don't know if it's been happening on anything else.
Thanks for your comment!
Yes, I've noticed that too. It seems as if some of the public Steem nodes are having some issues currently. I didn't change anything on my end regarding to receiving the account operations. You could try to switch to a different node in settings. I'm currently using 'rpc2.steemwiz.com'.
The counts in the stats come from my server and normally there is a 1-2 minute gap between the refreshs, because my current server parses new blocks in a one minute interval. That is going to change to a 30 seconds interval (maybe even more often) as soon as I can use my own seed nodes for receiving new blocks.
Very good. Thank you. I have changed to the node you suggested and the comments are definitely showing up in the list much faster.
It sounds like things are going to get even speedier. Overall, I really do like the performance of your site. It take quite a bit of time to load some of these others, let alone navigate from page to page. Having everything virtually at your fingertips is just so much faster. I really do hope you can have a successful run at witness. Your expertise in UI and development could help out a lot. I feel like the end user is as much in mind and a concern on steemworld as all of the backend stuff we never see. I hope we can support you enough that you can keep it up. :)
I spend more time looking at my steemworld screen than steemit or any other tool on steem blockchain. Although I still havent used all the aspects of steemworld, i am looking forward to the updates.
Resteemed :-) Thank you.
Möglicherweise bin ich nicht die erste, die darauf hinweist. Aber das folgende (also daß Einträge der Historie mehrfach auftauchen) fällt mir schon seit ein paar Tagen auf.
Vielen Dank für die Info! Ja, das habe ich auch schon bemerkt und es hat wieder mit den öffentlichen Nodes zu tun. Ich selbst habe nichts an der Logik geändert. Vermutlich verwendest du aktuell 'api.steemit.com' oder? Du könntest in den Einstellungen z.B. zu 'rpc2.steemviz.com' wechseln und es sollte dann wieder funktionieren.
Steemit hat etwas bezüglich der Speichernutzung der Full-Nodes optimiert und verwendet jetzt RocksDB für die Account-Historie. Diese Verbesserung scheint noch nicht ganz abgeschlossen zu sein, da nicht länger eindeutige Op-Nummern zurückkommen und SteemWorld bisher daran erkennt, ob eine Operation bereits geladen wurde. Ich werde mich mal schlau machen und schauen, ob das schon auf GitHub gemeldet wurde.
Update:
Wie sich herausgestellt hat, hat es nichts mit der Umstellung auf RocksDB zu tun, sondern damit, dass die Operations-IDs nicht unbedingt eindeutig sind, da sie je nach verwendeter Node unterschiedlich sein können. Ich habe aber eine Lösung gefunden, indem ich eigene IDs mittels 32-bit Hash aus den Op-Daten + Block-Nummer bilde. Sollte also ab jetzt nicht mehr auftreten ;)
Erwischt. Dabei hatte ich den doch schon mal nach einer Fehlermeldung, die ich Dir gegeben hatte, geändert.
Nach der Änderung und Neu-Laden ist es jetzt weg ...
Das ist ein weiteres Problem, das ich demnächst endlich mal angehen muss. Wenn eine Node für ein paar Anfragen nicht erreichbar war, schalte ich automatisch auf eine andere um, damit überhaupt noch etwas funktioniert.
Ich brauche wahrscheinlich doch noch weitere Prüfungen für die Nodes und müsste am besten selbst die Möglichkeit haben die Standard-Nodes von hier aus sofort ändern zu können, da einige manchmal Montags und manchmal Samstags ihre Tage haben (ansonsten aber immer stabil laufen) und es leider bisher keine immer funktionierende Quelle für die Steemdaten gibt.
Wenn ich das Geld dafür hätte, würde ich selbst eine Full-Node für SteemWorld betreiben, dann hätten wir diese Probleme nicht...
You're the best! Thank you, as always, for all you do. Very noble of you to refuse the witness "button" until you feel as though you're fully ready. To be honest, I think you have a lot more to offer than many others, but I'll just sit here and patiently wait instead. Cheers! Thanks again :)
Hi, Steemchiller!
Like many here, I'm a happy user of SteemWorld, and it has earned a permanent open tab in my browser! The only problem is that I'm a screen reader user, and am facing issues with a few "buttons" on the site, such as the one to simulate SP, the one to cast a manual vote, among others. They can't be focused nor activated with the keyboard, only with mouse clicks. I've peered at them using Firefox's HTML inspector, and noticed they have 'role="button",' but only this isn't enough to make them interactable with the keyboard. It would need 'tabindex="0"' to be focusable with the "Tab" key, and also respond to the "Enter" key (keycode 13) and the spacebar (keycode 32) to behave like a native HTML button and be accessible to screen readers. Other "buttons" like the "coming curation rewards" one don't even have 'role="button",' and aren't really accessible, either.
I don't know how many screen reader users SteemWorld has, but if you could do something about these accessibility bits, it would be great!
Thank you in advance! And thank you for SteemWorld!
Thanks for your feedback! I can add the tabindex-attributes without any problem. Even if there are not so many screen readers using the site, I think it's important to be able to use all buttons without mouse. May take a few days ;)
Great to hear! I'll be waiting!
Should work now for most buttons. If you should miss the tabindex anywhere else, just tell me and I will add it there.
That was fast! :D
The buttons I tested worked like a charm. Thank you so much!
I did a quick check and found a few buttons that didn't get fixed. One is at the stats tab, the restore SP button that appears after we simulate SP. Another one is at the followers tab, the button to follow someone. And another one is the preview button at the mentions tab.
And then there are the account operations, coming author/curation rewards etc., but I assume they are different beasts, and a job for another time. :)
Thanks for your help! There are way too many buttons... :) I've fixed it for the ones you've mentioned. The refresh buttons in coming author/curation rewards are working for me but I guess you mean the collapsable items (headers) for the sections?
Yes, the beasts will take some more time and they will come later ;)
Great! Thank you for the fixes!
Yes, I mean the headers.
Supported! :-)
Best steemit tool there is ❤ I use it everyday
and recommend it often to steemians 👍
I always feel wonder how much dedicate you for work and this platform, it will always inspire me to do more something like you. Steem is now struggling with the current price but sooner it will be cover its deficiency in a matter of time.
You are doing a very nice work dude. Your works is appreciable. I am a regular user of steemworld.com. It gives me almost every information related to my account. Thank you. I am resteeming this post so that it may reach to many.
Thank you! I hope you meant steemworld.org, not .com ;)
Welcome dear! It was typing mistake. I meant steemworld.org which is my favorite.
Resteemen wir natürlich, damit die Serverkosten für das beste Steemit-Tool gedeckt werden können.
Yes, use your site on a regular basis to monitor my account. Really appreciate the work that you have put into it.
Keep up the good work. It's appreciated.
steemworld.org is the best, I have used it from day one. You have my vote.
Das beste accounttool überhaupt ! Mein erster Rat an alle neulinge: steemworld.org
Vielen Dank für die tolle Arbeit :)
You deserve all what I have
Thank you for your really hard work @steemchiller. always happy to support you as I can
Excellent application @steemchiller, I use it all the time, thanks
Steemworld is a very useful application, thank you for your effort.
Thanks for keeping up the work on Steemworld. It's the single most important tool I use for tracking my account.
Great work, @steemchiller!
Could you add https://steempeak.com in “Settings” / Steem Site“?
Thank you! 🙏🏻
Yes, I will add the site with the next update (today/tomorrow).
🙏🏻 Oh thank you: you are so kind! 🤗
Thank you so much @steemchiller for providing such a great service. Sorry to bother you on your post but i did not know how else to contact you about a potential bug.
I wanted you to be aware just in case it turns out to be a bug for all members.
I have a post that will pay out in about 1 hours. However, the 'coming author rewards' section of my page lists my nearest payout is 20 hours away. Here are some screenshot for you investigate.
Oh no not again... :) I thought the bug is finally fixed. It's timezone related. In my timezone it's working:
Can you please refresh the page and tell me if they show up now?
Edit:
I've changed some things regarding timezone calculations today (subtract 8 days instead of 7 days from current utc time). Please check if it will work for your coming posts. Thanks!
Sorry @steemchiller. By the time I got around to checking your reply, that post got paid out. I don't post often enough to have another one end soon, but I will keep an eye on it and will notify you again if I see it not working properly. Thanks again for your prompt reply. BTW, my setting had wss://rpc.buildteam.io as a default Steem Node butI changed it to wss://rpc2.steemviz.com as you recommended.
Thanks again for such a great tool, matey! I recommend it to all me pirate captain friends.
This is the most informative post and I think it also a big platform for steemains to know their currency report and compare the market price.. and always your data based work so useful and necessary for us.. I really appreciate your work that's inspired myself..
Love the site, very useful. Have you considered adding SteemPeak to the list of sites to open links? Or maybe even add the option to enter a URL of choice (assuming it supports the standard URI scheme of Steemit and Busy.org)
Thanks! I've added the site in settings ;)
Hi,
very good application, any error or something strange I tell you, to improve your software
And you would not be able to make your program work with golos.io ?
Thank you for steemworld, I use it every day.
I'd vote for you I think I am about 2 gVests or something close for my vote.
This post has been just added as new item to timeline of SteemWorld on Steem Projects.
If you want to be notified about new updates from this project, register on Steem Projects and add SteemWorld to your favorite projects.