[EN]
This post is intended to cover the server costs for SteemWorld.org and to enable me to continue my development in the future.
In case you missed the first part:
SteemWorld ~ Weekly Support ~ #1
[DE]
Dieser Post ist dafür gedacht die Serverkosten für SteemWorld.org zu decken und mir in Zukunft weitere Entwicklungen zu ermöglichen.
Falls du den ersten Teil verpasst haben solltest:
SteemWorld ~ Weekly Support ~ #1
Updates
I'm in the last steps of switching to the new server and my time is running out, so I will keep it short today. The plan was to start the new server directly with my coming SDS (Steem Data Service), but even if I made great progress in the past week, it is not ready to be used in production mode yet, therefore I need to setup my old PHP based web service and use it as a temporary solution there. Otherwise many features like mentions and the follower history wouldn't work in the coming week.
I think it's a better way to have a smooth transition, optimize SDS as much as possible to get the best out of it and finally release it, when it's really ready and stable enough to see the world. SDS includes never seen compression techniques, so that the blockchain can be stored in a super compact format. Trust me, we will need something like this when Steem goes mainstream and servers begin to run out of disk space.
Some of you may know the XZ compressor (based on LZMA), which on a level 9 (extreme mode, param: -e) setting compresses blocks to approximately 1/8 of the original size. With my SDS it is possible to compress them to a size that is ~ 0.63 times the size of the best XZ compression result. I hope that I will find the time in the coming week for an additional post to explain how I achieved that and to talk about what I'm really building here.
For most SteemWorld users that's all not really important to know, because it has to do with laying the groundwork for the future and coming SteemWorld features on the server side. I hope you don't mind, but a few 'boring' posts may still need to come until the more interesting things and the new SteemWorld features will see the light.
Coming Open Source Steem Data Service (SDS)
Some basic points I already can state here:
- Runs on Linux, Mac and Windows
- Can be run locally or as public web service
- Enables
each of usto analyze and work with the Steem blockchain data in a simple and very efficient way (SQL and custom queries can be used too) - Works as REST API:
//[domain or localhost]/sds/get_block/1
=> returns block 1 as JSON
/sds/get_blocks/1-1000
=> returns 1000 blocks (1 to 1000) with one request in second(s)
/sds/get_followers/steemchiller
=> returns my followers
... - Syncing is possible via available public instances (very fast) or regular Steem nodes
- Provides automatic downloading of already parsed data (no need to start from block 1)
- No need to setup any database services (all file-based -> no layers in between -> lightning fast)
- Offers configurable datasets (if you just need a DB with all transfers for example, you can run an instance with only the desired parser module 'transfers' enabled)
- JS developers can easily implement own parser modules (just add a custom script in the modules path and the service will call the
parseBlock,parseTransaction,parseOperationandparseVirtualOperationmethods within your script for each block) - Also 'virtual operations' can be parsed (or downloaded from one of my servers that will provide all parsed datasets in nearly realtime)
- Highly compressed data (some crazy stuff and a bit of magic involved here, more details on this soon)
- Interfaces to other database systems planned
- Completely free to use, will be released under MIT license
In the making
- Open Source Steem Data Service (SDS)
- Post Viewer & Editor
- Different views / no limit in the posts overview
- Details for coming rewards (number of posts/comments, maybe a pie chart)
- Improved coming curation rewards overview
- RC costs calculation
- SteemWorld -> Open Source?
Links
EN | What is SteemWorld?
-> Welcome to SteemWorld.org!
DE | Was ist SteemWorld?
-> Willkommen auf SteemWorld.org!
Thank you for supporting my work!
Positive thoughts create a positive world. It all begins with connecting our thoughts with emotions. What we focus on is what becomes powerful. We are the creators of our world.
Much love,
Bitte melde dich bei mir im Discord! Wichtig.
Weiter so 👍👏
Thanks for all your hard work @steemchiller, and particularly for the thorough and measured approach you take to keep up and release product improvements when they are ready and working properly! Having worked with many in the IT industry, that's a great approach to have... because many tend to work on a build, and then "just throw it out there and hope," after which a long stream of breakdowns and patches follow.
So thanks for the work you do, and the WAY you do it!
SDS sounds very promissing. Looking forward for more details.
Awesome work as always.
'ditto'
First, thank you for all the awesome work you do...
Second, I have a question if you don't mind. I'm trying to use steemworld to "create" an account (I have successfully "claimed" many thanks to your tools), but I keep getting an error message when using steemconnect. Is the "create" account part working properly and if so, is there a post detailing how to use it?
Thanks if you take the time to answer, I appreciate it!
There seems to be an issue with the new
create_claimed_accountoperation on SteemConnect. I don't know if/when they will fix that, but I will temporarily remove the possibility to use SteemConnect for creating accounts on SteemWorld.Looks like the operation is being detected correctly and even the op's data is being displayed as expected before the actual signing process, so I guess it is just a small bug on their side.
How to create an account:
The new account won't have any SP, therefore it may be required to delegate a few SP to it. Private keys are always derived from the master key and public keys are derived from the private ones. To see all keys for an account at any later point you can use the Key Generator tool on SteemWorld.
Thank you very much for taking the time to clarify that... Very helpful for sure! :)
Have a wonderful day and again you're awesome!
Sounds like a foreign language to me - but I am happy that you are doing it :) And everything you have done so far is super useful even if I have no clue how you made it 🤪
I love foreign languages, LOL!
I feel @steemchiller like Leonardo Da Vinci: he seems enlightened by a superior and vast intelligence, isn’t it?
I can’t wait to see the results.
Multi-platform service and @steemworld Open source sounds really good: to me, it could be the time to learn how to programming effectively and efficiently!
Posted using Partiko iOS
🐕 Harf harrf! 🐾
Do you understand me? 🐶
Posted using Partiko iOS
you, I do understand. Easy - either feed me, pet me, or let's go for a run!
LOL, you are an expertise dog interpreter! 👏😍
Posted using Partiko iOS
You got that right 😀👍
Posted using Partiko iOS
" ~ 0.63 times the size of the best XZ compression result."
That literally sounds like the best thing since quantum computing was introduced! WTF?
With masses of data being produced every day, such compression is going to be insanely valuable. I hope that you will be able to sell this technology to some tech giant (hopefully not google or fakebook :D) and make a billion from that.
Always happy to support SteemWorld and your work @steemchiller! This community would not be nearly as enjoyable without SteemWorld to help keep track of everything. Really looking forward to seeing what you come up with as a post viewer/editor!
=^..^=
Thank you very much for your formidable work!!
Are you half superhuman being? My brain could never creat such complex endeavor!!
Best wishes.
Hey, I just tried to use SteemWorld to claim an account. It wouldn't seem to work for some reason using Keychain on Firefox. It was just forever saying that it was writing it to the blockchain, yet nothing ever showed on the chain.
So I ended up claiming an account with Steem Connect, then trying to create the account name via SteemWorld. Had the same problem with keychain on that step.
Tried Steem Connect then, and it just said "error" every time I tried.
Eventually had to disable it too and use just SteemWorld directly.
Not sure if it was just my bad luck, but thought I'd let you know in case there's some kind of bug with how the accounts are claimed.
Both SteemConnect and KeyChain seem not to work for creating claimed accounts with the new Steem API methods that I'm using. I had hoped it will resolve itself, but that's still not the case, so I will remove the possibility to use SC and KC for account creation. Using SteemWorld directly will always work.
I come to the same conclusion as many times before in my career: Implementing external solutions will in most cases set you back and hit the quality of your own product at some point in future... Lesson learned.
btw, even though I was able to claim 2 account tickets with my valid active key. It barks at me saying I don't have authority when I try and create the accounts with my active key...And I certainly have enough credits and it is my valid active key....
Well...they could fix it eventually...but you can't really rely on others all the time. Keychain also seems to work pretty crappy with SteemPeak and a few other projects. I'm sure eventually they'll get all the bugs ironed out. Well...most of them...
I can sympathize, We all have these travails...
I have the problem with amazon MWS all the time vs. eBay and NewEggs APIs....
They don't document at all, have no centralized organized place for there published revision history and don't inform anyone in advance about up coming changes that will break your software.
I find that people designing and programming in the B.S. Agile environments like Scrum make the worst design and technical support documents, plus alot of spaghetti bloatware.
That's the major disadvantage of decentralized open source projects for the most part.
There is a lack of a uniform vision that guides the product path, and internal and external documentation is a complete afterthought. That's why it's a mess an tough to figure out most of the time. Even for a guy like me who has written case tools, compilers, worked for hundreds of companies and knows at least 100 languages and, frameworks.
Committees are great for codifying standards after a shake out.
But during design and development the old saying "to many cooks spoil the broth comes into play".
Just my sense for someone who's been in the field since 1980. Yea this old dog has some bite...
on another been working on a group app for my team....
calling it the SteemConsole here's a screen shot (pre-alpha)....about to add a second status button that opens another tab that brings up your site for the user.
LMK if you'd like to get an Alpha Copy...
Erstmal wieder herzlichen Dank für deine unermüdliche, großartige Arbeit! Hier eine kleine Ungereimtheit, die dich interessieren dürfte, zusammen mit einem Verbesserungsvorschlag:
Gestern habe ich als Stammtisch-Organisator die Autoren-Rewards für meine Einladung und für den Last Call dazu rausgesucht, um diese an die Teilnehmer zu verteilen. Da der Payout für die Einladung 8 Tage her war, also einen Tag zu lang, um in der SteemWorld in den Account Operations noch angezeigt zu werden, habe ich in steemd nachgeschaut.
Ich habe dann aber gemerkt, dass ich den Betrag von 6,036 SP doch schon zuvor aus der SteemWorld notiert hatte. Nun zeigt aber steemd stattdessen in der Auflistung 6,039 SP an, in der Block-Detailansicht aber nur den zugehörigen VESTS-Wert (sh. Hardcopy). Ich vermute, dass es an unterschiedlichen VESTS-SP-Umrechnungs-Zeitpunkten liegt, da sich der Umrechnungsfaktor ja kontinuierlich erhöht.
Nun stellt sich die Frage, welcher SP-Wert der richtige ist (auch wenn es hier keine nennenswerte Auswirkung auf die Verteilung hatte, nämlich 0,0005 SP je Teilnehmer). Ich habe den Wert der SteemWorld verwendet. Toll wäre es, wenn auch bei den Finished Posts die Payouts neben den $ auch in STEEM, SP und SBD stehen würden – und am besten auch noch die VESTS!
Vielen Dank für die Info! Ja, das hast du richtig vermutet, also die ausgezahlten 'vesting shares' ändern sich nie, aber natürlich die daraus errechnete SP je nach aktuellen Umrechnungswerten der Blockchain. Ich denke auch, dass der Wert zum Zeitpunkt der Auszahlung der wahrscheinlich 'richtigste' wäre.
Was die angezeigten Summen in den bereits abgerechneten Posts betrifft, bist du an die Grenzen des derzeit mit der Steem API Machbaren gestoßen und daher kann ich da noch nicht alle Werte anzeigen, wie ich sie gerne hätte ;)
Das ist unter vielen anderen ein Grund dafür, warum ich jetzt an einer eigenen API (SDS) arbeite. Ich habe geplant, zum Zeitpunkt der Abrechnung eines Posts die genauen Werte für diesen Post in meiner Posts-DB zu speichern. Dafür werde ich die virtuellen Operationen parsen und für jede nach 7 Tagen eintreffende 'author_reward' Operation die endgültigen Werte übernehmen.
In ein paar Wochen wird SteemWorld über die neue API laufen und die vielen Grenzen sind dann endlich aufgehoben. Auch die noch fehlenden Summen für die Allzeit-Rewards kann ich dann endlich reinbringen. Bezüglich der Account-Historie bin ich mir noch nicht sicher, ob ich das auch über meinen Server laufen lassen kann (könnte aus Speicherplatzgründen eng werden), aber es hätte seine Vorteile.
Mit der jetzigen Steem API kann ich nämlich nicht einfach die Operationen eines bestimmten Datums ermitteln sondern nur rückwärtszählend vermuten/prüfen, wo ich mich befinde, was auch der Grund für die aktuelle 7-Tage-Grenze ist. Bald wissen wir mehr :)
Danke für deine schnelle, interessante Antwort, und den 100%er!
Wenn ich das als Laie richtig sehe, müsstest du über die Blocknummern alles auf 3 sec genau ansteuern bzw. prüfen können (natürlich nur, soweit sie zur Verfügung stehen).
So kann ich die einzelnen virtuellen Operationen ermitteln (woraus sich eine Account-Historie aufbauen lässt) aber leider nicht gezielt die Account-Historie für einen Benutzer. Es würde also bedeuten, dass dein Browser alle Operationen downloaden müsste, um danach die für dich relevanten rausfiltern zu können.
Die Daten aller Steemians für eine Woche zu laden liegt im GB-Bereich und es würde die Nodes ziemlich überlasten, wenn ich das für jeden Benutzer auf SteemWorld machen würde. Der API-Aufruf
get_account_historyliefert mir gezielt für einen Benutzer die letztenXOperationen ausgehend von der gewählten Start-Operations-IDY. Die ID ist nur ein Index (der auch von Node zu Node unterschiedlich sein kann) und sie ist nicht an ein Datum geknüpft.Ich denke, es führt nichts daran vorbei, einen eigenen Index zu führen der die einzelnen Operations-IDs mit einem Zeitstempel verknüpft oder (die bessere Lösung) die gesamte Account-History per User zu speichern und mit einem Datumsfeld zu versehen (was ich wahrscheinlich tun werde).
Danke, dass du mir/uns trotz deiner knappen Zeit so ausführlich so interessante und laienverständliche Infos gibst! Mir ist da ein On-demand-Verfahren als Alternative (?, vielleicht ja völlig abwegig) in den Sinn gekommen:
Wenn ein User seine SteemWorld-Seite aufruft oder die autom. Aktualisierung ansteht, wird eine SQL-Abfrage o. ä. (ggf. mit userdefiniertem beliebigem Zeitraum, aber wohl umfangsmäßig begrenzt) direkt an der Datenquelle – also wohl bei den Nodes – ausgelöst und nur das Ergebnis zurückgegeben. Hab natürlich keine Ahnung, ob sowas möglich und/oder wirklich sinnvoll wäre.
Vielen Dank für deine tolle Arbeit, lieber Chiller!
Und gutes Gelingen bei der weiteren Entwicklung!
Diese SDS-Sache klingt ja richtig spannend.
To the contrary of what you said, the boring parts are important if those of us who do not code are not to be left behind.
I read through all of it, despite not understanding most of it, but the brain is an odd thing; it takes tiny bits it can make sense of and stores them and the next time we read, it adds a few more tiny parts, until we reach the stage where we go 'aha' that is what he is talking about, or else, you make the service or program available and it is just that little bit easier to to learn.
Thank you for taking the time to help us grow along with you.
lol, I code alot and don't understand half the gibberish most programmers write.
And I have a masters in compiler design and construction...
if your not in the room working on that little chunk of flotsom, why the hell would you know what they are talking about....
It's not you, it's gibberish...lol...
"It's actually a cry for help because they are lonely working alone and need to vent..."
They need to get more self actualized (to many of them don't ind this out till they are to late in life)...
So, I find it best to say, boy that's interesting smile
and change the subject to women, sports, poker, music, family or food...
Or if you want a blank face Politics or Religion....
A pity my upvote is under 2c so you get nothing, for I enjoyed reading your comment.
When I was much younger, I had a Sinclair Spectrum and I learnt to use Superbasic and make little programs and games. Does that mean I am entitled to call myself a programmer?
:))
ǝɹǝɥ sɐʍ ɹoʇɐɹnƆ pɐW ǝɥ┴
i was struggling to read this, till i fell off my chair. now it all makes sense
Great
Exciting stuff @steemchiller.
For me, while all of it is very interesting (not to worry about it be "boring" ...), this is of particular interest to me. Specifically, to find out exactly when you will be enabling us to do with it.
Thanks for the great update and all you do!
the best to manage our accounts :)
SDS? Obergeil!
Da bekomme ich ja richtig Lust den Deutsch-Report komplett neu zu schreiben. :)
VG Max
Thank you!!!
I really hope you get what you need I love this project 😁😀😁
Posted using Partiko Android
Thank you 🙏 for the work that you do for all of us. @SteemChiller
Posted using Partiko iOS
brother, I want to congratulate you.
I feel a lot of admiration for your work. I did not know steemworld.org
I discovered it a short time ago and it's great.
I thank you for developing this valuable tool.
@steemchiller you the best. Steem is not the same without you. I got a upvote for you when I can - Always! Your service is awesome! I don't know all the techy talk you be writing but, ROCK-SHOCK-ALOTTA UPVOTES REPLY & RESTEEM!
I am impressed at how you are always focusing on improvements. I don't understand some of what you talk about being I'm a non techie, but am hopeful that they will be able to use your compression system to further offset their costs of running things. Starting to think they (Steemit) should have tried to hire you (or still try).
Hi @steemchiller friend.
as always supporting his great work. I regret that I can not have a great power of vote to deliver that 100% that you deserve for your excellent work with the community.
A big fraternal hug from Venezuela.
Hi @steemchiller!
Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 6.239 which ranks you at #237 across all Steem accounts.
Your rank has not changed in the last three days.
In our last Algorithmic Curation Round, consisting of 187 contributions, your post is ranked at #3. Congratulations!
Evaluation of your UA score:
Feel free to join our @steem-ua Discord server
thanks for info
Wanna give it a go! Always making huge progress man... i know it’s crazy the workload behind this. Amazing stuffs.
Posted using Partiko iOS
Thank you for what you do!
No clue what you just said, but it sounds really impressive! Thank you!
SteemWorld open source? Sounds great!
Keep going, @steemchiller! Good luck with your development project. Thank you for your services.
Muchos existo, en la nueva incorporación de tu servidor. Soy fiel usuario de tu pagina, lo uso a diario. Estuve ausente, he tenido full de trabajo, buena suerte amigo.
Danke Dir!
upvote&resteem
Very interesting info on the compression tool. Sounds promising.
The last couple of weeks, calculating the rewards got a lot slower. Do you know why?
Keep up with the excellent work! 🙌
I guess it has to do with the Steem node you are using. You can switch to a different node in settings.
rpc.steemviz.comis very fast and works well for me.You're right. Thanks for the help. That made a huge difference!!! 😀
It took the time down from 4 or 5 minutes to around 30 seconds or so!
Always a pleasure to support you with my meagre 100% upvote
Posted using Partiko Android
Hello @steemchiller. I have a question regarding some Steemworld data.
At the Rewards section there are two columns, Curation SP and Author SP. How are those number calculated?
You mean the row 'All Time'? They are not (yet) being calculated by me and I receive them as shown from the connected node. With my coming SDS I will be able to calculate them and it will all make more sense. Currently the STEEM and SBD rewards are included in the SP, which might be a bit confusing, but I can't do anything about it.
Thanks for the info, I wasn't sure but that's what I was thinking too.
Thanks for the info,
I wasn't sure but that's what
I was thinking too.
- erikah
I'm a bot. I detect haiku.
Stop spamming.
This post has been just added as new item to timeline of SteemWorld on Steem Projects.
If you want to be notified about new updates from this project, register on Steem Projects and add SteemWorld to your favorite projects.
Thank you!
Thank you!!
Congratulations @steemchiller!
Your post was mentioned in the Steem Hit Parade in the following category:
Wow I can tell you have put in a lot of work @steemchiller. I found steemit yesterday and created an account. Is there somewhere, or a post I can find that explains all those numbers and steemit terms and goals so I can fully utilize this service?
Awesome! ...thought you might be coming to the Intro - Prosperity Party to celebrate new crypto-token trading system!
Nevertheless we FOLLOW @steemchiller to support your outstanding work.
Now URR the @steemchiller and to show our appreciation for the amazing work you do, we present this music gift - it's a money song written and sung by Eddie Watfins, Jr. - it's called "Money Is Coming To Me!"...it is the Cooperative Economic Organization anthem – contributed on YouTube by Florin Kis.
UPVOTE - REPLY&RE$TEEM TO URR DREAM @$TEEMCHILLER...
Steemworld is a TREASURE! Ahhrrr... Thanks for being a helpful landlubber!
We will be adding you to our upvote list!
