A few months ago, I announced anyx.io, a high performance Full-Node API. The goal of this infrastructure project is to create a high throughput, high availability alternative to Steemit Inc's own API offering (api.steemit.com). Furthermore, the idea is to promote and increase the decentralization of Steem: if everyone uses Steemits' severs, we lose the ability to publicly audit the information they provide us, and moreover, succumb to to any censorship they might enact.
In this post, I wanted to outline a few of the things I've been working on since the announcement. To summarize, here's whats new:
- Hivemind full-node queries are now available at https://hive.anyx.io.
- Legacy API Support Continues at https://anyx.io.
- HTTP (non-ssl, port 80) support now offered.
- Improvements to custom middleware solution (Jussi replacement).
- Bugfixes (e.g. large payload issues).
1. Hivemind Support Now Offered
Steemit recently announced the use of Hivemind in production. What this means for application developers is that many API offerings have changed, and going forward, certain API's traditionally offered have become deprecated. In this sense, api.steemit.com no longer provides a "full node" as we are used to, but has moved to a different standard.
In the goal of increasing decentralization of API services, I have added a new stack to my infrastructure that includes a full hivemind node, accessible at hive.anyx.io. This offers a public alternative to the same semantics that api.steemit.com offers.
How this was done specifically will be explained in part 4, but one important note that needs to be made about hive is that it requires a steemd node to link into in order to build its state. I've noticed that some people offer hivemind nodes but are not clear about what steemd node provides the back end, and indeed, if one uses api.steemit.com to fill the information in the hive node, it's not really auditing the information. In my case, the steemd node it retrieves its information from is part of the anyx.io stack.
2. Legacy API Support Continues
Since Steemit dropped support for previous "full node" semantics quite quickly, many developers were caught unprepared and have not updated their applications yet. To support these developers, anyx.io will continue to support the legacy "full node" API for as long as it's economical to do so. (Please consider voting for me as a witness!)
In addition to the "full node" semantics, websocket support continues for legacy applications such as the desktop wallet Vessel.
Adding hive.anyx.io is intended to aid developers using my infrastructure to try out their applications with the new API semantics without having to rely on api.steemit.com. Eventually legacy support will likely end, and so developers relying on legacy support should try out their applications sooner rather than later!
3. HTTP Support Added
Previously, anyx.io was only reachable via SSL (https, port 443). The reason for this was due to a limitation of Jussi, Steemit's provided middleware. As I have dropped this and replaced it with my own middleware (see part 4), I now have also opened regular http (port 80) support.
For most users, you should continue to use https. Honestly, if you don't know why you would want to use http instead, you should certainly continue to use https. Only those who know and understand the trade-offs and ramifications of http-only should consider it.
That being said, for anyone testing latency as opposed to throughput as a performance metric (looking at you @holger80), testing http://anyx.io is preferable, as it is slightly more optimized for latency. https://anyx.io continues to be optimized for throughput.
For those that don't understand the difference: Latency is a metric of how quickly you can retrieve a response back after you request it. Throughput is how many total requests can be served. As an example, if 100 clients can retrieve data every 1 second, the throughput is 100 r/s with a latency of 1s. However, if 500 clients can retrieve data once every 2 seconds, the throughput is 250 r/s with a latency of 2s. At a high level, the entirety of the anyx.io stack is optimized for throughput, as the intent is to offer a public node that gives fairness to many, many concurrent clients.
4. Custom Router Development
As mentioned previously, I have replaced Jussi (Steemit's middleware solution) with my own custom solution. There were several reasons for avoiding Jussi, primarily:
- Lack of support for port management
- Poor performance in throughput
- Caching often too overzealous
- Does not support unix sockets
For the replacement, I built a custom solution in Golang that connects to steemd via unix sockets (this is I feature I added to Steem itself, here, for much better local performance and to avoid the tcp/ip stack where possible) and offers better performance in general due to being a compiled, static language rather than a dynamic one (how python operates). As an outcome, I've noticed a drastic decrease in timeouts compared to jussi, as all requests are served concurrently with excellent throughput.
For caching, my solution is less zealous and will attempt to retrieve new information as soon as possible. In general, this will mean more up-to-date results compared to solutions with heavy caching.
Finally, hivemind support was added to the stack. If your request is made to hive.anyx.io instead of anyx.io, the hivemind API calls (which can be found here) are intercepted by this middleware and sent to the hivemind stack, but any other calls will continue to the anyx.io stack as usual.
Notably, with this new middleware has come a few issues -- as semantics do not perfectly match those of Jussi (which many other API's offer). As such, this is a work-in-progress, so if you notice any discrepancies between my API node and any others, please feel free to let me know.
5. Bugfixes and Performance Improvements
A side note that's important to mention is that tweaks and improvements are ongoing! Certain issues like caching returning out-of-date results have been resolved (opting to be more sensitive to time), and some nginx edge-cases like payload size causing interference have been fixed (see this hivemind issue). I also recently added support for batched requests (of limited size) since certain application developers require it.
If you run into any issues, please feel free to poke me so that I can resolve them! The goal is to provide a feature-complete API alternative to remove dependency and reliance on Steemit Inc., and so any improvements I can make will help me reach that goal.
Like what I'm doing for Steem? You can read more about my witness candidacy here:
https://steemit.com/witness/@anyx/updated-witness-application
Then please consider voting for me as a witness here!
https://steemit.com/~witnesses
ǝɹǝɥ sɐʍ ɹoʇɐɹnƆ pɐW ǝɥ┴
Nice work! Do you plan to release your router as open source so others can use and improve upon it?
Yep, definitely. It's not actually that complex, but it was basically designed directly for my stack... so I'll clean it up a little first then throw it on GitHub sometime soon.
Great work! Jussi was a weak point so its great that you've designed a replacement. Looking forward to it being open source.
Cool stuff. We need people like you to keep Steem working as it grows. Thanks for all your work.
Posted using Steeve, an AI-powered Steem interface
Great work!
Just tested ur node in SteemWallet, works flawless. (as expected :D) - nice that you redirect any non hivemind calls to the "regular" full node.
Great! Do you actually use hivemind calls though? If not just use anyx.io :)
Good job. Hivemind seems good, but how does one use it? How are requests made, what are the methods/functions and their parameters to retrieve data? Can you help developers find information in this regard? The github for Hivemind doesn't even demonstrate how this can be used by an app/site, for instance. Thanks!
Indeed, the sparsity of "how to use it" is one of the reasons many developers were left out when steemit quickly jumped over.
Basically, it has taken many of the API calls, such as those from here:
https://developers.steem.io/apidefinitions/#apidefinitions-follow-api
And replaced the way they are responded to with a different program, in an attempt to save costs.
The GitHub shows which calls were replaced.
Otherwise, it should be the same semantics for using an API node -- it's the back end that a front end like an app talks to.
Awesome, I'll try to catch up just after going back from my trip.
Nice update!
This story was recommended by Steeve to its users and upvoted by one or more of them.
Check @steeveapp to learn more about Steeve, an AI-powered Steem interface.
Throughput is how many total requests can be served.
'Total' is volume. Throughput is volume per unit of time.
Your post has been good. I love it. This type of post further did.I would like to see more about this.
🎁 Dear @naeem61,
SteemBet Seed round SPT sale is about to start in 2 days!
When our started the development of SteemBet Dice game, we couldn’t imagine that our game would go so viral and that SteemBet would become one of the pioneers in this field.
In order to give back to our beloved community, we’ll distribute 4000 STEEM to SPT holders immediately after Seed sale. Plus, investors in this earliest round will be given 60% more tokens as reward and overall Return on Investment is estimated at 300%!
Join the whitelist on SteemBet webiste now and start investing! Feel free to ask us anything on Discord https://discord.gg/tNWJEAD
Congratulations @anyx!
Your post was mentioned in the Steem Hit Parade in the following category:
This post has been included in the latest edition of SoS Daily News - a digest of all you need to know about the State of Steem.
I have a question for @anyx. Is there any danger that my posts will get flag/vote/comment from @cheetah or @steemcleaners if I use @dlike or @share2steem?
Posted using Partiko Android
Cheetah might pop up if you copy and paste a lot of text, that's what she does -- finds copy and paste. She doesn't flag though, just comments.
Steemcleaners would only come after you if you're performing some kind of fraud like identity theft or stealing other peoples content.
Regardless of which platform you use, the above apply.
Congratulations @anyx! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :
Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
Congratulations @anyx you won the 11th place on Who makes the difference on Steem Poll!
https://steemit.com/dpoll/@psos/and-the-winner-is-s9nlytcf
Posted using Partiko Android
That's an amazing infrastructure! Community on Steem should vote for you! Snax wants Steem to grow, and to become larger than ever before and we want to help Steem. People can earn more money on their posts, with Snax and without leaving Steem or other socials:
https://steemit.com/steemit/@snaxteam/decentralizing-social-media-rewards-steem-and-snax-comparison
Dear anyx:
We are SteemBet, the next generation STEEM based gaming platform. We are honored to invite you to join our first fantastic dice game, which is just the beginning of SteemBet game series. Our dividend system has now launched. The prize pool has already accumulated 2,000 STEEM and more than 60 players have participated in staking mining token SBT. A huge reward of 40,000 STEEM is awaiting! Join us NOW with other 500 STEEM users to loot HUGE dividend reward!!
SteemBet Team
Official Website
https://steem-bet.com
Discord Server
https://discord.gg/95cBN3W
Telegram Group
https://t.me/steembet
Cheetah-appeals
Good morning, Anyx. I got your info from Logic, I tried messaging you in discord but I’m not computer savvy enough I guess, it said we didn’t share the same server?? Whatever that means. 🤔
Hey man, I’m here because cheetah hits our @foodfightfriday blog, a weekly cooking initiative, every.single.friday. I was told because our opening is the same on the page every week.
Will you please take a look at our initiative and call off cheetah? I appreciate what cheetah does, trust me! In this case, however, it’s negative energy that isn’t necessary, there’s no plagiarism going on, we’re simply using our own personal format. From there, we’re featuring multiple chefs every week, I’m the one responsible for putting our contender posts together, I have between an hour and two hours vested into each post, every week, and each week we’re interrupted by cheetah.
Thank you!
@anyx. @foodfightfriday, we’ve even began switching up our opening paragraph and, every week, the images are different chefs cover images, a new picture each week. What else do we have to do to get off @cheetah’s list.
Thank you, on behalf of @foodfightfriday.
I’m @foodfightfriday and I approve this message!
Cheetah-appeals
Hello @Anyx, we got your notice on our post today. It is quite unfortunate that we had complained about your comment and explained who we are and how we need to report to our delegators updates, membership status, number of SP the community has, leased SP and other details to you about 2 weeks ago when you first sent your comment. You stopped sending the comments but we are surprised again to see you comment on our post today.
We ask that you take a look at our post and remove it from your cheetah list. We understand what you do but you can't be harming other good initiatives that are helping to build the Steem community. We will appreciate it is this is done urgently.
Thank you!
Hello!
As far as the golos.io community is aware, you delegated your golos power to olegst for support the @kulturagolosa project.
And everything would be fine, but the problem is that the one to whom golospower, was delegated obviously abuses it and behaves like a victim of the “watchman’s syndrome” - (this is when a useless, but malicious and envious person accidentally get some power and he begins to use this power in order to revenge and settle accounts with old foes).
If you follow the activity, and in particular, olegst, you can clearly see that all of its flags are not a reduction of really junk content but the desire to harm those people who he does not like personally.. For example, the conflict with @cognition community and @lindsay:
https://golos.io/ru--megagalxyan/@cognition/analitika-21-02-2019-vecher
https://golos.io/ru--megagalxyan/@lindsay/zlaya-nastya-znakomimsya-zanovo-ili-kak-nachinalas-istoriya-o-dvukh-kapitanakh
In response to any attempt to object, in response to any request to substantiate the correctness of the punishment, Oleg responds in this way: he sets flags to all posts of this author. Rather, on all posts that are waiting for payments to show: "I have great strength, I will do whatever I want and no one will bother me"
Most of the community do not think that this posts deserves whale flags (its flag was the first, then with the support came his friends whom he called and only then the clash began in comments), and this is not the first time. This man asserts himself in such a petty way. - having gotten a whale power in his hands, he bullied a girl for the fact that once she once quite rightly told him everything she thinks of him. And provokes on purpose, knowing how emotional she is and how acutely it reacts to whale flags, which cutting off substantial amounts from the post and lowering the reputation. As a result, the situation turns out that a person is clearly wrong, but even to argue with him is impossible because his only argument is the whale flag.
This conflict is already beginning to develop into completely unacceptable forms, a critical mass of dissatisfied people is recruited, on the sidelines of the chats there is already talk of deanonymization and “courtesy visit” to Oleg in real life.
Also not very kind words are said about you, because you provided so much authority power to a person with such dubious moral and human qualities.
On the one hand, I am a friend of Nastya, on the other hand, I love your @cheetah very much and respect you very much for the work you do.
Knowing that you are an authority on both sides of the conflict, I beg you to play the role of the mediator until the conflict has gone too far and something bad has happened. And if you can not or do not want, then I ask you to at least think about whether your golos power is used as you would like it to be used?
Regards, @kamelotan
thanks so much for running a node, I see its an option on steempeak and good to know there are other options out there instead of using steemit's api.
Cheetah-Appeals
Hi,
your Cheetah Bot has blackliste my own content and is flagging me since february.
I am only copying my own disclaimer at the end of each post and content of my own website.
It is a bad idea to flag and blacklist users own content each time.
So it would be a good thing to remove me from the blacklist of this stupid bot who is blacklisting me, because I publish my own content of my website, even of the past years.
Best regards and thanks in advance.
IT24