Explaining why there were problems with RPC nodes and what was the problem and why you must update your application.
Target audience
I explained everything as simple as I could. The main call is for the app owners to update their applications with the updated libraries.
After the recent timeout problems, it was necessary to take these steps and update the libraries.
I updated 2 major JS libraries, hive-js and dhive, to help resolve the timeout problems.
The "call" method
The "call" method was a way of talking to hived (the hive blockchain's application). This method got deprecated and removed from hived with the appbase update (HF19?).
So what has happened since then? Well, we were still using the "call" method for most of our applications. But the "call" method is not known to hived anymore, so the requests have to go through a proxy called "jussi". Jussi is installed in front of all the public RPC nodes.
Jussi takes the "call" method and translates it into the correct format then sends it to the hived. The problem is, you are adding an extra step for the API requests and it slows them down.
An example of the "call" method:
{"id":0,"jsonrpc":"2.0","method":"call","params":["database_api","get_dynamic_global_properties",[]]}
Notice "method":"call"
. So jussi takes that and translates it into the new format:
{"id":0,"jsonrpc":"2.0","method":"condenser_api.get_dynamic_global_properties","params": []}
See what happened? The user thinks the call was going to database_api
but actually, jussi translates it into condenser_api
.
To be honest, this is not the best way of doing things. The idea behind jussi was good but implementation was a mess.
What is the "new" format?
The "appbase" style calls are the calls that are supported directly by hived and don't need a translation.
It's not new and has been there for years but we didn't bother updating our libraries to use the supported methods.
With switching to the appbase calls, every request to the RPC nodes becomes faster.
This is the first change to both hive-js and dhive libraries. They both were using the deprecated "call" method.
You might not need jussi
Till now, hive-js wouldn't work with hived nodes until you added jussi in front of them. But with this update, hive-js will work with hived nodes. But remember, you will still need jussi for the hivemind calls. Also, there might be caching benefits in using jussi.
Broadcasting transactions
This is the important part. There are two methods for broadcasting a transaction.
broadcast_transaction_synchronous
broadcast_transaction
The first method sends the transaction to the RPC node, the RPC node validates the transaction, then waits for the transaction to be included in a block, then sends back a response including the transaction id and block number.
The second method sends the transaction to the RPC node, the RPC node validates the transaction, then sends an empty success response back.
In the second method, the library has to generate the transaction id locally. Which is way too much faster. In this method, the only way for a transaction to fail is expiration or micro forking maybe. So in most cases, it's good enough.
Of course, hive-js and dhive both were using the first method. When many people connect to the RPC node and wait there, it becomes problematic. The best way for handling huge traffic is to release a connection as soon as possible so other connections can be made.
Both libraries now use the second method for broadcasting transactions. You may already notice its huge impact on the speed of your votes and comments.
Peakd is already updated. Keychain is waiting for google approval and the mobile version will be soon updated. Some apps are catching up like Hivesigner, and Ecency. There was a problem updating hive.blog and that will be also updated.
Why bother updating
- Everything becomes faster for the end-user
- RPC nodes don't suffer overloading
- Apps become future proof
- It's easy as running a command
How to update your apps
hvie-js v2.0.1 - Release notes
Used in hive.blog, peakd, keychain, and many other apps.
npm install @hiveio/hive-js@latest --save
The minified JS for browsers:
https://cdn.jsdelivr.net/npm/@hiveio/hive-js@2/dist/hive.min.js
dhive v1.0.0 - Release notes
Used in ecency (mobile & web) and many other apps.
npm install @hiveio/dhive@latest --save
The minified JS for browsers:
https://cdn.jsdelivr.net/npm/@hiveio/dhive@1/dist/dhive.js
hive-tx v4.0.0 - Release notes
Used in keychain mobile and other apps.
hive-tx already is using the appbase methods and there is a function called .broadcastNoResult()
for the fast transaction broadcasting. But I did update hive-tx so now the .broadcast()
function is also updated.
npm install hive-tx@latest --save
The minified JS for browsers:
https://cdn.jsdelivr.net/npm/hive-tx@4/dist/hive-tx.min.js
Note:
@foxon and @emrebeyler confirmed that beem
doesn't need an update.
Scaling problems?
The recent problem doesn't seem to be a scaling problem. It's more related to the libraries and apps using the deprecated methods.
The recent library changes seem to be solving most of the problems. Although there are investigations still going on by @blocktrades regarding the timeout problems.
Hive blockchain in theory can handle much more activity than this. Some say like 1,000 transactions per second. But there might be minor problems because those scenarios are not actually tested.
Final words
I'm sure all the problems will be solved and we will see much better growth. I'm bullish on Hive and will continue my contributions as far as I can.
Thanks to the other devs who updated their applications on time to help with the problem.
Image source: pixabay.com
In general terms, everything is clear. Technical problems are being solved. This is good news. Everyone needs it. Thank you very much for your work.
@cherepin0512 you rock bro, you are correct. Everything is starting to flow better again.
I am sure this informative post from @mahdiyari will benefit others and bring peace of mind to many who have been frustrated.
Thank you for being a solid, constant, and just overall amazing person on Hive, Mahdiyari!
Thanks a lot for your work ! It's actually quite insane how much it improves UX
Already thought Hive is running into scaling problems, because of the success of Splinterlands and the growing number of transactions. Would be interesting to test, how much load the blockchain and api servers can actually handle.
@blocktrades said in a recent post that he thought his node alone could handle 12x more traffic. And i'm guessing he'll just add to it when it gets there.
There haven't been too many full blocks but if we get there I have also heard developers say they're willing to consider speeding up the block time. So if we go from 3 seconds to 1.5 then that seems like there is even more room for transactions.
As for peakd.com i've never heard the developer complain about us having too much traffic I guess i could ask at what point our servers would have issues with traffic. But as long as there isn't a crazy sudden spike i assume we'll be able to adapt in plenty of time.
But yes I would love to see some more specific tests and stats from people... i'm just using random things I've heard from others.
Thank you!
Does this mean @splinterlands can update their DHive and we can get our public nodes back now?
@yabapmatt promised to work on it. Might take a few days.
Thank you for the upgrades! I've reviewed all of my apps and made the corresponding upgrades for hive-js and hive-tx
Really appreciate your work on this update !!
I also want to thank you for the program for working on the blockchain Hive, Peakd. This is a really good job. I wish you success.
Peakd has been pretty solid. The automatic node switching function helped during this time. When it did not I was able to go to peakd settings and effortlessly manually switch to working nodes.
Also, I see I can view who is sending me HP delegation now on peakd, so thanks for the upgrade @peakd!
They are cool guys!
😎
Good job man!
I really appreciate @mahdiyari and @blocktrades or all other hive blockchain developers work, few days ago I starts faceing some problems while using my @ecency app, but after updating it from Google play store, I feel the difference all problems are solved, speed of the app also increased, now I understand after reading your article what's happening in the background development, thanks again nice work keep it up
Thank you.
// By the way: it's one of those things that were supposed to be done by
ned(may the Sun shine in his face) back in 2018. Well not done.Thank you for the detailed info, learned something!
In a scenario like eg.: a scheduled post. Would it make sense to still use
broadcast_transaction_synchronous
and would a success response pretty much guarantee a success or would you recommendbroadcast_transaction
, followed by eg. an attempt to retrieve the post? Or is a transaction 'guaranteed' only after its block number is less or equal to 'last_irreversible_block_num'?On a side node, I've recently checked the various implementations of client libraries and noticed that for broadcasting transactions some library chose 'ref_block_num' equal to value of
'head_block_number'
and other library uses'last_irreversible_block_num-1'
. Is there a difference and does it matter which block a transaction uses as a 'ref_block_num'?I'm asking in order to comprehend how things work so I can learn what the preferred usage is.
The synchronous method is deprecated and most likely will be dropped soon. So use the other one. The suggested API for checking transaction status is
transaction_status_api.find_transaction
. You can find it in the devportal. developers.hive.ioYes, when the transaction is included in a irreversible block, there is no way for that to reverse or fail.
AFAIK for
ref_block_num
it doesn't matter. It can be any recent block.Good job!
Good job !😍😍
The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.
Congratulations @mahdiyari! Your post has been a top performer on the Hive blockchain and you have been rewarded with the following badge:
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOP
Check out the last post from @hivebuzz:
$WINE
Congratulations, @theguruasia You Successfully Shared 0.500 WINE With @mahdiyari.
You Earned 0.500 WINE As Curation Reward.
You Utilized 5/5 Successful Calls.
Contact Us : WINE Token Discord Channel
WINE Current Market Price : 0.270
This might be off-topic, but I wanted to ask you this question. Is it true that 100% of an upvote goes to the voter, the curator, after 5 minutes on Hive? Back on Steemit, it was set at 15 or 30 minutes, something like that. I recently read on Quora.com and I think other places too that it is at 5 minutes currently. I don't know but that may be the case.
The curation penalty was removed in HF25. Read posts by blocktrades and hiveio regarding the previous HF.
Thanks.
Just want to note that for any existing applications a simple npm update is probably not going to pick up the latest version because of the major version change. Need to do it manually.
!LUV
<><
@mahdiyari, you've been given LUV from @donchate.
Check the LUV in your H-E wallet. (1/1)
I feel a little relieved but also worried, reading this after 3 years of the posting; because I'm starting to study the apis and went to https://developers.hive.io.
The tutorials and repos there are using the deprecated methos. And I'm afraid that new developers like me will likely come and take those as the recommendation.
Is there a way to update those tutorials and docs for developers?
Also, have there been any changes in the API since this post?
Things should work unless stated otherwise somewhere else.