Your 1st Python Hive Bot Deployment

in #dev2 months ago (edited)

Context

This is for beginners.
For when your first Python bot or script for Hive goes to production.
I am not the right guy to post about clean code, but I've learned a few things.
I've made some mistakes and will make more, but maybe I can help you avoid some.

tipps.png

server ban

When querying from a public Hive node, that means using https.
The nodes are behind firewalls and other protection.
If you spam queries, you can get banned by these systems.
That's not cool for you and also it's friendlier to just avoid that altogether.

loop

To infinitely loop a reoccuring job, like a daily compilation of votes or something, cron can be good. Or schedule. But even then: If you have to make a lot of calls at once, maybe better include time.sleep(2). At least don't blindly iterate over pages of results.

3 second loop

I'll be writing a lot more about calling for a new block every 3 seconds (block stream).
For now: it's pointless to do the query more than once every 3 seconds.
The main loop should idle for the rest of the time. (time.sleep(3 - (time.time() - start_time)))

async vs. threads

While true: At the end it's just an infinite loop anyways, but asyncio.loop.run_forever looks nicer.
If it's the first time you see co-routines: Don't worry.
Forget about multiprocessing, btw.

signal interrupt

Using an example, I came up with this:

import signal, time

user_interrupt = False

def interrupt_signal_handler(sig, frame):
    global user_interrupt
    user_interrupt = True

signal.signal(signal.SIGINT, interrupt_signal_handler)

def thing():
    print("CTRL + C can't...")
    time.sleep(1)
    print("...interrupt me")
    
while user_interrupt == False:
    thing()

At least you can break out of the loop somehow cleaner.
Edit:

It's more important, when working with threads, and then you should combine it with threading.Events, like in this video

You can also use signal for networking and interprocess communication. Powerful.

Linux

At the end, you'll most likely want to install it all on a server and that's when you'll find that Linux is best for that. At which point you might as well use Linux layer itself to handle task queues and processes, like I already mentioned, when I wrote about cron. And the link, one paragraph above, even demonstrates network sockets.

That's taking it a bit far though; Most bots can probably live in a single main loop, but keep scalability in mind at least.

Still ontopic: terminal arguments could be useful.

Server Security

Talking about remote servers and their security is not something I should be giving advice about.
Still better than nothing: login via ssh, use Fail2Ban and ufw.

Errors

Here's a good video on error handling in Python.
In short: don't make errors.
Whether in development or production: if it's a critical error, it should probably crash and you need to fix it. If the error isn't critical, and you can't avoid it, catch and and make it a warning.

Storage

When your bot crashes, Hive probably keeps running.
When restarting, it can be important to know where it stopped.
Those problems can only be solved by storage and forget writing your own storage system with file writing and other such experiments.
SQLAlchemy (especially sessions) can make stuff robust and scalable. And once you understand it, you already have a strong foundation to learn flask next...

Service

It can be fun in the beginning, but servicing bots and managing servers and whatever 24/7 can be a nightmare. So before you commit, think about long term, too.
It can eat up hours and breaks at the worst moments and causes stress.
If it turns into a job, it better be worth it.

Conclusion

Don't be that guy.
Shit bots can cause trouble, so please be considerate.
Expecially when broadcasting: Once something was sent, it's out in the world (of blocks) forever.

That's all I could think of right now. I commented some place else that infinite loops are cringe, so I wanted to share some alternatives and considerations. In the links you should find helpful concepts to get you started on the right foot at least.

Please comment, if you want to add something or ask or whatever.

Sort:  

Here's a good video on error handling in Python.

A very well done video: do you use or reccomend using Merry? Also, the concept of bringing errors to the higher scope looks cool and it helped me understanding a bit more the use of "raise". Not that I am now able to implement it in my code, but at least I have a new concept I can try to (slowly) make mine.

SQLAlchemy

This is something I have to start studying, because you mentioned databases so many times and I don't know anything about them... and this is starting to be a problem too big to keep ignoring it.

Btw, in the snippet you shared, what is the signal that interrupts the script changing the boolian value False to True? An input from the coder?

what is the signal that interrupts the script

SIGINT (interrupt signal) is a predefined signal-> it's Ctrl + C
The way I use it ensures, that the program finishes thing() before exiting.
What SIGINT stands for was explained in the example I linked a line above my code...
It was confusing me too, when I first saw it: I thought SIGINT was some sort of integer.

do you use or recommend using Merry?

No XD

SQLAlchemy

This is something I have to start studying

It's really worth looking into. My SQL is ass. But I understand (for the most part) what SQL does and that's kinda important.
SQLAlchemy isn't necessary to use SQL, but makes it neater Python code.
I'll write a big long series about how I build things and why and I'll be using it, and once you touch flask and flask tutorials (by the same guy I linked the error video from) you can't avoid it.

I am not using SQL because it is fast (which it is), but mainly because the built in serialization saves me so much work. It's also easy to backup.

snippet you shared

I've thought about and edited the post above.
I now recommend this video:

SIGINT (interrupt signal) is a predefined signal-> it's Ctrl + C

I've seen that it's also in your snippet, but yesterday I somehow only red "can't...", and not "CTRL + C can't..."... nice, I'm blind 😅

I now recommend this video:

I think I might drop a follow to this guy: he has a lot of undervalued content I have to check!

yesterday I somehow only red "can't...", and not "CTRL + C can't..."...

Yesterday it said that. I edited it after replying to you earlier.

I think I might drop a follow to this guy: he has a lot of undervalued content I have to check!

He's got the best flask guide. Not in videos, but on his homepage.
When I first found it, I didn't really get it, because it's more targeted at people, who already tried doing the same things in other languages/libs or by themselves.