How does TrendChecker work? | Steem-Python Bot Tutorial Part 1

in #utopian-io7 years ago (edited)

trendchecker_tutorial_logo.png

I recently released the bot called TrendChecker, which helps you to find out how high your post is in Hot/Trending/Promoted. If my bot receives a transaction with a valid url as the memo it checks where your post is in Hot or Trending and then sends your money back. The bot is completely build only with steem-python. This is a tutorial for people who are interested in the code behind the project and want to learn something about steem-python and how to develop bots for Steem. I will try to explain the backbone of this project and explain one aspect that might help you with your project.

What will I learn?

  • How to find out how high your post is in Hot or Trending.
  • How to check for incoming transactions.

Requirements

  • Python 3.6
  • steem-python
  • requests (only if you want to run the full bot)

Difficulty

  • Basic

Tutorial Contents

Setup

This is pretty simple because it is basically the same for every project: just import Steem and initialize the class. In addition, you will need the build in module json. In addition, I created two .json files: the first is for storing the account name and the private keys and the second is to avoid double spending. More on that later on in the tutorial.

image.png

Finding a specific post in Hot or Trending

Only a very small percentage of all posts reach the general top 100 of Hot or Trending and therefore we want to get the top 100 for every tag of our post.

To get the tags we need to use s.get_content(author,permlink). However, the user only sent us a full url of the post with this format: steemit.com/main-tag/@author/permlink. If we split this url every time a “/” comes up the author will be fifth element and the permlink the sixth one. The only thing left to do is to remove the “@” from the string with the author’s name.

After getting the content, we can finally extract all the tags. However, these tags are located in the json_metadata of the post and if we call post[“json_metadata”][“tags”] we will get an error. We need to load this .json “file” as a json first and then we can get the tags.

image.png

Sometimes users send wrong urls and we don’t want our bot to stop. Therefore we will put these few lines of code into a try: | except: construct. If something fails for whatever reason the bot will simply return "TRENDCHECKER error! wrong url or post doesn't exist...".

image.png




These are all the functions from the official documentation to get posts by some factor.

image.png

To try out which post they will return you can simply test most of them by going to the corresponding steemit.com url.

get_discussions_by_votes() --> https://steemit.com/votes/
get_discussions_by_active() --> https://steemit.com/active/
get_discussions_by_payout() --> https://steemit.com/payout/

Now we can get all hot, trending and promoted post for every tag in our list of tags. Then we will create 3 variables. Please note here that that could be done with only one variable! These variables will count what rank our post has in Hot/…. .

image.png

As the last step, we’ll create a for loop for the Hot, Trending and Promoted posts. If the author of the post in one of this loops is equal to the author of our post we will stop the for loop by writing break. If this isn’t the case we will add 1 to our counter (variable).

image.png

Note: While I’m writing this tutorial I noticed that checking for the author isn’t enough because if you released multiple post in a short period of time they all could be in Hot/Trending. Now If your second post was more successful it likely surpassed the first post in its ranking. However, if your looking for your first post you will instead get the ranking of your second post. To fix this we will need to check if the permlink is the same, too.



The beautiful ability of strings is that you can add them to each other similarly to numbers. For our memo (the message that will be send back to the user) we will start with a little “brand” message.

image.png

Now for every tag we’ll add (Tag): to the memo. If the tag is nothing (general hot/trending page) we will add All: .

image.png

If we find our matching post in Hot or Trending we will add str(rank) + “.” + “ in hot/trending/promoted”.

image.png

After we gone through all post in Hot/Trending/Promoted we will add a simple comma as a separation.

image.png

The last step is to return this memo which will now be send back to the user.

image.png


Checking for incoming transactions

Obviously, we want our bot to run without any breaks and delays. Therefore the first thing we have to do is an infinitive loop. The simplest one is while 1:.

Secondly we need to import one more steem-python class to get the incoming transactions: the account class.

It easily allows us to access account data. It needs to
be initialized with your account name and the Steem() class.

image.png

With the help of this class we now can get our account history which we can filter_by transfer. There is also another command for getting the account history which relies on the standard Steem class.

image.png

The only catch with this function is that you can’t filter it.

image.png

The -1 sets the order: the newest transaction/index comes first. 500 is the limit of the history. Getting 500 items from our history is definitely overkill and we would need much less. However, if the bot was in real use and went down for some hours it would still process most transactions and send the money back after it went back online because your transaction is still likely to be within our limit of 500. If the limit were 3 and I got 3 Bid bot advertisements in that time frame the bot wouldn’t see your transaction and therefore not send your money back.

Next up are the conditions for when we want our bot to act:

  • the transaction has never been processed before --> is not in our trx_list
  • the transaction was sent to me not from me
  • the transaction has a memo
  • the memo contains a url

If all these conditions are true we can be pretty certain that the user wanted to use our bot. Now we can add this new trx_id to our list of transaction so that it doesn’t get processed twice, generate a memo (discover where to post is in hot…) and send the money back to the user.

If you want me to cover these three functions write a comment down below.

Now we are basically finished, however, we want to ensure that our bot can run without any downtime. The bot sometimes failed at this part for whatever reason so we again put it in a try: | except: construct. If the bot fails to execute these previous lines of code he waits 30 seconds and the initializes the Steem() and Account() Class again. For the ability to sleep we need to import the build in package time.

With this code the bot ran successfully for 3-5 days straight. However, the bot stilled stopped once or twice with this error. If anyone could help me please write a comment or contact me on Discord (wil1liam#8715).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "trendchecker.py", line 118, in <module>
    s = Steem(node,keys=keys)
  File "/home/konstantin/steem-python/steem/account.py", line 34, in __init__
    self.refresh()
  File "/home/konstantin/steem-python/steem/account.py", line 37, in refresh
    account = self.steemd.get_account(self.name)
  File "/home/konstantin/steem-python/steem/steemd.py", line 168, in get_account
    return first(self.call('get_accounts', [account]))
  File "/home/konstantin/steem-python/steembase/http_client.py", line 223, in call
    *args)
  File "/home/konstantin/steem-python/steembase/http_client.py", line 243, in call
    return_with_args=return_with_args)
  File "/home/konstantin/steem-python/steembase/http_client.py", line 263, in _return
    raise RPCError(error_message)
steembase.exceptions.RPCError: Assert Exception:api_itr != _registered_apis.end(): Could not find API 

I hope you liked this tutorial! Leave any feedback or questions in the comments below!

You check out my previous work if you liked it!

To view the full code visit my Github repository.

Curriculum

Bot Development
Steemit Python Bots
Steemit Python Analysis



Posted on Utopian.io - Rewarding Open Source Contributors

Sort:  

Thank you for the contribution. It has been approved.

  • Awesome work !

You can contact us on Discord.
[utopian-moderator]

@rdvn Thanks man! and thanks again for the fast approval! :)

Hey @rdvn, I just gave you a tip for your hard work on moderation. Upvote this comment to support the utopian moderators and increase your future rewards!

Hey there!

A little tip for your coding style.

Here you are nesting way too many ifs.
It is alot nicer to read if you get rid of the indentions like this:

if(i["trx_id"] in trx_list["trx_id"]):
    continue
if((i["to"]!=acc_name):
    continue
if(i["memo"]=="")
    continue
....etc.... 

Greetings,
Martin

Thanks for the advice! :)

Really cool article. Too bad I only found it now. I added it to my steemit wiki: https://yoirtuts.com/index.php?title=Steemit

I will add you on discord. Did you end up solving the issue?

Thanks for adding it :)

I currently really busy right now so I haven't fixed the error yet

Hey man, great post! Most of it went straight over my head because i'm not really intelligent in coding. Nonetheless, I saw you were a teen. Me and a couple of the teens on this platform are banding together to make a teen community. I was wondering if you would like to join? We definitely need the coding help, and I think you would make an excellent member to the teen team!

@iamturtless I would definitely like to join!

Alright man, great! Just discord me and we can talk there. @Iamturtless#0853

If this is true you are a really cool kid @wil1liam. When did you start programming? I am really honored to meet you. My older son is in middle school and he gets sick of programming. I asked him to start with scratch and he doesn't like it.

@mineopoly I think I started 2-3 years ago with a little bit HTML and CSS . However, I never really got into it. Then last year I made a school project with 2 friends and we programmed it with Python. I couldn't write a single if statement at the time but after the project I started learning it, too... and now I'm here today.

Scratch may be an easy thing to learn although I would still not completely recommend it because I didn't like it back then as well as now.

I see. Thanks a lot. I know you are busy at school and with projects. I will try python. He doesn't see the usefulness yet.

Maybe he will release it one day but don't try to force him too much because kids often do the opposite of what their parents suggest (especially in puberty) :(

thank u very much!! i think u shouldn't of done that as it is dropping at the moment and keep it and wait till it goes up. i will try send you something when i gain more or remember. lol i put it straight in savings. i didn't realise it was from you till i scrolled down woops! very great idea. amazing your 16 keep going !!!

Thanks! :) and no please don't send me 0.001 back xD

lol its amazing ur doing this!!! hahaha xxx

wtf i just saw my wallet wtf !!!! thank u xxxx stop it !!!! lol xxx

omfg thank u!!!xxx

I just wanted to say thanks for your posts on bot development. I'm in the middle (ok, more like very early stages) of teaching myself python and your tutorials are just what I need to continue moving forward.

Thank you! I hope to see your bots one day...

Nice job, any ideas for your next bot?

Thanks! One the 5th I will post a development update for this bot with the design contest winner and the switch to a web based interface.

Furthermore I will launch two bots in the coming weeks:

  • minnow-incubator: a bot that rewards posts picked by the community
  • a bot that upvotes utopian.io posts that don't get much attention

Awesome dude, great work, I'm learning Solidity myself, and have dabbled in Python. I really want to write a couple of scripts, so I'm sure there's plenty I can learn from you!

Cheers
Cg

Thanks, man!

Solidity will be one of the languages of the future so I should pick it up, too!

i have up voted your 39 post you can check and now i am following you i support my followers and expect same from them and thanks for free money dear ,Its you you can have it any time when you need it. I am interested in steemit earning if you can guide me .It will be very help full.I am waiting for your reply dear

THANKS for your amazing support.

Generally you shouldn't upvote any posts that are older then7 days because after 7 days the rewards will be paid out to the author and after that the author can't receive any rewards from that post.

Oh I see i don't know that .thanks for telling me and sorry for bad typing .So could U guide me about steemit earning ?

@hashmieali77 sorry for the late reply! If you want to talk add me on Discord wil1liam#8715

ok i am also on discord,dear

Good work! Keep doing your awesome work, it will pay off ;)
Greetings

Thanks for your nice words! I will definitely keep posting 💪

Hey!
Sau cooles Tutorial! Sehr unterstützenswert!
Als ich in deinem Alter war, also vor 3 Jahren (:D), hab ich auch noch extrem viel programmiert.
Mittlerweile habe ich gar keinen Plan mehr..
Hast nen Follow ;)
Greets!

Danke Mann! Cool mal jemand deutschen zu treffen!

Hast du angefangen Partys zu feiern oder warum programmierst du nicht mehr? xD

Meine Hasslehrerin und 12. Klasse Theorie haben's mir versaut..
Außerdem habe ich jetzt andere Interessen.^^
Aber mal sehen wohin mich mein Weg führt ;)
greets!

Mies... meine Mathe Lehrerin macht mir auch gerade das Leben schwer.

Aber "andere Interessen" klingt ziemlich gut ^^

Ach die Deutschen sind hier stark vertreten ;)
Und weil davor mal was von wegen discord gesagt wurde D-A-CH Discord.

Auf alle Fälle mal vorbeischauen (falls du das noch nicht gemacht hast :) )

You seem like a smart dude who doesn't post shitty content. May be a good idea to quit sending memos, as most see them as spam.

You may want to involve in some discord communities, you may get more exposure that way.

As for the bot crash, why don t you use a process controll app? Just restart the thing , haha.

I've used pm2 in the past, and i just love it .Mostly used for node js, but if you specify an interpreter , python works too, i quess .

Thanks for the advice I think I should switch...

Yeah, I also should probably quit spamming memos and engage more but otherwise my reach is very low :( This issue really got worse with the invention of bid based voting bots. Now posts from normal people can't reach the top in Hot or Trending anymore...

Awesome work man, you got it right 🙋🙋🙋

Thanks man!

excellent post sir @wil1liam about steem python bot. Very informative for us. Thanks for sharing.

WARNING - The message you received from @iripilippopolis is a CONFIRMED SCAM!
DO NOT FOLLOW any instruction and DO NOT CLICK on any link in the comment!
For more information, read this post: https://steemit.com/steemit/@arcange/phishing-site-reported-uppervotes-dot-ml
Please consider to upvote this warning if you find my work to protect you and the platform valuable. Your support is welcome!

You got a 11.65% upvote from @slimwhale courtesy of @prometheus21!

Did you know you can make a passive income by simply delegating steem power?
@slimwhale offers the best return on your investment, sharing 100% of the bidding pool rewards, daily, proportional to your investment.
Let's grow together, start earning now by clicking the following links: 10SP, | 50SP | 500SP, | any amount |
For more information, see here or join me on Discord

Hey @wil1liam I am @utopian-io. I have just upvoted you!

Achievements

  • You have less than 500 followers. Just gave you a gift to help you succeed!
  • Seems like you contribute quite often. AMAZING!

Community-Driven Witness!

I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!

mooncryption-utopian-witness-gif

Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x

Hello,
I'm bot. I am here because @wil1liam wanted to know how high this post is in Hot or Trending.

You want to know what your post's rank in Hot or Trending is?

1. copy your Steemit.com post url
2. send 0.001 SBD/STEEM to @trendchecker with the url as the memo
3. wait 20 seconds to receive the results

As a gift your post will be upvoted for free!


- This bot was development by @wil1liam.

It works 🎉🎉🎉🎉🎉

Thank you for the Steem!