When do whales upvote?

in #programming8 years ago

Ever wondered when whales are upvoting? I was wondering the same thing. I thought: perhaps some times are better to post than others.

So I wrote a program in Python to plot the upvote times for July 2016 for a random selection of whales. Here are the results:

The founders

@ned and @dan appear very limited in the time they have to upvote. They're both similar in the number of votes they give.

If you get upvoted by @dan or @ned, you've done well. Not only because the founders like your work, but also because they aren't the most active upvoters.

Whale bots

The accounts @steemed, @itsascam, and @steemroller are all bots run by the same whale. These graphs illustrate bot behaviour: the distribution of upvotes is more even, and of course, they never seem to sleep.

All these graphs really tell us are when the authors in the bot's author list have published an article.

Others

Here are a bunch of other whales. Maybe some of them are bots, let me know if that's the case. It's fun making assumptions about the whales, like when they sleep.

berniesanders

Between 07:00 UTC and 14:00 UTC @berniesanders is mostly inactive. At the other times though he's a generous upvoter and quite consistent.

blocktrades

@blocktrades is very likely asleep between 06:00 UTC and 14:00 UTC. After this time there's a bit of voting, but not a great deal.

complexring

@complexring is either or bot or someone who has trouble sleeping. While upvoting between 06:00 and 12:00 UTC is significantly lower, there are still some votes taking place. @complexring is much more active in upvoting than @berniesanders.

nextgencrypto

@nextgencrypto does most upvoting during the weekend with some activity in the evening (their local time), assuming the gaps indicate sleep.

pharesim

@pharesim's behaviour resembles @complexring. Perhaps a bot. There are lower periods of activity which could indicate sleep. Friday seems to be the most stable day where there's a clear pickup in voting after 12:00 UTC.

rainmain

It's clear that @rainman is sleeping between around 22:00 UTC and 04:00 UTC. Another low upvoting whale, @rainman is pretty consistent during the day.

smooth

@smooth's behaviour is similar to rainmain in that it's consistent during voting times, but there's no clear downtime, which raises the suspicion of bot use. @smooth is a slightly more active upvoter during the weekend.

tombstone

@tombstone's graph looks empty, but that's caused by the bizarre outlier on Friday at 10:00 UTC. I have no idea what @tombstone was doing then, but it was an upvote frenzy.

The rest of the time @tombstone is quite consistent with voting, with slightly more activity between 02:00 UTC and 08:00 UTC.

When should I post?

I've intentionally not drawn too many conclusion from these graphs. Perhaps the most useful information is knowing which whales upvote the most and when, so you can time your publication correctly.

Do you want to increase the chance that @berniesanders will upvote you? Don't post at 07:00 UTC because for the next 8 hours he's unlikely to see it; better to post around 03:00 UTC on a Wednesday.

With so many whales you're more or less covered whenever you post. What you're posting is far more important than when you post it.

Show me the code

Feel free to use and adapt the code below as you like. Any bugs, please let me know in the comments. Read @furion's post for more information on parsing the blockchain.

The program requires Python 3 and the following libraries, which you can install with pip:

  • matplotlib
  • numpy
  • pandas
  • seaborn
  • steem

The program runs on the command-line. Provide a list of whale username and optionally, either the start and end blocks to analyze or start and end dates. The latter will convert the dates into the appropriate block numbers (this isn't fast - there's probably a better way to do it).

python3 vote_dist.py dan ned smooth berniesanders -f 2016-07-01 -t 2016-07-31

Enjoy!

import argparse
import datetime
import math
from collections import defaultdict

import pandas as pd
import seaborn as sns
import numpy as np
from steemapi.steemnoderpc import SteemNodeRPC


def get_stats(rpc, users, beg_block, end_block):
    print("Getting stats for {} from block {} to block {}".format(
          users, beg_block, end_block))

    stats = defaultdict(list)
    current_block = beg_block
    while current_block < end_block:
        block = rpc.get_block(current_block)

        if "transactions" not in block:
            continue

        for tx in block["transactions"]:
            for op in tx["operations"]:
                op_type = op[0]
                op_data = op[1]

                timestamp = pd.to_datetime(tx['expiration'])

                if op_type == "vote":
                    author = op_data['author']
                    voter = op_data['voter']
                    weight = op_data['weight']
                    if voter in users and weight > 0:
                        stats[voter].append((timestamp, weight))
        current_block += 1
    return stats


def get_block_num_for_date(rpc, date):
    """
    Gets the first block number for the given date.

    This is anything but fast. There's probably a better way, but this was the
    first thing that came to mind.
    """
    print("Getting block num for {}".format(date))

    block = rpc.get_block(1)
    bc_date = pd.to_datetime(block['timestamp'])

    SECONDS_IN_DAY = 24 * 60 * 60
    NUM_BLOCKS_PER_DAY = math.floor(SECONDS_IN_DAY / 3)

    # Estimate the block number
    block_num = (date - bc_date).days * NUM_BLOCKS_PER_DAY

    # Use estimation to find the actual block number
    best_block_num = block_num
    best_block_diff = None
    while True:
        block = rpc.get_block(block_num)
        block_date = pd.to_datetime(block['timestamp'])
        diff = (date - block_date).total_seconds()

        if best_block_diff:
            if abs(diff) > abs(best_block_diff):
                break

        best_block_num = block_num
        best_block_diff = diff
        if diff > 0:
            block_num += 1
        elif diff < 0:
            block_num -= 1
        else:
            break
    return best_block_num


def create_plot(user, votes):
    print("Creating plot for {}".format(user))

    df = pd.DataFrame.from_records(votes, columns=['time', 'weight'])
    df['day'] = df['time'].dt.weekday_name
    df['hour'] = df['time'].dt.hour

    col_order = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday",
                 "Saturday", "Sunday"]

    plot = sns.FacetGrid(df, col="day", col_order=col_order, col_wrap=3)
    plot = plot.map(sns.plt.hist, "hour", bins=np.arange(0, 23),
                    color="c").set_titles("{col_name}")
    plot.set(xticks=np.arange(23, step=2), xlim=(0, 23))
    plot.set_axis_labels('Hour', 'Upvotes')
    plot.fig.suptitle(user, size=16)
    plot.fig.subplots_adjust(top=.9)

    return plot


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('usernames', nargs='*',
                        help='Usernames to show statistics for')
    parser.add_argument('-s', '--server', default='ws://localhost:8090',
                        dest='server',
                        help='Address of the steem JSON-RPC server')
    block_group = parser.add_argument_group('blocks')
    block_group.add_argument('-b', '--begin-block', default=1, type=int,
                        dest='beg_block',
                        help='The block to begin on.')
    block_group.add_argument('-e', '--end-block', default=None, type=int,
                        dest='end_block',
                        help='The block to end on. Default is last_irreversible_block_num.')
    date_group = parser.add_argument_group('dates')
    date_group.add_argument('-f', '--from-date', type=str, dest='from_date',
                        help='The date to end on.')
    date_group.add_argument('-t', '--to-date', type=str, dest='to_date',
                        help='The date to end on.')
    args = parser.parse_args()

    # Connect to steem rpc server
    rpc = SteemNodeRPC(args.server, "", "")

    # Get the block numbers
    if args.from_date and args.to_date:
        args.from_date = pd.to_datetime(args.from_date)
        args.to_date = pd.to_datetime(args.to_date)
        args.beg_block = get_block_num_for_date(rpc, args.from_date)
        args.end_block = get_block_num_for_date(rpc, args.to_date + datetime.timedelta(days=1)) - 1
    else:
        props = rpc.get_dynamic_global_properties()
        if args.end_block:
            if args.end_block > props['last_irreversible_block_num']:
                args.end_block = props['last_irreversible_block_num']
        else:
            args.end_block = props['last_irreversible_block_num']

    # Validate block numbers
    if args.beg_block < 0:
        print("begin-block must be greater than 0")
        sys.exit(1)

    if args.end_block < args.beg_block:
        print("end-block must be greater than beg-block")
        sys.exit(1)

    # Get the stats
    stats = get_stats(rpc, args.usernames, args.beg_block, args.end_block)
    for user, votes in stats.items():
        plot = create_plot(user, votes)
        plot.savefig('{}.png'.format(user))

    if len(stats) == 0:
        print("Nothing found for the given parameters")

Like my post? Don't forget to follow me!

Sort:  
There are 2 pages
Pages

Oh, man, @bitcalm. Your posts never cease to amaze me. I don't know if it's just because you're posting topics that I can relate to or are interested in, but you get all my upvotes. I've been meaning to code something like this for a while now, but opted to experiment posting on different times with different kinds of content. This post is very insightful and is shaping up to be one of my most favorite posts in Steemit so far. I guess an improvement to this is to factor in the tags or types of post they upvote along with their voting time. This really has a potential to analyze whale voting behavior.

By the way, @complexring is 100% human, and an accomplished mathematician. As to why his voting behavior looks like that, I'm not sure. But, he's a cool guy who has an inclination to fiction posts.

@bitcalm nailed it on this post. I would love to see this trend month over month!

Right you are!

Good to know @complexring is human. Hope he figure out the sleep problem ;) Actually, it'd be cool to have a mathematician weigh in on this. Can you pass the link on?

Like ur post @bitcalm , follow u

I was giggling as I read, wondering if any of the possibly-a-bot-but-really-a-human whales were rethinking their sleep schedules.

The more you know :)

Sure thing! I hope he replies on this thread with his thoughts

What @jedau said. I've been posting whenever and wherever I can since I've got to wait about 2 weeks to get internet at my house, so parking lot hopping with free wifi has been my life the last few days since I moved. Yay Me! Anyway - I saw your list of bots and I'm thinking, "Well, I'm a frikken idiot" because I've been seeking those usernames out and trying to engage...maybe get on their radar...and I might as well be talking to my stereo. sigh I kind of have to laugh at myself. LOL

It's good you were notified they were bots before you got emotionally invested in them LOL :D seriously though, great effort trying to find places to post from. I really hope that my upvote for you was worth more but sadly it isn't.

That made me laugh.

You're welcome. I'm glad you enjoy my posts. This one took a bit more effort than the others, but it was a lot of fun. It's only touching the surface of the kind of statistical analysis you can do on the blockchain.

Good for you for blazing the trail. Personally, I've been mostly scared of making any analysis out of fear of making a mistake and being criticized for it. I really hope posts like this gain some traction to increase the confidence of people like myself. I agree that there are tons of insights to be mined.

Everyone makes mistakes. Hopefully if there's a mistake someone spots it, and I learn from it.

Yeah, I guess so. Maybe it comes down to strength of character regarding that aspect. Though I do appreciate criticism directed my way, using them as a means to improve.

Do you think you could find out the general topics they upvote on? Not really because I want to posting about those topics but more to see the direction in which steem it might be heading in. After all the whales are the curators of steem.

I think there was already a post that covered this. I can't remember who it was by. It covered the tags they upvote in most.

Producing stats for topics based on the content would be a bit harder. You could try a keyword based approach with clustering, but that'd be a lot of work for some shaky results. Fun to do if you have the time for it :)

Data is beautiful.

I agree with this wholeheartedly. If only data wasn't shared by all, I would've put a ring on it instantly. That's how much I like it.

Wow that was way to dam helpful !! OK if anyone agrees smash dat mfcking up vote! Also how has this helped others??

The upvote button for this post has been so smashed, I don't know if we could even resuscitate it.

I can assure you that I am not a bot ... I just happen to enjoy most posts I read.

Hehe my analysis was anything but in depth, and I'm no statistician. Hope you weren't offended :D I hear you're a mathematician. Any tips for future stats posts?

Absolute friggin gold.

Your perverted, you know that! lol

I see what you did there...

Oh Man, you stole my thunder! http://catchawhale.com
Was going to publish my results soon, but looks like you beat me to the chase.
I will still of course be publishing my findings but this looks pretty damn good.

But you made a cool web app out of it. I was too lazy to do that (well actually it's cause I'm going on holiday soon and wouldn't finish it in time).

Looking forward to your post. Oh and I'm following you now :D

As am I! We should collaborate in the near future!

Interesting research, but people can still post everyday, this is not a reason to just post on those specific days in those specific hours.

I don't have time for all that. Steemit is just one of the many things I do, I can't be up all in this bizness 24/7 I have other things to do. Money doesn't run my life :)

Now?

Some fantastic work by you here. It is a real pity there are so few whales - there just arn't enough to go around.

Thanks! They say this will change as time goes on and power is more distributed. We'll see.

Good stuff, I was wondering what best post times would be. Steemians, it's wise to do your best with the knowledge @bitcalm just posted!

This was actually much more interesting that I expected, good job :)

I can confirm I don't use a bot to vote for me; it's all done manually by yours truly, at least for now. I'm a bit surprised that I'm a "low upvoting whale" since I feel like I vote a lot, but as I don't use bots or have a team of people curating for me I suppose that's normal.

I'm also trying to refrain from voting on posts that already have $1000+ payouts in order to balance things out a little, so I guess that factors in as well.

Hi Rainman,

its great to see you communicate with us. i think it doesn't matter, if one uses a bot or votes himself.
through you own reading , you will catch more valuable stuff than the bots, the bots on the other hand, will be able to spread more earnings to many authors. so both sides have good and bad aspects.

maybe you could do me a favor and check this out,
https://steemit.com/food/@knozaki2015/britafilters-hacked-read-the-whole-hacking-story-steemit-exclusive

i co-authored it with a friend (100% of the Steem $ going to him) , and this is my piece i think was my best post of all in my Steemit history.

thanks !

Nice to hear you don't use a bot. It was difficult to tell with you because it could have just been (and probably was) the way things averaged out....or you're telling porkies ;)

It makes sense that the bot powered whales are upvoting more. Perhaps that's a good metric to determine if someone is a bot - I can imagine bots upvote more than regular users (and also never post articles). At the same time, I literally just looked at the images to compare them. I wanted to add the total count to each plot, but I'd already generated them and took 30 mins - laziness won. :)

Nice to hear that you hold back on large reward posts. I think a lot of users will be happy to hear that.

You did get my vote nevertheless ;)

I could be telling porkies of course but that doesn't come natural to me so I don't :) My curation rewards probably would've been way higher had I used a bot in any way, and being sick like I was for the last 3-4 days would not have made such a big impact.

I'm fine with the lesser rewards though, and to be honest I feel the system is a bit too skewed towards whale curators at the moment. Us whales working hard to optimize our curation rewards will not help with the distribution issue, so I prefer to take a relaxed attitude towards it.

Your power has grown! Impressive. Most impressive.

This is cool. Great you made this statestic. ^^

I'd say avoid 8-16UTC time, which is 9-5 UK time! kinda convenient if you live in UK and want help with your Steem addiction!

Yeah. The next question to answer is after posting, when do the most rewards happen. Do most come in the first hour? Is 30 minutes really the cutoff? Is there a point where posts just stop getting attention even though it's inside the reward time limit?

The blockchain has the answer! :)

This was my next question as I read through your post. Are you reading my mind? You have a new follower.

Any whales want to vote on this one perhaps ;)

I spent my birthday helping the homeless in Argentina. Upvoting = making a donation for free! https://steemit.com/travel/@budgetbucketlist/an-unforgettable-birthday-party-with-the-homeless-people-of-buenos-aires-made-possible-by-steemit

Great analysis. Now we can cast our fishing rod at the right time trying to catch a whale :)

That was a good laugh. lol.
There are real people behind whale :)

Looks like a tombstone have a nice Friday morning

omg nice job ;))) hahaha

Good work @bitcalm, I really like the way you displayed your information. I assume the times shown here are UTC?

I posted something that looked at similar data not too long ago, but I tried looking at it on a calendar. Check it out if your have a minute, I'd appreciate your input.

I thought bots weren't allow. Now I'm wondering if the only way to make money is to get upvoted by a whale. I'm new here and just figuring things out before I do my first post.

I liked you post...upvote has to be always there for a good post like this which explains the statistics well...

Excellent post @bitcalm. I'll take a look at that code as well. Upvoted. Oh, also take a peek at this, @katecloud! Hopefully this can help you with your awesome artwork!! :)

I posted a Visual Timezone Aide over here, citing this post for the inspiration. Hope this helps people!
https://steemit.com/programming/@scottvanfossen/need-a-visual-aide-in-time-zones-post-when-the-whales-will-see-you

Thats some mighty fine whale stalking you've been doing here. Bravo for the effort man!

This is the Willy Report of Steemit.

following you now of course! and amazing analysis! Loved the geeky graphs! haha So conclusion... post whenever :P

Don't make fun of my conclusion! :) Yes, conclusion is there are enough whales (probably) that it doesn't matter. Although, targeting the avid upvoting big whales might pay off.

ummm ;P

I'm hoping all these comments will help it trend, but I think it's just purely based on the power of the upvoters.

I think the whales hate me.

This is really helpful. Thanks for building. What lis next?

Great work! And thanks for the code that you used!

You're welcome. Hopefully others can use it as a basis for even cooler stats.

I think we need ro focus on our posts. If not we will get crazy trying to catch a whale

You're right, people shouldn't "go overboard" (see what I did there?) trying to get whale votes. The reason I did this was for the fun and just in case there was a really obvious pattern, like whales never upvoting on Fridays. Not the case.

Anyway, I think you always make good points and post that generate discussion. You were one of the first ones I followed. Keep the good work!

very detailed, making a clearer picture about "whales"

Brilliant work and I imagine it took hours of work. My experience on social media has been to post whenever I can. Preferably in the am since most of my followers are in North, Central or South America. On Steemit with the window being shortlived I think 10am EST is a safe time for my account.

Yeah it took about 5 hours. I spent ages tweaking it. Converting a date to a block number wasn't really necessary, but it makes it easier to use. Also, that method is super slow. I bet it can be done better.

wow! i was thinking about this the other day. unfortunately i dont have the knowledge to this. so thanks a mil for this

So, if @steemed, @itsascam, and @steemroller are all bot, how do they judge quality of a post? Does it mean that these whale bots are upvoting relying on some pre-defined parameter instead of post quality?

To get on the author list you need to be pre-approved, and from what I've heard, what gets upvoted is checked and if the quality is poor, you risk being removed.

So it's automated but still sort of curated.

I get why you're not happy with it though. It's a big presumptuous to think that someone always writes good content; a kind of halo effect, if you like.

We need to check

But what do the whales actually vote on? I was hoping that my latest post would catch a few of them. How does that happen? Do I need more followers? And what's with the 30-minute talk about rewards? I would think that a post like this would at least get a little more attention from whales and bots:

https://steemit.com/anarchism/@ats-david/enriching-lives-through-the-power-of-steemit

LOL! Great :)

I really love posts with stats because they show me so much more than reading words that make me scared to say them out loud. Thank you for sharing this with us all and giving us some more insight on what plays around here and around what time. Not that I think it will do me good seeing I am awake when all the whales sleep. But then again, I am rather enjoying myself here even without the whales. I love goldfish.

I'm sure this isn't a very original thing to say but kudos to you for compiling this and coding the script/program? yourself. Very interesting idea, I never really gave much thought as to posting and upvoting times trends throughout the day until now. I think I'd to like follow further posts relevant to this. Could you maybe suggest a few of the steemtools you think are most useful or interesting for a new member?

I use steemstats and not much else. I'm in steemit.chat a lot, but that's not what you mean. @furion has a nice developer's guide if you're interested in getting into it yourself, and @blueorgy has a stats website that works with live data. I'm sure there are more.

Alright cool I'll be sure to check steemstats, furion's guide and the chat and stats website!
Thanks again :)

lol you crack me up with the "whales".

Very interesting even tho I will admit I don't understand the details😍

Nice tip man!!

This is like asking ,"When do Sith Lords force choke someone?"

Whenever they want.

Fantastic post! Thank you for doing the analysis and sharing the code. This post is definitely making it to my FAVORITES!

Steem on!
Mike

Interesting. Ordered by what?

There's no ordering. It's grouped by day and distributed over the number of upvotes in each hour in the day for all upvotes in July.

I'm saving this post in my browser favs; it's worth to revisit from time to time. I like what you did here @bitcalm

Awesome info now I just need a script to guide them to my blog lol just kidding you post is awesome.

@mrgrey

Good to have this information @bitcalm, its very useful to people like me who wanted to be upvoted by whales. The only trouble here is the time, I live on the other side of the world where days here are night there but I can find a way, I have to find a way.

Fascinating and clever. Thank you!

There are 2 pages
Pages