Part 19: Analysing The Steem Blockchain From A Custom Block Number For A Custom Block Count

This tutorial is part of a series where different aspects of programming with steem-python are explained. Links to the other tutorials can be found in the curriculum section below. This part will focus on how to analyse the Steem Blockchain. Instead of streaming operations as they are coming in, a custom starting point will be set with a custom time frame to collect data for.

---

What will I learn

• How does the Steem Blockchain work?
• Stream data starting from a custom block number
• What does an operation look like?
• Counting the amount of blocks analysed
• Filter operations and store statistics

Requirements

• Python3.6
• steem-python

Difficulty

• Intermediate

---

Tutorial

Setup

Download the file from Github. There is 1 file analyse_blockchain.py which contains the code. The file takes 1 argument from the command line which sets the amount of blocks to analyse.

Run scripts as following:
> python analyse_blockchain.py 100

How does the Steem blockchain work?

Every type of action on Steem is stored as an operation on the Steem Blockchain. These operations are stored in blocks which are created every 3 seconds in ascending order. Hence the name blockchain. The most recent block is the head block, while the rest of the chain is revered to as the tail.

from steem.blockchain import Blockchain

blockchain      = Blockchain()

Stream data starting from a custom block number

Until now all tutorials which applied streaming the blockchain worked with new blocks that were coming in. However, it is also possible to start from a custom block number and work from there. This is useful when you want to look back and analyse blocks. For obtaining the current head block the following function can be called.

blockchain.get_current_block_num()

This script will look back a custom amount of blocks into time from the current head block.

start_block     = head_block - <amount of blocks to analyse>

The stream can be created with the following function, where start_block defines the starting point in the blockchain.

stream          = blockchain.stream_from(start_block=start_block)

New blocks are generated every 3 seconds. Analysing 1 day therefore sums up to 24x60x20 = 28 800 blocks

What does an operation look like?

One block can contain multiple operation, like the one below.

{
    'trx_id': '6a49a53070279a59ade771d8573800afa9d38f2e',
    'block': 19424314,
    'trx_in_block': 0,
    'op_in_trx': 0,
    'virtual_op': 0,
    'timestamp': '2018-01-30T06:53:57',
    'op': ['transfer', {
        'from': 'minnowbooster',
        'to': 'elizzium2018',
        'amount': '0.002 SBD',
        'memo': 'You got an upgoat worth roughly 0.44$ in post rewards that will be done by blockchainttmft. We detected an open value of 0.005 SBD, worth 0.002 SBD in send and refunded that. '
    }]
}

op designates what kind operation was used . There are over 30 different kind of operations.

Counting the amount of blocks analysed

The time frame will the amount of blocks to be analysed. Since 1 block can contain multiple
operations counting the amount of operations will not work. The block number in which the operation is stored can be used to determine if a new block has been reached.

current_block   = start_block
counter                 = 0

for post in stream:
        if post['block'] != current_block:
            counter += 1
            print ("Block {}/{} {:.2f}%".format(counter, block_count, counter/block_count*100))
            current_block = post['block']

Filter operations and store statistics

To count the amount of each unique operation a dict is made, in this dict each operation is stored as a key pair with the amount of occurrences.

stats           = {}
operation   = post['op'][0]

if operation not in stats:
    stats[operation] = 1
else:
    stats[operation] += 1

Running the script

Running the script will analyse the set amount of blocks back in time from the current head block. Depending on the amounts of blocks to be analysed this can take some time.

python analyse_blockchain.py 28800
...
Block 28797/28800 99.99%
Block 28798/28800 99.99%
Block 28799/28800 100.00%
Block 28800/28800 100.00%

operations 1836907

custom_json 350000
vote 710553
claim_reward_balance 59814
comment 281821
transfer 48456
producer_reward 28800
curation_reward 166849
comment_benefactor_reward 11297
author_reward 57549
...

Curriculum

Set up:

• Part 0: How To Install Steem-python, The Official Steem Library For Python
• Part 1: How To Configure The Steempy CLI Wallet And Upvote An Article With Steem-Python

Filtering

• Part 2: How To Stream And Filter The Blockchain Using Steem-Python
• Part 6: How To Automatically Reply To Mentions Using Steem-Python

Voting

• Part 3: Creating A Dynamic Autovoter That Runs 24/7
• Part 4: How To Follow A Voting Trail Using Steem-Python
• Part 8: How To Create Your Own Upvote Bot Using Steem-Python

Posting

• Part 5: Post An Article Directly To The Steem Blockchain And Automatically Buy Upvotes From Upvote Bots
• Part 7: How To Schedule Posts And Manually Upvote Posts For A Variable Voting Weight With Steem-Python

Constructing

• Part 10: Use Urls To Retrieve Post Data And Construct A Dynamic Post With Steem-Python

Rewards

• Part 9: How To Calculate A Post's Total Rewards Using Steem-Python
• Part 12: How To Estimate Curation Rewards Using Steem-Python
• Part 14: How To Estimate All Rewards In Last N Days Using Steem-Python

Transfers

• Part 11: How To Build A List Of Transfers And Broadcast These In One Transaction With Steem-Python
• Part 13: Upvote Posts In Batches Based On Current Voting Power With Steem-Python

Account Analysis

• Part 15: How To Check If An Account Is Following Back And Retrieve Mutual Followers/Following Between Two Accounts
• Part 16: How To Analyse A User's Vote History In A Specific Time Period Using Steem-Python
• Part 18: How To Analyse An Account's Resteemers Using Steem-Python

---

The code for this tutorial can be found on GitHub!

This tutorial was written by @juliank in conjunction with @amosbastian.

---

Posted on Utopian.io - Rewarding Open Source Contributors

---