This tutorial is part of a series on ‘How To Build Bot’s Using Steem-JS Open Source Library’ it aims to demonstrate practical applications for bots and demystify the code used to create them.

The completed bot code is here - I know some people find it easier to see everything in one block and follow along that way. ⬅️

Outline 📝

In this tutorial we’re going to create another bot to assist with running tutorials. The bot will not respond directly but will help gather a list of accounts who have re-steemed a specific post. For examples - used for contests that ask entrants to Re-steem for a chance to win. This was inspired by @money-dreamer who posted about wanting to do exactly this.

Requirements

A plain text editor (I use atom.io )

Difficulty

intermediate

Please run through all other tutorials in the series to better understand what we’re working on.

(I appreciate feedback on the teaching level).

Learning Goals

re-use knowledge from previous tutorials
Look at the getBlock() function of Steem-js
create a that finds re-blogs of a certain Steem post

Step 1 - information 💻

A couple points to note before starting. When accessing data about particular posts on the blockchain there is a reblogged_by property. Unfortunately, the reblogged_by always returns empty, regardless of the number of reblogs(confirmed through the web interface). There is no information in the Steem-js library about re-blogs and I have found almost no information online.

reblogs are broadcast across the network as custom_json and can be recorded they are just not part of the steem-js database API. We have two options.

1). Stream transactions and save any custom_json transactions that match our required data. This is the plan today.

2). Use a third-party set of Data that has already been databased and streams the transactions themselves. see this post for an example of this method, requires a paid subscription to third-party service.

Note that today's tutorial might not be the fastest or most technically sound method of retrieving this data but in order to keep the tutorial available for a wider advice I’ve decided to keep it as simple as possible.

Step 2 - Setup 📈

We’re once again working in the browser, but just as easily could work from node.js. Starting template with script tags setup

To kick things off we’re going to want four pieces of information that relate to the post we’re checking for reblogs. First is the permlink and can be seen in the URL of the post.

Screen Shot 2018-02-11 at 16.04.20.png

const TARGET_PERMLINK = 'finally-a-comments-system-that-rewards-its-authors-steemcomments-js-got-an-upgrade'

Next is the block number where our post was included into the blockchain. (More on why in the next section). The easiest way I’ve found to find this is on steemd for example I would navigate to my page https://steemd.com/@sambillingh and my recent activity will show in the feed. In the top right-hand corner of each activity there is a transaction id link, click that to find specific information about the transaction. Here we can see the exact block number the post was included in.

Unfortunately I’ve not found a way to retrieve this from the steem-js API

Screen Shot 2018-02-11 at 16.08.55.png

Screen Shot 2018-02-11 at 16.09.12.png

Store that block number as a variable.

const BLOCK = 19642524

Third, also available on the transaction screen in the time of the block we’re interested in. To make out date times easier to work with we’ll create a javascript data object and use valueOf() this gets the milliseconds between 1 January 1970 00:00:00 UTC (know as The Unix Epoch) and our date. Having a single integer number that represents the date make it really easy to check if one time is before or after another by using greater than or less than.

``const BLOCK_TIME = new Date('2018-02-06T20:59:36').valueOf()```

Finally we’ll create an end time. Our script will stop after this time. My full set of starting information.

const TARGET_PERMLINK =  'finally-a-comments-system-that-rewards-its-authors-steemcomments-js-got-an-upgrade'
const BLOCK = 19642524
const BLOCK_TIME = new Date('2018-02-06T20:59:36').valueOf()
const END_TIME = new Date('2018-02-07T20:59:36').valueOf()

Step 3 - Loop Over Steem Blocks 🏋️‍♂️

Block on the Steem Blockchain are created roughly every 3 seconds by one of the Steem witnesses and contain a number of transactions. I’ve observed roughly 50 transactions per block, but I’ve not looked into this too much and likely fluctuates because of a number of variables.

We have a couple options. A) We could start from the latest Steem block and work backwards through each block until we get to the time the competition started. B) We can start from the point the post was created then work forwards until a certain time/date perhaps 24hours or 7 days etc.

We’re going to go for option B today and that’s why we looked for the block number in step 2.

The getBlock() function of the Steem-js API allows us to request individual block information. We can investigate the transactions included.

steem.api.getBlock(BLOCK, function(err, block) {
    console.log(err, block);
});

Each block contains an array of transactions.

Screen Shot 2018-02-11 at 17.10.03.png

The plan is to check each block individually. As we know the current block number if we want to move onto the next one when done all we need to do is increase the block number by + 1.

first, we’ll turn the API call into a reusable function. To ensure our we don’t stream block forever we’ll add an if statement to check if the current blockTime is past our initial data of END_TIME

function getBlock(blockNumber, blockTime){
    if (blockTime < END_TIME){
        steem.api.getBlock(blockNumber, function(err, block) {
            console.log(err, block);
        });
    }
}

Once we’ve checked this first block we should immediately move to the next block on the chain. We’ll also store the timestamp to check against our if statement.

let nextBlock = blockNumber + 1
let blockTime = new Date(block.timestamp).valueOf()

This will sit inside of our getBlock function along with another call to getBlock creating a recursive function that stops only when our if statement does not pass.

function getBlock(blockNumber, blockTime){
    if (blockTime < END_TIME){
        steem.api.getBlock(blockNumber, function(err, block) {
            let nextBlock = blockNumber + 1
            let timestamp = new Date(block.timestamp).valueOf()
            getBlock(nextBlock, timestamp)
        });
    }
}

This function will now run continuously through all of the blocks until we reach our end time. FYI that’s a lot of blocks. Estimate one block every 3 seances gives us 28,800 blocks over a 24hour period we need to investigate. 🙈

Without worrying too much about how long this will take we’ll look at finding the reblog data

Step 4 - Check Transactions

Similar to what we’ve done in previous tutorials we’ll loop over transactions first checking for custom_json then checking if the type of custom_json is reblog before finally accessing the data. Passing a transactions array in preparation for use within the getBlock function.

function checkTransactions(transactions, timestamp){
      transactions.forEach(function(tx) {
                console.log(tx)
            })
}

Access the transactions array within the getBlock() response with block.transactions.

Time to narrow down the results. We’ve done this before in previous tutorials so to keep things short we’ll inspect the operations array of each transaction storing the type and data. Following that if we see a type of custom_json we’ll check the JSON property to see if it includes reblog.

transactions.forEach(function(tx) {
    let txType = tx.operations[0][0]
    let txData = tx.operations[0][1]
        if(txType === 'custom_json') {
                        let jsonData = JSON.parse(txData.json)
                    if( jsonData[0] == 'reblog' ){
                                console.log(tx)
                        }
            }
})

We’re closer. Now we’ll compare the permlink in the transaction with the one of the post we’re look at for the competition. To tie everything together we can do two things, first push the data into an array for later use and second display it on the screen.

if( jsonData[0] === 'reblog' && jsonData[1].permlink === TARGET_PERMLINK) {
              console.log(jsonData[1])
              reblogs.push(jsonData[1])
              $('body').append(`${jsonData[1].account}}" : ${timestamp}
`)
}

We’re simply appending this data to the screen here. Check the previous tutorial if you’re interested in creating a table and choosing a random entry.

p.s make sure you’re adding your checkTransactions() function to the getBlock() function.

It took 3 minutes and 580 requests to the Steem API to find my first reblog. Thank for the reblog @jeffbernst.
Screen Shot 2018-02-11 at 22.01.25.png

Screen Shot 2018-02-11 at 22.01.46.png

step 5 - Thoughts and Optimisations 🚀

It’s worth noting that this process takes time. There are a lot of blocks to work through. When looking at the network tab my requests take ~90ms. At 28,000 blocks added to the blockchain each day, it will take ~45minutes to request one day worth of transactions.

Is this the best way to get this data? Perhaps not but I think it’s important to illustrate the technique. While it’s certainly possible to use this technique it would be better to create a database that holds all of the Steem data as seen with [SteemData] & [SteemSQL]. Keep that data up to date and then make juries against it.

The script we’ve crated works through block by block in a sequence, this helps to keep the tutorial accessible. If you’re looking to use this code more than as a proof of concept I would certainly think about how you might asynchronous make the queries to the blockchain and run requests in parallel to reduce wait time.

Here’s the full code 🤖

Let me know what you think of this robot tutorial.

TUTORIAL - Reblog Contest - Building With Steem-js #10