Candle Maker: Generate your own candlestick data from internal HIVE market prices

IMG SOURCE

Hey everyone, hope all is well.

Candle-maker is a small open source script that monitors the latest internal HIVE market prices & generates custom candlestick data. Written in Javascript.

This post will be a breakdown of the planning & basic thought process that I went through while writing the code for this project.

The goal:

Before we can jump straight into coding the script, we first need to have a rough idea of what we want it to do.

Candle-maker should repeatedly fetch data on the latest market prices & then at certain periods, process the data & create candlesticks.
The 3 prices that the script will focus on are: Last price, Lowest Ask & Highest Bid.
From these prices, 3 different candles should be generated.

Based on the candle limit variable, that amount of the latest candles will be saved as a local copy. The script also has the option to broadcast the latest candle to the Hive blockchain in JSON format.

Note:

The performance + accuracy of this script is fully dependent on the processing speed of your machine & the speed of your internet connection.

---

1.Planning:

Planning is the most important part of tackling any new project. Without it, the whole process of building a fast & functioning script/app/whatever can be a lot harder. So let's start from the beginning and take a look at what is needed before we start coding.

What are candlesticks & how are they calculated?:

Open up any trading platform & chances are you will be greeted by a bunch of green & red bars. This indicator known as a candlestick shows the price change over a certain period.

img source

To create a single candlestick, we need 4 data points:

• Open => The opening/starting price for the period
• Close => The closing/end price for the period
• High => The highest price during the period
• Low => The lowest price during the period

1, 3, 5, 10, 30 & 60 minutes are really popular time frames used by most trading platforms to create their candlestick data.

How do we get the price data?:

Since this project is 100% written in Javascript & uses Node.js to run, there are 2 popular library options available, dhive & hivejs. Only hivejs was used for this script.

Creating a map of the script:

Understanding how each candlestick is calculated is the most important part of the planning for this project, so now that we have that we can start creating a rough plan of what the script's logic will look like:

The function updatePrice() will be used as our main loop. It will constantly, at a fixed interval, fetch & push new price data into the 3 temporary price trackers.
Each tracker holds one of the 3 types of price data that we are interested in (Last, LowestAsk, HighestBid)

At certain intervals, the function createCandle() will clean out the temporary price trackers, use that data to create a candle object & convert it into a JSON object with the following format:

{
   "start":"2022-02-06T01:04:39.544Z",
   "end":"2022-02-06T01:05:39.579Z",
   "candleNum":292,
   "candleSecs":60.035,
   "latest": {
      "open":1.1310160427807487,
      "high":1.1310160427807487,
      "close":1.1310160427807487,
      "low":1.1310160427807487
   },
   "lowestAsk": {
      "open":1.2481369088169112,
      "high":1.2481369088169112,
      "close":1.2481369088169112,
      "low":1.2481369088169112
   },
   "highestBid": {
      "open":1.2436109487507927,
      "high":1.2436109487507927,
      "close":1.2436109487507927,
      "low":1.2436109487507927
   }
}

The timestamps are UTC+0 (Coordinated Universal Time).

Once a candle is created, it is added to the CANDLE HOLDER tracker. This list of all candles is limited to N amount & only keeps the latest candles in memory. This is where we can enable/disable the broadcastCandle() & saveCandles() functions to save the data.

2.Coding:

Now that the planning is complete, the fun can begin! =)

First off we need a way to track everything that happens in the script. So we will create an object called globalState, & it will store the candle data, temp trackers & error counts:

globalState:

let globalState = {
    candleCounter : 0,
    priceUpdateErrors : 0,
    candleBroadcastErrors: 0,
    candleDataBase : [],
    tempPriceHolder1 : [],
    tempPriceHolder2 : [],
    tempPriceHolder3 : []
};

next, we need the main loop that will repeatedly poke the servers for the latest market data. We will call this function updatePrice():

updatePrice():

const updatePrice = () => {
    new Promise((resolve, reject) => {
        setTimeout( async () => {
            const timeDiff = (((new Date().getTime() - globalState.lastUpdate) / 1000))

            console.log(`*Price updated! => time diff: ${timeDiff} - Current candles: ${globalState.candleDataBase.length} / ${candleLimit} (Price check errors: ${globalState.priceUpdateErrors} - Candle broadcast errors: ${globalState.candleBroadcastErrors})`)
            globalState.lastUpdate = new Date().getTime()

            if (globalState.lastUpdate - globalState.lastCandleCreated >= (candleSize * 60) * 1000) {
                createCandle(globalState.lastCandleCreated, globalState.lastUpdate)
                globalState.lastCandleCreated = globalState.lastUpdate;
            }

            if (globalState.candleDataBase.length == candleLimit + 1) {
                globalState.candleDataBase.shift();
            }

            try {
                hive.api.getTicker(function(err, data) {
                    if (data) {
                        globalState.tempPriceHolder1.push(Number(data.latest))
                        globalState.tempPriceHolder2.push(Number(data.lowest_ask))
                        globalState.tempPriceHolder3.push(Number(data.highest_bid))
                    } else {
                        globalState.priceUpdateErrors++;
                    }
                });
            } catch (error) {
                globalState.priceUpdateErrors++;
            }
            updatePrice();
        }, updateRate * 1000)
    })
}

Once the loop is complete, updateprice() will recursively call itself again for the next cycle.

Now we need to create a function called createCandle() that will create & store the latest data in the temp trackers. For exact timing on all 3 types of candles, we will pass 2 arguments when calling the function, startTime & endTime:

createCandle():

const createCandle = (startTime, endTime) => {
    globalState.candleCounter++;
    globalState.candleDataBase.push({
        start : new Date(startTime - Math.abs(new Date().getTimezoneOffset())).toISOString(),
        end : new Date(endTime - Math.abs(new Date().getTimezoneOffset())).toISOString(),
        candleNum : globalState.candleCounter,
        candleSecs : (endTime - startTime) / 1000,

        latest : {
            open : globalState.tempPriceHolder1[0],
            high : Math.max(...globalState.tempPriceHolder1),
            close : globalState.tempPriceHolder1[globalState.tempPriceHolder1.length -1],
            low : Math.min(...globalState.tempPriceHolder1)
        },

        lowestAsk : {
            open : globalState.tempPriceHolder2[0],
            high : Math.max(...globalState.tempPriceHolder2),
            close : globalState.tempPriceHolder2[globalState.tempPriceHolder2.length -1],
            low : Math.min(...globalState.tempPriceHolder2)
        },

        highestBid : {
            open : globalState.tempPriceHolder3[0],
            high : Math.max(...globalState.tempPriceHolder3),
            close : globalState.tempPriceHolder3[globalState.tempPriceHolder3.length -1],
            low : Math.min(...globalState.tempPriceHolder3)
        }
    })

    globalState.tempPriceHolder1 = [];
    globalState.tempPriceHolder2 = [];
    globalState.tempPriceHolder3 = [];
    
    console.log('----------------------')
    console.log(`Candle created! #${globalState.candleCounter}`)

    if (broadcast) {
        console.log('Broadcasting now...')
        broadcastCandle()
    }
    console.log('----------------------')
    saveCandles();
}

If enabled, createCandle will automatically trigger broadcastCandle():

broadcastCandle():

const broadcastCandle = () => {
    const json = JSON.stringify(globalState.candleDataBase.slice(-1)[0]);

    try {
        hive.broadcast.customJson(bKey, [], [bUser], `${bUser}-candleMaker`, json, function(err, result) {
            if (err) {
                globalState.candleBroadcastErrors++;
            } else {
                console.log(`Broadcast success!`);
            }
        });
    } catch (error) {
        globalState.candleBroadcastErrors++;
    } 
}

broadcastCandle() simply looks at the latest candle created & broadcasts that to HIVE with the username & private posting key provided in settings.

This is what it will look like on https://hiveblocks.com/@usrnameHere :

And lastly, we need a function called saveCandles() to save the latest N candles as a local copy. The data will be stored as a .json file:

saveCandles():

const saveCandles = () => {
    if (keepCandles == true) {
        fs.writeFileSync('./candleDump.json', JSON.stringify(globalState.candleDataBase))
    }
}

The full script:

const fs = require('fs');
const hive = require('@hiveio/hive-js');
const {
    updateRate, candleSize, candleLimit, keepCandles, broadcast, bUser, bKey
} = JSON.parse(fs.readFileSync('./settings.json'));

let globalState = {
    candleCounter : 0,
    priceUpdateErrors : 0,
    candleBroadcastErrors: 0,
    candleDataBase : [],
    tempPriceHolder1 : [],
    tempPriceHolder2 : [],
    tempPriceHolder3 : []
};

const saveCandles = () => {
    if (keepCandles == true) {
        fs.writeFileSync('./candleDump.json', JSON.stringify(globalState.candleDataBase))
    }
}

const createCandle = (startTime, endTime) => {
    globalState.candleCounter++;
    globalState.candleDataBase.push({
        start : new Date(startTime - Math.abs(new Date().getTimezoneOffset())).toISOString(),
        end : new Date(endTime - Math.abs(new Date().getTimezoneOffset())).toISOString(),
        candleNum : globalState.candleCounter,
        candleSecs : (endTime - startTime) / 1000,

        latest : {
            open : globalState.tempPriceHolder1[0],
            high : Math.max(...globalState.tempPriceHolder1),
            close : globalState.tempPriceHolder1[globalState.tempPriceHolder1.length -1],
            low : Math.min(...globalState.tempPriceHolder1)
        },

        lowestAsk : {
            open : globalState.tempPriceHolder2[0],
            high : Math.max(...globalState.tempPriceHolder2),
            close : globalState.tempPriceHolder2[globalState.tempPriceHolder2.length -1],
            low : Math.min(...globalState.tempPriceHolder2)
        },

        highestBid : {
            open : globalState.tempPriceHolder3[0],
            high : Math.max(...globalState.tempPriceHolder3),
            close : globalState.tempPriceHolder3[globalState.tempPriceHolder3.length -1],
            low : Math.min(...globalState.tempPriceHolder3)
        }
    })

    globalState.tempPriceHolder1 = [];
    globalState.tempPriceHolder2 = [];
    globalState.tempPriceHolder3 = [];
    
    console.log('----------------------')
    console.log(`Candle created! #${globalState.candleCounter}`)

    if (broadcast) {
        console.log('Broadcasting now...')
        broadcastCandle()
    }
    console.log('----------------------')
    saveCandles();
}

const broadcastCandle = () => {
    const json = JSON.stringify(globalState.candleDataBase.slice(-1)[0]);

    try {
        hive.broadcast.customJson(bKey, [], [bUser], `${bUser}-candleMaker`, json, function(err, result) {
            if (err) {
                globalState.candleBroadcastErrors++;
            } else {
                console.log(`Broadcast success!`);
            }
        });
    } catch (error) {
        globalState.candleBroadcastErrors++;
    } 
}

const updatePrice = () => {
    new Promise((resolve, reject) => {
        setTimeout( async () => {
            const timeDiff = (((new Date().getTime() - globalState.lastUpdate) / 1000))

            console.log(`*Price updated! => time diff: ${timeDiff} - Current candles: ${globalState.candleDataBase.length} / ${candleLimit} (Price check errors: ${globalState.priceUpdateErrors} - Candle broadcast errors: ${globalState.candleBroadcastErrors})`)
            globalState.lastUpdate = new Date().getTime()

            if (globalState.lastUpdate - globalState.lastCandleCreated >= (candleSize * 60) * 1000) {
                createCandle(globalState.lastCandleCreated, globalState.lastUpdate)
                globalState.lastCandleCreated = globalState.lastUpdate;
            }

            if (globalState.candleDataBase.length == candleLimit + 1) {
                globalState.candleDataBase.shift();
            }

            try {
                hive.api.getTicker(function(err, data) {
                    if (data) {
                        globalState.tempPriceHolder1.push(Number(data.latest))
                        globalState.tempPriceHolder2.push(Number(data.lowest_ask))
                        globalState.tempPriceHolder3.push(Number(data.highest_bid))
                    } else {
                        globalState.priceUpdateErrors++;
                    }
                });
            } catch (error) {
                globalState.priceUpdateErrors++;
            }
            updatePrice();
        }, updateRate * 1000)
    })
}

const main = () => {
    globalState.startingTime = new Date().getTime()
    globalState.lastUpdate = globalState.startingTime;
    globalState.lastCandleCreated = globalState.startingTime;

    console.log('Starting...')
    updatePrice();
}

main();

3.Installation & execution:

You can find the latest version of this script on Github. If you have Nodejs installed you can simply clone & run the repo with the following quick-start guide:

• Clone this repo & cd into it
  
  git clone https://github.com/louis23412/candle-maker.git && cd candle-maker
  

• Install dependencies
  
  npm install

• Open settings.json & change as needed.
  

  {
   "updateRate" : 5,
   "candleSize" : 1,
   "candleLimit" : 500,
   "keepCandles" : false,
  
   "broadcast" : true,
   "bUser" : "usernameHere",
   "bKey" : "privatePostingKeyHere"
  }
  

updateRate => The time interval in seconds at which the script will fetch the latest price data
candleSize => The time interval in minutes of each candle.
candleLimit => The max number of latest candles to keep in memory & local copy.
keepCandles => true/false to save a local copy of latest candles

broadcast => true/false for broadcasting
bUser => the hive username to use when broadcasting
bKey => the private Posting to use with bUser for broadcasting

If you choose to active broadcasting, then for security reasons, only use your posting key(s)!

• Save the config file, then run the bot
  
  npm start

---

That's all I have, for now, more updates coming soon.
--------------------------------------------------------------------------
Add me on Discord if you'd like to talk: CreepyPastaZ ψ(｀∇´)ψ#1729
--------------------------------------------------------------------------