As I mentioned in my previous post @howo/hive-core-developper-meeting-8 we are going to setup multi nodes testnets and we want as many people participating as we can.

Testnets won't be using the main net chain data, so they won't be taking that much resources, (a node currently takes about 45mb of ram and since those testnets will be short lived I don't expect them to use more than 500mb).

We'll create two testnets sequentially:

• First testnet will be set without a supermajority support for hf24 to see how the transition goes at the hf24 time (nothing should happen and we should continue on hf23). We are just interested in the transition, so as soon as the test is successful, we'll kill it and switch to the second one. This testnet is already live and the hardfork date for it is 2020-08-27T20:00:00.000Z UTC (1598558400 unix time)

• Second testnet will be the same but one where the super-majority of witnesses run hard fork 24, we want to see how the transition from hf23 to hf24 goes (we should fork to hf24). This one is not live yet, it will be after the 27th when the first one has hard forked. We will hardfork on the 29th at 20:00 UTC

How to run your own testnet node

First you need to pick wether you want to be running a hard fork 23 or hard fork 24 node. If you don't know, pick hard fork 23 for the first testnet, and hard fork 24 for the second hard fork.

Hard fork 23 node setup

Do this if you want to run a hard fork 23 node, if you want to run a hard fork 24 node, skip to the next section.

I feel like this info is getting copy pasted a lot, but I need to write it here again because there are some slight variations.

Install the dependencies:

So first of all install all of the dependencies :

# Required packages
sudo apt-get install -y \
    autoconf \
    automake \
    cmake \
    g++ \
    git \
    libbz2-dev \
    libsnappy-dev \
    libssl-dev \
    libtool \
    make \
    pkg-config \
    python3 \
    python3-jinja2 \
    doxygen

# Boost packages (also required)
sudo apt-get install -y \
    libboost-chrono-dev \
    libboost-context-dev \
    libboost-coroutine-dev \
    libboost-date-time-dev \
    libboost-filesystem-dev \
    libboost-iostreams-dev \
    libboost-locale-dev \
    libboost-program-options-dev \
    libboost-serialization-dev \
    libboost-signals-dev \
    libboost-system-dev \
    libboost-test-dev \
    libboost-thread-dev

# Optional packages (not required, but will make a nicer experience)
sudo apt-get install -y \
    libncurses5-dev \
    libreadline-dev \
    perl

build hive

in hf23, hive still uses the steem naming, so that's why you're seeing a lot of steem things all around. But it is hive

git clone git@gitlab.syncad.com:hive/hive.git
cd hive
git checkout testnet-0.23.0
git submodule update --init --recursive
mkdir build
cd build

cmake -DENABLE_COVERAGE_TESTING=ON -DBUILD_STEEM_TESTNET=ON -DLOW_MEMORY_NODE=OFF -DCLEAR_VOTES=ON -DSKIP_BY_TX_ID=ON -DCHAINBASE_CHECK_LOCKING=OFF ..

make -j$(nproc) steemd cli_wallet

if you got an error while doing the git clone try with this url:

https://gitlab.syncad.com/hive/hive.git

if you got an error while doing the git clone try with this url:

https://gitlab.syncad.com/hive/hive.git

Then wait a bit, depending on your machine, this process can take a while (it takes about 5 minutes on my machine)

running it

Run the node a few seconds and then exit steemd like so

./programs/steemd/steemd -d testnethf23/

this will create a testnethf23 directory with the default config file.

then open the config.ini file

nano testnethf23/config.ini

The default config is okay, but you need to add these fields

# tells the node which seeds he should listen to to get blocks
p2p-seed-node = 95.216.25.162:2001 
p2p-seed-node = 95.216.25.162:2002
p2p-seed-node = hf24.mintrawa.com:2001
# more seed nodes will be added later 

# Local http endpoint for webserver requests.
webserver-http-endpoint = 127.0.0.1:8090

# Local websocket endpoint for webserver requests.
webserver-ws-endpoint =127.0.0.1:8091

# Tell which port should be used to broadcast blocks as a seed node
p2p-endpoint = 0.0.0.0:2001

And edit the plugin field to this:

plugin = webserver p2p json_rpc witness account_by_key reputation market_history

plugin = database_api account_by_key_api network_broadcast_api reputation_api market_history_api condenser_api block_api rc_api

It's a bit overkill to put all these, but it'll make for a better experience if you want to mess with the cli on your own later.

Then run steemd again, this time we want it to stay up so setup a way for it to persist even if you log out, like screen. I personally use pm2 for the testnets just because it's convenient. This is not optimal for a production setup but it's fine for those testnets. (see https://pm2.keymetrics.io/docs/usage/quick-start/)

I run steemd like this with pm2:

pm2 start "./programs/steemd/steemd -d testnethf23 " --name hf23

It will take some time (a few minutes) before you get your blocks, but at some point you should be synched with the testnet and you should see messages like this:

3|hf23     | 1526617ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4415 by howo -- Block Time Offset: -382 ms
3|hf23     | 1532618ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4416 by howo -- Block Time Offset: -381 ms
3|hf23     | 1538612ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4417 by howo -- Block Time Offset: -387 ms
3|hf23     | 1544620ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4418 by howo -- Block Time Offset: -379 ms
3|hf23     | 1550615ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4419 by howo -- Block Time Offset: -384 ms

now your node is running ! So that's great, but it's not producing.

So you need to setup your witness.

Setting up your witness

cli_wallet

Open up the cli wallet:

./programs/cli_wallet/cli_wallet --server-rpc-endpoint="ws://127.0.0.1:8091"

it'll ask for a password, so execute:

set_password yourpassword

then unlock it using:

unlock yourpassword

Now execute those commands:

import initminer's key to do various actions
import_key 5JNHfZYKGaomSFvd4NUdQ9qMcEAC43kujbfjueTHpVapX1Kzq2n

create your account
create_account "initminer" "yourusername" "" true

power up from initminer:
transfer_to_vesting "initminer" "yourusername" "100.000 TESTS" true

then generate keys for your witness:
suggest_brain_key

it'll be something like this:

{
  "brain_priv_key": "STRANGE ROADWAY TWASOME MOLER CENTIMO THOFT COMPORT VARIANT OUTSWIM EGGER SCRIBE GLAVER CROWDED DOLLIER AWNED DOPPER",
  "wif_priv_key": "5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF",
  "pub_key": "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ"
}

Save these for later !

Then set yourself as a witness (notice how I used the pub_key generated above)

update_witness "yourusername" "http://example.org" "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ" {"account_creation_fee":"0.000 TESTS","maximum_block_size":65536,"sbd_interest_rate":0} true

and finally vote for yourself so you get scheduled for blocks

vote_for_witness "yourusername" "yourusername" true true

and we are done with cli_wallet ! but not with the setup.

final config

Open the config one final time to set yourself as a witness by adding these fields:

I use the private key that you generated with the cli above not the one that I put here. I put one here to show that you don't put quotes around the private key.

# name of witness controlled by this node (e.g. initwitness )
witness = "yourusername"

# WIF PRIVATE KEY to be used by one or more witnesses or miners
private-key = 5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF

restart the node and you should be producing blocks, the logs should look something like this:

3|hf23     | 2687621ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4595 by howo -- Block Time Offset: -378 ms
3|hf23     | 2690613ms witness_plugin.cpp:343        block_production_loo ] Generated block #4596 with timestamp 2020-08-25T19:44:51 at time 2020-08-25T19:44:51

Hard fork 24 node setup

Do this if you want to run a hard fork 24 node, if you want to run a hard fork 23 node, see the previous section.

this may feel like a copy paste from above but there are some slight variations.

Install the dependencies:

So first of all install all of the dependencies :

apt-get install -y \
        autoconf \
        automake \
        autotools-dev \
        build-essential \
        cmake \
        doxygen \
        git \
        libboost-all-dev \
        libyajl-dev \
        libreadline-dev \
        libssl-dev \
        libtool \
        liblz4-tool \
        ncurses-dev \
        python3 \
        python3-dev \
        python3-jinja2 \
        python3-pip \
        libgflags-dev \
        libsnappy-dev \
        zlib1g-dev \
        libbz2-dev \
        liblz4-dev \
        libzstd-dev

build hive

git clone git@gitlab.syncad.com:hive/hive.git
cd hive
git checkout master
git submodule update --init --recursive
mkdir build
cd build

cmake -DENABLE_COVERAGE_TESTING=ON -DBUILD_HIVE_TESTNET=ON -DLOW_MEMORY_NODE=OFF -DCLEAR_VOTES=ON -DSKIP_BY_TX_ID=ON -DCHAINBASE_CHECK_LOCKING=OFF ..

make -j$(nproc) hived cli_wallet

if you got an error while doing the git clone try with this url:

https://gitlab.syncad.com/hive/hive.git

if you got an error while doing the git clone try with this url:

https://gitlab.syncad.com/hive/hive.git

Then wait a bit, depending on your machine, this process can take a while (it takes about 5 minutes on my machine)

running it

Run the node a few seconds and then exit hived like so

export HIVE_HF24_TIME=1598558400 && ./programs/steemd/hived -d testnethf24/

note the env variable HIVE_HF24_TIME. It's very important that you put it every time you run hived. It's what states at which time the fork will execute.

For the first testnet its value should be 1598558400, for the second testnet its value should be 1598731200

this will create a testnethf24 directory with the default config file.

then open the config.ini file

nano testnethf24/config.ini

replace the existing config this one (stolen and edited from @gtg: @gtg/witness-update-release-candidate-for-eclipse-is-out)

# tells the node which seeds he should listen to to get blocks
p2p-seed-node = 95.216.25.162:2001 
p2p-seed-node = 95.216.25.162:2002
p2p-seed-node = hf24.mintrawa.com:2001
# more seed nodes will be added later 

# Local http endpoint for webserver requests.
webserver-http-endpoint = 127.0.0.1:8090

# Local websocket endpoint for webserver requests.
webserver-ws-endpoint =127.0.0.1:8091

log-appender = {"appender":"stderr","stream":"std_error"}
log-logger = {"name":"default","level":"info","appender":"stderr"}

backtrace = yes

plugin = webserver p2p json_rpc
plugin = database_api condenser_api

plugin = witness
plugin = rc

plugin = market_history
plugin = market_history_api

plugin = account_history_rocksdb
plugin = account_history_api

plugin = transaction_status
plugin = transaction_status_api

plugin = account_by_key
plugin = account_by_key_api

plugin = reputation
plugin = reputation_api

plugin = block_api network_broadcast_api rc_api

account-history-rocksdb-path = "blockchain/account-history-rocksdb-storage"

shared-file-size = 20G

shared-file-full-threshold = 9500
shared-file-scale-rate = 1000

flush-state-interval = 0

market-history-bucket-size = [15,60,300,3600,86400]
market-history-buckets-per-size = 5760

p2p-endpoint = 0.0.0.0:2001

transaction-status-block-depth = 64000
transaction-status-track-after-block = 46000000

webserver-http-endpoint = 127.0.0.1:8091
webserver-ws-endpoint = 127.0.0.1:8090

webserver-thread-pool-size = 256

It's a bit overkill to put all these, but it'll make for a better experience if you want to mess with the cli on your own later.

Then run hived again, this time we want it to stay up so setup a way for it to persist even if you log out, like screen. I personally use pm2 for the testnets just because it's convenient. This is not optimal for a production setup but it's fine for those testnets. (see https://pm2.keymetrics.io/docs/usage/quick-start/)

I run hived like this with pm2:

pm2 start "export HIVE_HF24_TIME=1598558400 && ./programs/hived/hived -d testnethf24" --name hf24

It will take some time (a few minutes) before you get your blocks, but at some point you should be synched with the testnet and you should see messages like this:

3|hf23     | 1526617ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4415 by howo -- Block Time Offset: -382 ms
3|hf23     | 1532618ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4416 by howo -- Block Time Offset: -381 ms
3|hf23     | 1538612ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4417 by howo -- Block Time Offset: -387 ms
3|hf23     | 1544620ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4418 by howo -- Block Time Offset: -379 ms
3|hf23     | 1550615ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4419 by howo -- Block Time Offset: -384 ms

now your node is running ! So that's great, but it's not producing.

So you need to setup your witness.

Setting up your witness

cli_wallet

Open up the cli wallet:

./programs/cli_wallet/cli_wallet --server-rpc-endpoint="ws://127.0.0.1:8091"

it'll ask for a password, so execute:

set_password yourpassword

then unlock it using:

unlock yourpassword

Now execute those commands:

import initminer's key to do various actions
import_key 5JNHfZYKGaomSFvd4NUdQ9qMcEAC43kujbfjueTHpVapX1Kzq2n

create your account
create_account "initminer" "yourusername" "" true

power up from initminer:
transfer_to_vesting "initminer" "yourusername" "100.000 TESTS" true

then generate keys for your witness:
suggest_brain_key

it'll be something like this:

{
  "brain_priv_key": "STRANGE ROADWAY TWASOME MOLER CENTIMO THOFT COMPORT VARIANT OUTSWIM EGGER SCRIBE GLAVER CROWDED DOLLIER AWNED DOPPER",
  "wif_priv_key": "5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF",
  "pub_key": "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ"
}

Save these for later !

Then set yourself as a witness (notice how I used the pub_key generated above)

update_witness "yourusername" "http://example.org" "TST71oeHrdBZEKDG1GaC75VKDaRGcXk2R8htD8J6wkP6p1zDRPzFZ" {"account_creation_fee":"0.000 TESTS","maximum_block_size":65536,"hbd_interest_rate":0} true

and finally vote for yourself so you get scheduled for blocks

vote_for_witness "yourusername" "yourusername" true true

and we are done with cli_wallet ! but not with the setup.

final config

Open the config one final time to set yourself as a witness by adding these fields:

# name of witness controlled by this node (e.g. initwitness )
witness = "yourusername"

# WIF PRIVATE KEY to be used by one or more witnesses or miners
private-key = 5KJUxvUSdAV6H7LiutRJMVwEVptmoaDhM73Cg1itzYjG8zs7tWF

Use the private key that you generated with the cli when doing suggest_brain_key. Note you don't put quotes around the private key.

restart the node and you should be producing blocks, the logs should look something like this:

3|hf23     | 2687621ms p2p_plugin.cpp:212            handle_block         ] Got 0 transactions on block 4595 by howo -- Block Time Offset: -378 ms
3|hf23     | 2690613ms witness_plugin.cpp:343        block_production_loo ] Generated block #4596 with timestamp 2020-08-25T19:44:51 at time 2020-08-25T19:44:51

Final words

And done ! I hope you'll consider joining us on this testnet campaign.

I'd like to remind you that the first testnet is due to hardfork (and be killed an hour later on the 27th at 20 UTC) and the second testnet will be hardfork on the 29th at 20:00 UTC.

Seed node

Go the extra mile and setup a seed node ! this is quite important as the testnet have very few seed nodes, so it helps a lot if you make one.

Your node is already configured to be a seed node so you just need to open the corresponding port:
(note that this command may vary depending on the firewall setup of your computer)

sudo iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 2001 -j ACCEPT

And now you can communicate the new seed node to everyone. Please post your ip as a comment of that post so that I can update this link to use your seed as well :).

Thank you for reading.

If you liked this writeup, please consider voting for @steempress it's the witness that I co-manage with @fredrikaa:

^{You can vote for our witness directly using Hivesigner here.}