In this first part of a series of 3 posts about setting up a Witness server we will see how to install the prerequisites and do some tuning.
Hardware
- CPU: Intel Xeon E3-1230v6 4 Cores/8 Threads 3.50 GHz
- RAM: 32GB DDR4 ECC 2133MHz
- HDD: 1 To SSD NVMe
How-To: Secure Ubuntu server (18.04)
Increase the security and usability of your Ubuntu server is very important. There are few configuration/install that you should take early on as part of the basic setup.
Root Login
Never use directly the user root and prefer to create a new user. For our case we will use an account named witness with sudo power
As root, run this command to add your new user
root@witness:~$ adduser witness
The system will ask you for a password, use a secure one! (Upper, Lower, Number, Special)
Adding user `witness' ...
Adding new group `witness' (1000) ...
Adding new user `witness' (1000) with group `witness' ...
Creating home directory `/home/witness' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
When done, you just need to fill the Full Name and to write y to validate.
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for witness
Enter the new value, or press ENTER for the default
Full Name []: witness
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] y
Not forgot to run this command to add your new user to the sudo group
root@witness:~$ usermod -aG sudo witness
Add Public Key Authentication (SSH)
Set up public key authentication for your new user. Setting this up will increase the security of your server by requiring a private SSH key to log in.
Generate The Key Pair for SSH
If you haven’t an SSH key pair already you can create it by following this process. To generate a new key pair, enter the following command (use the option -b 4096 for higher security) in your terminal.
witness@witness:~$ ssh-keygen -b 4096
Assuming your local user is witness, you will see the following output:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/witness/.ssh/id_rsa):
Hit return to accept and securing your keys with a strong passphrases!
Created directory '/home/witness/.ssh'
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
At the end you will have an output like this
Your identification has been saved in /home/witness/.ssh/id_rsa.
Your public key has been saved in /home/witness/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:XXXXXXXXXXXXXXXXXXX5GcBMBXXXXXXXXXXM witness@xxx
The key's randomart image is:
+---[RSA 4096]----+
|XXXX |
|XX |
|X X XX XXX |
|XXX X XXX |
|X XXXX X XX |
| XX XX |
| X |
| X |
| |
+----[SHA256]-----+
You have now 2 files in the directory /home/witness/.ssh/ a private key id_rsa and a public key id_rsa.pub
Rename the Public Key
if you generate the keys directly on the server rename the public key id_rsa.pub in authorized_keys like this
witness@witness:~/.ssh$ sudo mv id_rsa.pub authorized_keys
And retrieve the private key id_rsa on your computer (not let it on the server!)
Putty case
Putty users, you need to load the private key id_rsa in PuTTYgen then save the private key for have it in .ppk format
Disabling Password Authentication and change the default TCP port
If you were able to login to your account using SSH with the private key then you have successfully configured SSH key-based authentication to your account. We can now remove the authentication with password only in the ssh config file (not hesitate to change also the default port number 22).
Edit the ssh config file
witness@witness:~/.ssh$ sudo vim /etc/ssh/sshd_config
Uncomment the # Port 22 line and change the number to 8022.
Port 8022
Change the value of PermitRootLogin to no
PermitRootLogin no
Change the value of PasswordAuthentication to no
PasswordAuthentication no
Save and close the file when you are finished. To actually implement the changes we just made, you must restart the service.
witness@witness:~/.ssh$ sudo service ssh restart
Basic firewall rules
The default firewall configuration tool for Ubuntu is ufw. It’s an interface to iptables.
Allow our specific TCP port for ssh
To add our TCP port 8022 for ssh you can use the command below
witness@witness:~$ sudo ufw allow proto tcp from any to any port 8022
Allow other ports needed for a Witness server
a Witness server use per default the TCP port 2001 and optionally need TCP port for running ws, http and rpc plugins. To allow them, you can use the command below as exemple (the port 443 is for https!)
witness@witness:~$ sudo ufw allow proto tcp from any to any port 2001,443
Activate ufw
To enable ufw, use this command:
witness@witness:~$ sudo ufw enable
Answer y to the question for proceed
Command may disrupt existing ssh connections. Proceed with operation (y|n)?
To list the active rules you can use the command
witness@witness:~$ sudo ufw status
The output will be something like this
Status: active
To Action From
-- ------ ----
8022/tcp ALLOW Anywhere
2001/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
8022/tcp (v6) ALLOW Anywhere (v6)
2001/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
Fail2ban
Fail2ban scan the log files of the server and bans IPs that show the malicious signs. Like for exemple too many password failures, seeking for exploits, etc.. It work as a service and create rules that automatically alter iptables configuration. All based on a predefined number of unsuccessful login attempts. This will allow the server to respond to illegitimate access attempts without manual intervention.
Install
Fail2ban is in package list of Ubuntu. To install it from a command prompt do like this (update first).
witness@witness:~$ sudo apt-get update
witness@witness:~$ sudo apt-get install fail2ban
Configure
Configuration files are in the /etc/fail2ban directory. Stop the service
witness@witness:~$ sudo service fail2ban stop
Duplicate the config file jail.conf to keep default options inside (this file can be overwriten when update applied). Put all the specific settings in jail.local
witness@witness:~$ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
Now we can modify the jail.local file to adjust to our server case
witness@witness:~$ sudo vim /etc/fail2ban/jail.local
The base
Setup the base consist to add one or more source @ip to ignore, the bann time and the number of retry allowed. To do this we need to modify the variables:
- ignoreip = @ip to ignore (separated by a space).
- bantime (in second) = parameter for banned client (default 10 minutes).
- findtime (in second) = a window of time to find a specific number of tries (see below).
- maxretry = number of tries before being banned.
By default ban 10 minutes a client after 5 tries in 10 minutes.
Email alerts
Configure email alerts with the variable destemail, sendername, and mta. To use it you need the Ubuntu package sendmail!
SSH
To activate a service juste need a line enabled = true in the appropriate section [ssh] and modify the port (remember we use specific!)
[sshd]
enable = true
# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
#mode = normal
port = ssh,8022
logpath = %(sshd_log)s
backend = %(sshd_backend)s
After editing, start the fail2ban service
sudo service fail2ban start
If you go want deeper in fail2ban I suggest this excellent post from Linode
Fail2ban Client
use the command fail2ban-client with one of these command to action/check information:
- start: Starts the server and jails.
- reload: Reloads configuration files.
- reload JAIL: Replaces JAIL with the name of a Fail2ban jail; this will reload the jail.
- stop: Stop the server.
- status: Show the status of the server, and enable jails.
- status JAIL: Show the status of the jail, including any currently-banned IPs.
Replace JAIL by the service you want to check, exemple with ssh
witness@witness:~$ sudo fail2ban-client status
Status
|- Number of jail: 1
`- Jail list: sshd
Or
witness@witness:~$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Filter
| |- Currently failed: 0
| |- Total failed: 0
| `- File list: /var/log/auth.log
`- Actions
|- Currently banned: 0
|- Total banned: 0
`- Banned IP list:
UTILITIES and TUNING
NMAP
Install and use the port scanning NMAP to check your open ports
witness@witness:~$ sudo apt-get install nmap
To scan the server use the command
witness@witness:~$ nmap -sV -p 1-65535 localhost
Starting Nmap 7.60 ( https://nmap.org ) at 2020-03-31 05:06 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000070s latency).
Not shown: 65532 closed ports
PORT STATE SERVICE VERSION
53/tcp open domain
953/tcp open rndc?
8022/tcp open ssh OpenSSH 7.6p1 Ubuntu 4ubuntu0.3 (Ubuntu Linux; protocol 2.0)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 12.89 seconds
Where:
-sV = service identification
-p = list of port to scan (range separate by -)
Time Synchronization with NTP
For a better time synchronization, we will use the NTP packet.
Before installing ntpd, we should turn off timesyncd (the default manager for time synchronization):
witness@witness:~$ sudo timedatectl set-ntp no
Verify that timesyncd is off:
witness@witness:~$ timedatectl
Local time: Tue 2020-03-31 05:32:52 UTC
Universal time: Tue 2020-03-31 05:32:52 UTC
RTC time: Tue 2020-03-31 05:32:52
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
systemd-timesyncd.service active: no
RTC in local TZ: no
We can see that systemd-timesyncd.service active is set to no
To install the NTP packet run the command
witness@witness:~$ sudo apt install ntp
ntpd will be started automatically after install. You can check the version with the command
witness@witness:~$ sntp --version
sntp 4.2.8p10@1.3728-o (1)
We need to edit the ntp.conf file to replace the default ubuntu.pool with something closer. To do this, you first need to check here https://www.ntppool.org/zone/@ which are the closest pool servers.
Edit the ntp.conf file
witness@witness:~$ sudo vim /etc/ntp.conf
Replace
# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
pool 0.ubuntu.pool.ntp.org iburst
pool 1.ubuntu.pool.ntp.org iburst
pool 2.ubuntu.pool.ntp.org iburst
pool 3.ubuntu.pool.ntp.org iburst
By your nearest NTP pool servers (fr in my case)
# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
pool 0.fr.pool.ntp.org iburst
pool 1.fr.pool.ntp.org iburst
pool 2.fr.pool.ntp.org iburst
pool 3.fr.pool.ntp.org iburst
Restart the service
witness@witness:~$ sudo service ntp restart
Check if everything OK
witness@witness:~$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
0.fr.pool.ntp.o .POOL. 16 p - 64 0 0.000 0.000 0.000
1.fr.pool.ntp.o .POOL. 16 p - 64 0 0.000 0.000 0.000
2.fr.pool.ntp.o .POOL. 16 p - 64 0 0.000 0.000 0.000
3.fr.pool.ntp.o .POOL. 16 p - 64 0 0.000 0.000 0.000
Increase the max open files limit
To avoid the problem of Too many open files we need to make a few changes to the ulimit
Edit the limits.conf file
witness@witness:~$ sudo vim /etc/security/limits.conf
Add the following lines
root soft nproc 999999
root hard nproc 999999
root soft nofile 999999
root hard nofile 999999
witness soft nproc 999999
witness hard nproc 999999
witness soft nofile 999999
witness hard nofile 999999
Edit the common-session file
witness@witness:~$ sudo vim /etc/pam.d/common-session
Add the following line
session required pam_limits.so
Reboot the server for the change to take effect.
Check the open files limit (need to be connected with the right user!)
witness@witness:~$ ulimit -n
999999
Install SCREEN and LNAV
Install the package screen (a terminal multiplexer) which will continue to run either if the window is not visible or if you get disconnected if not already installed.
witness@witness:~$ sudo apt install screen
Install the package lnav a log file navigator, streamer, regex searchable
witness@witness:~$ sudo apt install lnav
[WITNESS server - part II] HIVE - The manual way
If you liked Upvote, Follow, Reblog or Cross post are welcome - @mintrawa