Samhain Labs

By Rainer Wichmann rainer@nullla-samhna.de    (last update: Nov 21, 2008)

Introduction

During 2005, bute force attacks on the ssh (secure shell) service became pretty popular. These attacks are based on a rather simple idea: use an automated program for trying, one after the other, many combinations of standard or frequently used account names and likewise frequently used password (e.g.: guest/guest).

Defence methods

There are a number of methods to defend against such brute force attacks. The following list is intended to give an overview of them, and briefly mention their respective advantages and disadvantages.

• Strong passwords
• RSA authentication
• Using 'iptables' to block the attack
• Using the sshd log to block attacks
• Using tcp_wrappers to block attacks
• Port knocking

Strong passwords

Neither in a dictionary, nor trivial variations of trivial passwords (guest1 is just as bad as guest). Using the initials of the words in some sentence is a simple method to have a strong password that is easy to remind (Peter, Paul, and Mary went to school yesterday = PPaMwtsy).

• Advantage: Simple
• Disadvantages:
  • Requires enforcement (regular checks of user passwords with tools like e.g. john ("John the Ripper").
  • Does not reduce the (network, sshd) load caused by the attacks.

RSA authentication

If you don't use passwords, but only RSA keys for authentication, a brute force search for a valid password will obviously be useless.

(1) Generate an RSA key with ssh-keygen -t rsa. This will create the files /home/username/.ssh/id_rsa (the private key) and /home/username/.ssh/id_rsa.pub (the public key).

sh$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
The key fingerprint is:
32-digit_hexadecimal_fingerprint username@hostname

(2) On each machine to which where you want to login, put /home/username/.ssh/id_rsa.pub into /home/username/.ssh/authorized_keys. This file can hold more than one key, so it may be wise to concatenate the freshly generated key.

sh$ cat /home/username/.ssh/id_rsa.pub >> /home/username/.ssh/authorized_keys

(3) On each machine from which you want to login, place the file /home/username/.ssh/id_rsa into the directory /home/username/.ssh/.

(4) Disable password-based login by setting 'PasswordAuthentication no' in /etc/ssh/sshd_config, and restart the sshd daemon with /etc/init.d/sshd restart

• Advantage: Pretty secure, if done properly
• Disadvantages:
  • Users may use private keys without setting a passphrase to protect them. This implies that getting access to the private key would allow to login to any machine where the corresponding public key is installed.
  • Non-tech users may need help to generate RSA key for themselves (and for using them).
  • It is neccessary to carry the private key along to login from another host.

Using 'iptables' to block the attack

It is possible to set up iptables rules to block ssh attacks. The following ruleset (seen in the blog of Andrew Pollock) will allow at most 3 connections per minute from any host, and will block the host for another minute if this rate is exceeded.

es -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set \ 
 --name SSH -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl \
 --name SSH -j LOG --log-prefix "SSH_brute_force "
iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 \
 --hitcount 4 --rttl --name SSH -j DROP

For whitelisting, a possible variation (also described by Andrew Pollock) would be:

(1) Create a custom chain for whitelisting first:

iptables -N SSH_WHITELIST

(2) Whitelist any host(s) that you like:

iptables -A SSH_WHITELIST -s TRUSTED_HOST_IP -m recent --remove --name SSH -j ACCEPT

(3) Add the blocking rules:

iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set \
 --name SSH
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_WHITELIST
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update \
 --seconds 60 --hitcount 4 --rttl --name SSH -j ULOG --ulog-prefix SSH_brute_force
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update \
 --seconds 60 --hitcount 4 --rttl --name SSH -j DROP

• Advantage: Transparent for users
• Disadvantages:
  • Does not distinguish between successful logins and unsuccessful login attempts (i.e. three successful logins within one minute will trigger just like three unsuccessful login attempts).
  • As pointed out by one reader (L. Hamel), this method requires 'recent' (http://snowman.net/projects/ipt_recent/), an ipfilters patch that may not (yet) be in every distro's default kernel. So to use this technique with typical linux distros, you may have to patch the kernel.
  • R. McIntosh kindly notices that Debian/Ubuntu include this, but RHEL3/4/5 do not.

Using the sshd log to block attacks

It is possible to scan the syslog entries written by the sshd daemon for ongoing attacks, and block the attacker. There are several programs/scripts available that are specifically written for this purpose:

ssdfilter uses iptables for blocking (i.e. it dynamically adds custom firewall rules to block a specific attacker). You need to run sshdfilter instead of sshd. Sshdfilter will then start sshd and monitor its log.

Fail2Ban is a Python script which adds custom firewall rules to block an attacker. It can use the iptables, ipfwadm, or ipfw.

DenyHosts does not use firewall rules to block an attack. Rather, it writes blocking rules to /etc/hosts.deny. Thus, it requires an sshd daemon compiled with support for tcp_wrappers (default on most Linux distributions). Like Fail2Ban, DenyHosts is a Python script.

To find out whether your sshd daemon supports tcp_wrappers, you could use the command ldd /usr/sbin/sshd | grep libwrap. With tcp_wrappers supported you should get a line similar to libwrap.so.0 => /lib/libwrap.so.0 (0xb7f7e000) (numbers may be different). Without tcp_wrappers support, the output will be empty.

sh$ ldd /usr/sbin/sshd | grep libwrap
      libwrap.so.0 => /lib/libwrap.so.0 (0xb7f7e000)

Another method to test for tcp_wrappers support would be to add the line sshd: 127.0.0.1 to the file /etc/hosts.deny, and then try to connect to the local ssh server with the command ssh localhost. As you have just blocked localhost, this should fail with the error message ssh_exchange_identification: Connection closed by remote host:

sh$ ssh localhost
ssh_exchange_identification: Connection closed by remote host

• Advantages:
  • Transparent for users
  • Can distinguish between unsuccessful login attempts (which are blocked) and successful logins (which are not blocked).
• Disadvantages:
  • Many ssh attacks are fast and short (many connections in a short time). Thus, running a cron script to check the sshd logs for an ongoing attack may be pointless, as it may notice the attack only after it has finished (and the next attack will come from a different machine anyway).
  • Continuously monitoring the sshd log for an attack requires to run yet another daemon (in the case of sshdfilter, you have to run sshdfilter instead of sshd anyway).

Using tcp_wrappers to block attacks

In the preceding section the DenyHosts script was discussed, which scans the sshd logs to detect an attack, and then blocks it with a rule in /etc/hosts.deny. However, it is actually not necessary to scan the logs. It is possible to let the tcp wrapper library start a script whenever a connection is made, and let this script add rules to /etc/hosts.deny or /etc/hosts.allow, if the connecting host should be blocked.

(1) Download the sshblock.sh shell script (PGP signature), and copy it to /usr/local/bin/sshblock.sh (or wherever you like).

(2) Make it executable using the command chmod 755 /usr/local/bin/sshblock.sh

(3) Add the following three lines at the bottom of /etc/hosts.allow

#__START_SSHBLOCK__
#__END_SSHBLOCK__
sshd : ALL : spawn (/usr/local/bin/sshblock.sh %a)&

This will call the script /usr/local/bin/sshblock.sh for each ssh connection. The script will receive the remote IP address as the first and only argument. The script will write a temporary file /root/hosts.allow, and copy it to /etc/hosts.allow if it differs from that.

Requirements:

• The sshd daemon must support tcp_wrappers (see preceding section on methods to test this).
• The date command must support the '%s' format specifier to print seconds after the Epoch (use date +%s to test). Both GNU/Linux and FreeBSD date support this.

Configuration: at the top of the script, there are four variables that can be set: DONTBLOCK is the prefix of an address block that you do never want to be blocked (i.e. your own domain). The default is 192.168, which you can safely keep if you have no own domain, as this is a 'private' address block). At most (2) BURST_MAX connections from a host within BURST_TIM seconds are allowed (default: 5 within 60 sec), and blocks are removed after PURGE_TIM seconds (default: 3600 sec). In the script, this looks like:

# your own domain 
DONTBLOCK=192.168

# block host if more than BURST_MAX connections within BURST_TIM seconds
BURST_MAX=5
BURST_TIM=60

# remove block after PURGE_TIM seconds
PURGE_TIM=3600

• Advantage: Transparent for users
• Disadvantages:
  • Does not distinguish between successful logins and unsuccessful login attempts (i.e. five successful logins within one minute will trigger just like five unsuccessful login attempts).
  • To remove a block, the script must run, which requires a connection from a non-blocked host (or follow a suggestion by Andreas Rizzi and install a cron job that calls the script with the parameter 127.0.0.0 once per hour).

Port knocking

As pointed out by Imre Veres, another solution to the problem is using knockd, which eliminates the need for having ssh listen on an open port.

Knockd watches predefined patterns in iptables' log, e.g. one hit to port 6356, one hit to port 9356 and two hits to port 3356. I.e. this is equivalent to knocking at a closed door with a "code" that is recognized by knockd. Knockd will then use iptables to open a predefined port (e.g. tcp/22 for sshd) for a predefined time (e.g. one minute). If a ssh session is opened within that time frame, it will remain open, though the ssh port will get closed by knockd after the predefined timeframe expires.

Kevin Hilton recommends fwknop on the grounds that "its much more sophisticated, very easy to set up (at least for ubuntu — probably other distributions as well), allows for encrypted communications, randomizes the source port, and is supported."

Michael Rash points out that fwkop supports Single Packet Authorization, which does not suffer from the traffic monitoring attack mentioned below under 'Disadvantages'.

• Advantage: Fairly secure, as long as the attacker cannot monitor your traffic (since that could allow to determine and replay the portknock sequence)
• Disadvantages:
  • Rather complicated scheme, not suitable for 'mere mortals'.
  • Requires a suitable program (e.g. knockd-client) for "port knocking" on the client (as well as knockd on the server)
  • As pointed out by Bond Masuda, it is pretty easy to detect the traffic pattern of 'portknocks followed by ssh connection', once an attacker can monitor your traffic. Thus simple portknocking provides no security against attackers who can spy on the local network. This problem is addressed by Single Packet Authorization in fwknop.