SSH Security - Stopping Server Scanners

I have to maintain my SSH connection to the home network for various reasons.  Because of that, my SSH connection is open to the world - and I'm a paranoid.  So, what do I do to maintain my sanity?  Most Linux distributions include a handy little program called "swatch".  It's a "simple watcher" application that uses regular expressions (hooray for Perl people!) and acts when something is "found".

Here's an example.  Let's say you opened up your log file and see a number of these :

Apr 12 09:36:23 servername sshd[11307]: User root from 61.174.49.113 not allowed because not listed in AllowUsers
Apr 12 09:36:23 servername sshd[11310]: input_userauth_request: invalid user root
Apr 12 09:36:23 servername unix_chkpwd[11316]: password check failed for user (root)
Apr 12 09:36:23 servername sshd[11306]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.174.49.113  user=root
Apr 12 09:36:23 servername sshd[11308]: reverse mapping checking getaddrinfo for 113.49.174.61.dial.wz.zj.dynamic.163data.com.cn [61.174.49.113] failed - POSSIBLE BREAK-IN ATTEMPT!
Apr 12 09:36:23 server name sshd[11308]: User root from 61.174.49.113 not allowed because not listed in AllowUsers

For anyone NOT security minded, here are a couple of quick points :

• You know immediately that someone is scanning your server, trying to find an open account that is easily compromised.
• You ALSO know that their attack fills up your network pipe - and communication is vital.

So, what do you do?  We simply watch the logs, and then trigger adding a route to the loop back interface.  This causes us to suddenly become "unresponsive" to whomever is doing the scan.  If, after a minute, they continue to scan, we simply block for a little longer each time, ultimately just making it semi-permanent.  So, here's how.

We create an rc file containing our instructions.  Create a configuration file in /etc (say, /etc/swatchrc), and add the following :

# Bad authentication attempts from ssh
watchfor   /Failed password for/
        exec "/usr/local/bin/failed_password.sh $1 $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 $12 $13 $14 $15"

This simply looks for the regular expression /Failed password for/ in /var/log/secure, and then runs a script of ours, /usr/local/bin/failed_password.sh.  This script consists of simple rules :

#!/bin/bash

ATTEMPTS_LIMIT=4
NOTIFICATIONS_TO='email@address.com'

# get the IP address :
IP=`echo $* | sed 's/^.* from //' | awk '{print $1}' | sed 's/::ffff://'`

# get the number of attempts from this IP :
ATTEMPTS=`grep $IP /var/log/secure | grep "Failed password for"  | wc -l`

if [ $ATTEMPTS -gt $ATTEMPTS_LIMIT ]; then

 # black list the IP by sending it to the loop back interface
 route add $IP lo

 # in the calculated number of minutes, un black list the IP
 # but, make this somewhat exponential
 ATTEMPTPOVER=`expr $ATTEMPTS - $ATTEMPTS_LIMIT`
 let MINUTES=$ATTEMPTPOVER*3
 echo "route del $IP lo 2> /dev/null" | at now +$MINUTES minutes 2>&1 > /tmp/.bad_user.$$

 # since we get a lot of people from China and Europe scanning
 # us, let's only send a notification if we hit a count of 5, or more than 20 attempts

 # first, five attempts
 if [ $ATTEMPTS -eq 5 ]; then
  # now let's send a notification for good measure
  (hostname ; echo $* ; echo "IP=$IP" ; echo "ATTEMPTS=$ATTEMPTS" ; \
   echo "Blocking for $MINUTES minutes" ; \
   cat /tmp/.bad_user.$$ ) | Mail -s "Scan Running From $IP" $NOTIFICATIONS_TO
 fi
 # next, 20 or more - and, let's simply iptables them until the next reboot
 if [ $ATTEMPTS -gt 19 ]; then
  /sbin/iptables -I INPUT 4 -s $IP -j REJECT
  # now let's send a notification for good measure
  (hostname ; echo $* ; echo "IP=$IP" ; echo "ATTEMPTS=$ATTEMPTS" ; \
   echo "Blocking for $MINUTES minutes" ; \
   cat /tmp/.bad_user.$$ ) | Mail -s "Permanently Blocking $IP" $NOTIFICATIONS_TO
 fi

 # also, ensure we log that we are blocking, and for how long
 /bin/logger -p authpriv.warn "Saw auth attempt $ATTEMPTS from $IP - blocking for $MINUTES minutes"
fi

# clean up after ourselves
rm -f /tmp/.bad_user.$$

The script is explained by comments, but here's the gist. Ths script is executed every time /var/log/secure matches a "Failed password for" along with the full log line (including the IP address). It then "greps" for that IP in /var/log/secure and grabs a total of the failed attempts. If that number of events is greater than ATTEMPTS_LIMIT (4), we route anything to that IP through loopback and schedule a job to delete that route $MINUTES out (calculated as the number of attempts over the ATTEMPTS_LIMIT multiplied by 3). Then, if we have 5 attempts - it's a script that someone is letting run, so we send a single notification. If we get to 20 attempts (the last one is nearly an hour of being blocked before it can try again), we send a new notification that we've blocked the IP, and we run an iptables command to insert it into our firewall (the block should disappear on the next host reboot). So, that's how it works.

Next, we have to start swatch up :

/usr/bin/swatch --config-file=/etc/swatchrc --tail-file=/var/log/secure \
 --awk-field-syntax --tail-args "-F" &

Also, make sure you add it to your /etc/rc.local in order to automatically start it up on boot (e.g. in case of a power outage).