• Skip to main content
  • Skip to footer

NetworkJutsu

Networking & Security Services | San Francisco Bay Area

  • Blog
  • Services
  • Testimonials
  • About
    • About Us
    • Terms of Use
    • Privacy Policy
  • Contact Us

Ubuntu

FreeRADIUS 3.0 with Two-Factor Authentication (2FA)

06/01/2018 By Andrew Roderos 13 Comments

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Last year, I talked about migrating my FreeRADIUS server with two-factor authentication (2FA) to a Docker container. Today, I will cover how to configure FreeRADIUS 3.0 with two-factor authentication using Google Authenticator in a Docker container with Ubuntu 18.04 image.

Related: What is multi-factor authentication (MFA)?

The new version of Ubuntu Server LTS edition (18.04 at this time of writing) changed the FreeRADIUS version from 2.x to 3.0. The change made my old post as invalid. With that said, I want to share my working configuration with you.

If you wish to learn more about FreeRADIUS, there is a book out there available for purchase. However, it is an older book, so you may need to do some more research. Though, one of the Amazon reviews mentioned that the difference between 2.x and 3.0 are minor so it may still be helpful.

Writing FreeRADIUS 3.0 Dockerfile

I am sure there are FreeRADIUS 3.0 Docker images out there, but I like to write my own as much as I can. By writing a Dockerfile, it helps me learn more about Linux and Docker. I do, however, recognize that I am running the Docker container as root. As I learn more about this, I will eventually write one that isn’t using root. If you decide to copy this Dockerfile, beware the security risks running root on your container.

Without further ado, below is my Dockerfile that I wrote that satisfies my needs.

# Use Base Ubuntu image
FROM ubuntu:18.04
# Author of this Dockerfile
MAINTAINER NetworkJutsu <networkjutsu.com>
# Update & upgrades
RUN apt-get update && apt-get dist-upgrade -y
# Install FreeRADIUS and Google Authenticator
RUN apt-get install freeradius libpam-google-authenticator -y
# Clear local repo
RUN apt-get clean
# Add user to container with home directory
RUN useradd -m -d /home/networkjutsu -s /bin/bash networkjutsu
# Add password to networkjutsu account
RUN echo 'networkjutsu:letsmakemypasswordgreatagain' | chpasswd
# Edit /etc/pam.d/radiusd file
RUN sed -i 's/@include/#@include/g' /etc/pam.d/radiusd
RUN echo "auth requisite pam_google_authenticator.so forward_pass secret=/etc/freeradius/3.0/networkjutsu/.google_authenticator user=freerad" >> /etc/pam.d/radiusd
RUN echo "auth required pam_unix.so use_first_pass" >> /etc/pam.d/radiusd
# Edit /etc/freeradius/3.0/mods-config/files/authorize file
# This is the real file for /etc/freeradius/3.0/users
RUN sed -i '1s/^/# Instruct FreeRADIUS to use PAM to authenticate users\n/' /etc/freeradius/3.0/mods-config/files/authorize
RUN sed -i '2s/^/DEFAULT Auth-Type := PAM\n/' /etc/freeradius/3.0/mods-config/files/authorize
# Copy existing /etc/freeradius/sites-available/default file to container
# This is the real file for /etc/freeradius/3.0/sites-enabled/default
COPY default /etc/freeradius/3.0/sites-available/default
# Change owner of the file to freerad
RUN chown freerad:freerad /etc/freeradius/3.0/sites-available/default
# Copy existing /etc/freeradius/clients.conf file to container
COPY clients.conf /etc/freeradius/3.0/clients.conf
# Copy existing .google_authenticator file to container
COPY .google_authenticator /home/networkjutsu
# Create a symbolic link
RUN ln -s /etc/freeradius/3.0/mods-available/pam /etc/freeradius/3.0/mods-enabled/pam
# Create a folder in /etc/freeradius equal to the user name
RUN mkdir /etc/freeradius/3.0/networkjutsu
# Change owner of the directory to freerad
RUN chown freerad:freerad /etc/freeradius/3.0/networkjutsu
# Copy .google_authenticator file to /etc/freeradius/networkjutsu
RUN cp /home/networkjutsu/.google_authenticator /etc/freeradius/3.0/networkjutsu
# Change owner to freerad
RUN chown freerad:freerad /etc/freeradius/3.0/networkjutsu/.google_authenticator
# Expose the port
EXPOSE 1812/udp 1813/udp 18120/udp
# Run FreeRADIUS as a foreground process
CMD ["freeradius","-f"]

FreeRADIUS changes in Ubuntu

If you compare my old post and this post, you could tell that the directories have changed from /etc/freeradius to /etc/freeradius/3.0. One of the reasons why my old post would not work in Ubuntu 18.04.

Another change is the /etc/freeradius/3.0/users file. It is now a symbolic link compared to a regular file in Ubuntu 16.04. That said, I had to edit the original file and not the symbolic link.

The last change, at least for my purposes, is the requirement to create a symbolic link for the /etc/freeradius/3.0/mods-available/pam file. We need this file to enable PAM, without it the two-factor authentication wouldn’t work.

FreeRADIUS configuration files

I copied a lot of configuration files to the container because it was much faster for me to do it in a text editor than trying to figure out the proper sed commands. I am still new to sed command so it will take me several minutes or hours to figure out a simple thing to do in VI editor. While it will help me learn more about it, I haven’t had much time on my hands lately.

Some of the configuration files may have changed contents as a result of the upgrade. However, my old post covered all the changes I’ve made to them. Well, you could say I revised /etc/freeradius/3.0/radiusd file. The revision was very minor. I only did it because I wanted to show how to edit files without using a text editor, like VI editor.

Final Words

The changes to FreeRADIUS in Ubuntu 18.04 is minor, at least for my purposes. However, if you decide to upgrade your host or edit the Dockerfile to use the latest Ubuntu version without making the changes covered here, then it will break your instance.

A few weeks ago, I made a mistake of just changing the FROM ubuntu:16.04 to FROM ubuntu:18.04 and broke my FreeRADIUS container. If the FreeRADIUS version didn’t change, upgrading the OS would’ve been easy and fast compared to a VM. One of the reasons why I like to use Docker container as much as possible.

With this FreeRADIUS container, you could point your devices to this server as your primary RADIUS server. Since this server also makes use of Google Authenticator, you gain two-factor authentication feature. I use this container for my remote access VPN at home and also pointing my networking devices that support RADIUS authentication.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

You might also like to read

Adding Two-Factor Authentication to FreeRADIUS
Securing SSH with Google Authenticator
Adding Two-Factor Authentication to TACACS+

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Adding Two-Factor Authentication to TACACS+

06/02/2017 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Several months ago, I covered how to add two-factor authentication (2FA) to FreeRADIUS using Google Authenticator. Today, I will cover the TACACS+ version of it.

Related: What is multi-factor authentication (MFA)?

I’ve written a blog post on how to build tac_plus server using Ubuntu. The guide was written in 2011, while it’s an old blog post, the instructions are still valid using Ubuntu Server 16.04. Please use that guide on how to build one, then use this guide to add multi-factor authentication (MFA) to TACACS+.

Related: Deploying TACACS+ on a Docker container

Installing Google Authenticator PAM

It is super easy to install Google Authenticator on Ubuntu. Below is the command we need to install Google Authenticator PAM on Ubuntu.

$ sudo apt-get install libpam-google-authenticator -y

Configure tac_plus

As mentioned earlier, the instructions in my old blog post are still valid. We’re going to use only some of them in this post for the purpose of demonstration only.

By default, the /etc/tacacs+/tac_plus.conf file looks like this:

accounting file = /var/log/tac_plus.acct
key = testing123
user = DEFAULT {
       login = PAM
       service = ppp protocol = ip {}
}

Let’s change the key and user information fields to look something like this:

accounting file = /var/log/tac_plus.acct
key = tacacskey1234
user = tacacsuser {
        member = Administrators
}
group = Administrators {
        default service = permit
        login = PAM
        enable = file /etc/passwd
}

Restart TACACS+ daemon

Since we made a change to our tac_plus config file, we need to restart the service for our changes to take effect. Issue the command below.

$ sudo /etc/init.d/tacacs_plus restart
[ ok ] Restarting tacacs_plus (via systemctl): tacacs_plus.service.

An alternative command is shown below.

$ sudo service tacacs+ restart

Generating Google Authenticator Secret Key

This step is covered in my old blog post so head over there and skip to the generating the secret key section. Alternatively, we could use the same secret key(s) from another system with Google Authenticator. However, this is not the recommended practice.

To get the secret key from another system, just copy and paste the ~/.google_authenticator file of each user, like the one below.

tacacsuser@tacplus:~$ more .google_authenticator
UXQLCMOLT2QLSMVE
" RATE_LIMIT 3 30 1436015893
" DISALLOW_REUSE 39787632
" TOTP_AUTH
55312114
13740459
80118802
81859009
79311140

If you copy and paste it to a file, make sure that the permission is set to read only.

tacacsuser@tacplus:~$ ls -l .google_authenticator
-rw-rw-r-- 1 tacacsuser tacacsuser 129 May 29 17:54 .google_authenticator
tacacsuser@tacplus:~$ chmod 400 .google_authenticator
tacacsuser@tacplus:~$ ls -l .google_authenticator
-r-------- 1 tacacsuser tacacsuser 129 May 29 17:54 .google_authenticator

Configuring TACACS+ PAM

Since we instructed tac_plus to use PAM, we now need to create a file called /etc/pam.d/tac_plus, so PAM knows what to do. The file should look like:

$ more /etc/pam.d/tac_plus
auth requisite pam_google_authenticator.so forward_pass
auth required pam_unix.so use_first_pass

IOS configuration

Before we can verify that our tac_plus config is working, let’s configure a CSR1000V router running IOS-XE.

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#aaa new-model
R1(config)#aaa authentication login default group tacacs+ enable
R1(config)#aaa authentication enable default group tacacs+ enable
R1(config)#aaa session-id common
R1(config)#tacacs-server host 192.168.250.250
 Warning: The cli will be deprecated soon
 'tacacs-server host 192.168.250.250'
 Please move to 'tacacs server <name>' CLI
R1(config)#tacacs-server directed-request
R1(config)#tacacs-server key tacacskey1234
R1(config)#end
R1#

While the configuration above still works, it is a good idea to move towards the new way of doing things. Here’s the new way of configuring TACACS+:

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#aaa new-model
R1(config)#aaa authentication login default group tacacs+ enable
R1(config)#aaa authentication enable default group tacacs+ enable
R1(config)#aaa session-id common
R1(config)#tacacs server tac_plus
R1(config-server-tacacs)# address ipv4 192.168.250.250
R1(config-server-tacacs)# key tacacskey1234
R1(config-server-tacacs)#end
R1#

Verification

This CSR1000V router is using version 15.4(2)S image. The output may vary depending on what platform and the IOS version.

$ ssh tacacsuser@192.168.250.250
Password & verification code:
R1>en
Password:
R1#

For completeness sake, I will list the passwords entered in the example above. The tacacsuser account is a valid account on the Ubuntu server running TACACS+ daemon. For example, the tacacsuser account has a password of tacacsuserpassword1234. Next, the verification code is the six-digit number displayed on Google Authenticator app. For example, the six-digit number is 567 890. With this example, the user will enter tacacsuserpassword1234567890 in the password & verification prompt.

For entering the privileged EXEC mode, we’ll again use tacacsuser’s password. If we look at the tac_plus config file, the enable = file /etc/passwd is what we defined.

Final Words

I am quite biased towards TACACS+. One of the reasons why is because of the command authorization piece. TACACS+ authentication and authorization are completely separate. That said, we could assign different command authorization level for the user or group.

With RADIUS, it combines authentication and authorization. Once the user authenticates successfully, the access-accept packet sent by RADIUS server to the device contain authorization information as well. If we configure the device similar to the example here, then the user will have full access.

To overcome RADIUS’ drawback, we could configure the device to use a local enable secret. This password is then shared only with the necessary user(s) or group(s). The issue with this approach is that the password is then a shared password. A lot of information security professionals do not like shared passwords because it is insecure.

Having said all that, I think it’s better to use TACACS+, especially with the Cisco-centric environment. While a lot of vendors support TACACS+, there might be some limitations on the authorization piece.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

You might also like to read

How to build tac_plus server
TACACS+ (tac_plus daemon) ACL
How to configure AAA on Cisco router/switches
Enabling AAA on Cisco ASA
Deploying TACACS+ on a Docker container

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

SSH Brute Force Attack

01/01/2016 By Andrew Roderos 2 Comments

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Almost every day, I log in to my Ubuntu Server edition distro and one day I noticed something odd with the disk space. I noticed that it shot up from around 50% to 80% of 6.5GB. I’ve had this server since Hardy Heron days (8.04), it is now on Trusty Thar (14.04.3) Bionic Beaver (18.04.1), and never once it consumed more than 60% of the allocated disk space. That said, I was curious why all of a sudden the disk space became so big when I did not even install new packages.

The df -ah command was not very helpful since it only listed that root directory was consuming ~80%. I needed to figure out which directory was actually consuming those ~30% of disk space. After doing a web search, I found a command that allowed me to discover an important log that I should have known from the beginning.

$ sudo du -a / | sort -n -r | head -n 10
3571449	/
2589064	/var
2065580	/var/log
1762760	/var/log/btmp.1
613604	/usr
304008	/lib
274824	/var/cache
239600	/var/lib

Brute Force Attack Discovery

This was when I discovered the /var/log/auth.log file. Yes, I am still a Linux newbie. Every authentication attempt is listed here and also their IP address. That’s when I saw a bunch of SSH connections from different IP addresses I do not recognize and different usernames that the system does not even have or has been disabled. Most of the username I’ve seen are ubnt (Ubiquiti’s username on a lot of their products), pi, admin, etc. There was one IP address that has been brute forcing my box for a month without me knowing! Below is a snippet of my compressed auth.log file.

$ zcat /var/log/auth.log.4.gz | grep -v CRON | tail -n 500
Dec 29 06:43:17 ubuntu sshd[17503]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=43.229.53.54 user=root
Dec 29 06:43:18 ubuntu sshd[17501]: message repeated 2 times: [ Failed password for root from 43.229.53.54 port 59428 ssh2]
Dec 29 06:43:18 ubuntu sshd[17501]: Received disconnect from 43.229.53.54: 11:  [preauth]
Dec 29 06:43:18 ubuntu sshd[17501]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=43.229.53.54  user=root
Dec 29 06:43:18 ubuntu sshd[17521]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=43.229.53.54  user=root
Dec 29 06:43:18 ubuntu sshd[17523]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=43.229.53.54  user=root
Dec 29 06:43:20 ubuntu sshd[17509]: message repeated 2 times: [ Failed password for root from 43.229.53.54 port 11637 ssh2]
Dec 29 06:43:20 ubuntu sshd[17509]: Received disconnect from 43.229.53.54: 11:  [preauth]
Dec 29 06:43:20 ubuntu sshd[17509]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=43.229.53.54  user=root
Dec 29 06:43:20 ubuntu sshd[17521]: Failed password for root from 43.229.53.54 port 22024 ssh2
Dec 29 06:43:20 ubuntu sshd[17511]: message repeated 2 times: [ Failed password for root from 43.229.53.54 port 11819 ssh2]

Options

There are several options one can implement to mitigate SSH brute force attack. One option is to not allow passwords and just use SSH keys. This is not a good option for me because I want to use this server with any computer and without using any type of keys. That said, password-based authentication is what I need. Another option is to implement two-factor authentication (2FA), which I covered here.

Firewall Option

Initially, I decided to start blocking the IP addresses I’ve seen in auth.log using my PA-200. It worked for a while, but every day I see new IPs popping. I then decided to implement Geo-based IP rule to lower the amount of attack. While it lessens the attacks significantly, I still needed something to help with the attacks that still goes through the firewall.

Enter Fail2ban

This was suggested by a friend of mine, @guerilla7. Thanks, Ron! Fail2ban scans log files (eg. /var/log/auth.log) and bans IPs (using iptables) that show malicious signs – too many password failures, seeking for exploits, etc. By default, Fail2ban monitors the /var/log/auth.log only. Obviously, this can be configured so that it can monitor more log files.

Installation and Configuration

The installation of Fail2ban will vary depending on your distro. Since Ubuntu is Debian based distro, the package manager is apt-get. It is very simple to install in Ubuntu. The command below is how to install the software and its dependencies.

$ sudo apt-get install fail2ban

Once everything is installed, it is time to configure Fail2ban. But, before we edit the configuration file that Fail2ban uses, it is a good idea to use a different file for custom configurations since the original configuration file can be overwritten by updates. That said, we need to create a copy of the configuration file.

$ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Once copied, we are now ready to configure the jail.local file.

$ sudo vi /etc/fail2ban/jail.local

The configuration file is around 480 lines (including comments), but don’t be afraid because only a few lines will be changed. Of course, that depends on one’s needs. For my needs, I only touched four lines and are listed below.

ignoreip = 127.0.0.1/8 192.168.1.0/24
bantime = -1
findtime = 31536000
maxretry = 3

The ignoreip is basically the whitelist.

The bantime sets the length of time that a client will be banned for failed authentication attempts. The negative value sets it forever. This feature was added to version 0.6.1 (03/2006).

The maxretry and findtime parameters work together in establishing the conditions under which a client is determined by an unauthorized user. By default, the findtime is set to 600 seconds (10 minutes) and maxretry to 3 attempts. This means that Fail2ban will ban an unauthorized user when it attempts to log in three times within 10-minute window. Above configuration is 365-day window. This should cover slow brute force attacks but the majority of the brute force attacks I have seen are within 2-second window.

The changes on the configuration file will not take effect until the restart so sudo /etc/init.d/fail2ban restart must be issued.

Optional Configurations

Feel free to skip these configurations if you don’t find them useful.

Persistent Ban

While the above changes are good enough, the bans are not persistent. Once the server or Fail2ban service was restarted, the banned IPs will not survive. Some will be fine with that configuration, but that is unacceptable for me. I want to be the one who will unban IP addresses not because the server or service was restarted.

Update: As of Fail2ban 0.9.x, the bans are now persistent, by default, after reboot or restart. It now maintains a database found in /var/lib/fail2ban/fail2ban.sqlite3 file. That said, for new installs, there is no need to modify the iptables-multiport.conf file. However, if you already have existing ip.blacklist file, then you may want to still modify the iptables-multiport.conf file.

If you want to verify, look at the fail2ban.conf file and look for the dbfile section. You’ll also see the dbpurgeage section that has a 1-day setting. I think this setting is ignored so long as the bantime is set to any negative number.

$ cat /etc/fail2ban/fail2ban.conf
<-- Output omitted for brevity -->
# Options: dbfile
# Notes.: Set the file for the fail2ban persistent data to be stored.
#         A value of ":memory:" means database is only stored in memory
#         and data is lost when fail2ban is stopped.
#         A value of "None" disables the database.
# Values: [ None :memory: FILE ] Default: /var/lib/fail2ban/fail2ban.sqlite3
dbfile = /var/lib/fail2ban/fail2ban.sqlite3
# Options: dbpurgeage
# Notes.: Sets age at which bans should be purged from the database
# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 1d

First Step

The first step in making the ban persistent is to create a file where the list of banned IPs will be added. So that when the service gets restarted, for whatever reason, the software will load the file and issue the proper commands to re-ban the IPs. To create the file, issue the sudo touch /etc/fail2ban/ip.blacklist command. This will create a blank file called ip.blacklist. Feel free to call it different than mine but make sure to use the same file name on the configuration on the next step.

Second Step

The second step is to verify that we are actually going to edit the right configuration file. This is done by viewing the /etc/fail2ban/jail.local file and look for the first banaction = iptables-multiport configuration. At this time of writing, Fail2ban 0.8.11-1 on Ubuntu 14.04.3 is using iptables-multiport, which points to the iptables-multiport.conf file. It’s always a good idea to save a backup configuration, so issue the sudo cp /etc/fail2ban/action.d/iptables-multiport.conf /etc/fail2ban/action.d/iptables-multiport.conf.bak command.

Once completed, we can now move on to the final step. Edit the configuration iptables-multiport.conf file. To edit the iptables-multiport configuration file, issue the sudo vi /etc/fail2ban/action.d/iptables-multiport.conf command. The configuration file has around 70 configuration lines including the comments. There are only two configuration sections that we need to edit. These two are the actionstart and actionban section. The configuration will look as below.

actionstart = iptables -N fail2ban-<name>
              iptables -A fail2ban-<name> -j RETURN
              iptables -I <chain> -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
              # This configuration loads the ip.blacklist file every time Fail2ban service is started.
              if [ -f /etc/fail2ban/ip.blacklist ]; then cat /etc/fail2ban/ip.blacklist | grep -e <name>$ | cut -d "," -s -f 1 | while read IP; do iptables -I fail2ban-<name> 1 -s $IP -j DROP; done; fi
actionban = if ! iptables -C fail2ban-<name> -s <ip> -j DROP; then iptables -I fail2ban-<name> 1 -s <ip> -j DROP; fi
            # Add offenders to ip.blacklist file, if it is not already there yet.
            if ! grep -Fxq '<ip>,<name>' /etc/fail2ban/ip.blacklist; then echo '<ip>,<name>' >> /etc/fail2ban/ip.blacklist; fi

Update: Jim commented that the Fail2ban 0.9.3 on Ubuntu 16.04 changed from fail2ban to f2b. Thanks for the comment Jim!

actionstart = iptables -N f2b-<name>
              iptables -A f2b-<name> -j RETURN
              iptables -I <chain> -p <protocol> -m multiport --dports <port> -j f2b-<name>
              # This configuration loads the ip.blacklist file every time Fail2ban service is started.
              if [ -f /etc/fail2ban/ip.blacklist ]; then cat /etc/fail2ban/ip.blacklist | grep -e <name>$ | cut -d "," -s -f 1 | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done; fi
actionban = if ! iptables -C f2b-<name> -s <ip> -j DROP; then iptables -I f2b-<name> 1 -s <ip> -j DROP; fi
            # Add offenders to ip.blacklist file, if it is not already there yet.
            if ! grep -Fxq '<ip>,<name>' /etc/fail2ban/ip.blacklist; then echo '<ip>,<name>' >> /etc/fail2ban/ip.blacklist; fi

Update: With Fail2ban 0.10.2 on Ubuntu 18.04, the default config was slightly changed. However, the line that we added on previous versions remains the same.

actionstart = <iptables> -N f2b-<name>
              <iptables> -A f2b-<name> -j <returntype>
              <iptables> -I <chain> -p <protocol> -m multiport --dports <port> -j f2b-<name>
              # This configuration loads the ip.blacklist file every time Fail2ban service is started.
              if [ -f /etc/fail2ban/ip.blacklist ]; then cat /etc/fail2ban/ip.blacklist | grep -e <name>$ | cut -d "," -s -f 1 | while read IP; do iptables -I f2b-<name> 1 -s $IP -j DROP; done; fi
actionban = <iptables> -I f2b-<name> 1 -s <ip> -j <blocktype>
            # Add offenders to ip.blacklist file, if it is not already there yet.
            if ! grep -Fxq '<ip>,<name>' /etc/fail2ban/ip.blacklist; then echo '<ip>,<name>' >> /etc/fail2ban/ip.blacklist; fi

Since we’ve made a modification to a configuration file, we need to restart the service by issuing sudo /etc/init.d/fail2ban restart or sudo service fail2ban restart command.

Note: I haven’t figured out why the IPs in the ip.blacklist file do not load after reboot or service restart. It will only add the list once a new failed SSH attempt has been made.

DROP vs REJECT

At the time this post was written, fail2ban used DROP as the default block type. Now, they changed the behavior to REJECT with an ICMP message of unreachable.

The biggest difference between the two is that DROP won’t send anything, while REJECT will send a message back to the source.

If you want to change the block type to DROP, then edit the /etc/fail2ban/action.d/iptables-common.conf file. The configuration below shows that I commented out the default behavior and changed it to DROP instead.

$ sudo more /etc/fail2ban/action.d/iptables-common.conf | grep "blocktype = "
# blocktype = REJECT --reject-with icmp-port-unreachable
blocktype = DROP
# blocktype = REJECT --reject-with icmp6-port-unreachable
blocktype = DROP

Verification

Depending on how often the attack occurs, check the iptables after several hours or a day. To check the iptables, issue the command below.

$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
fail2ban-ssh  tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
Chain fail2ban-ssh (1 references)
target     prot opt source               destination
DROP       all  --  43.229.53.54         0.0.0.0/0
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

Manually unbanning an IP address

To unban an IP address, issue the command below.

$ sudo fail2ban-client set <jail_name> unbanip <ip_address>
# Example using Fail2ban 0.8.11
$ sudo fail2ban-client set ssh unbanip 43.229.53.54
43.229.53.54
$ sudo fail2ban-client reload
# Example using Fail2ban 0.9.3 and 0.10.2
$ sudo fail2ban-client set sshd unbanip 43.229.53.54
43.229.53.54
$ sudo fail2ban-client reload

While the command above is enough without the optional configuration (discussed above), this command is not the only thing needed with the optional configuration since the ip.blacklist file still contains the IP address that we’re trying to unban. If the server or service was restarted, then the IP address will be banned again. That said, it is necessary to take it out from the ip.blacklist file. To do this, issue the command below.

$ sudo sed --in-place '/<ip>,<name>/d' /etc/fail2ban/ip.blacklist
# Example
$ sudo sed --in-place '/43.229.53.54,ssh/d' /etc/fail2ban/ip.blacklist
$ sudo fail2ban-client reload

When unbanning fails

When someone issues the sudo fail2ban-client reload command then there is a very high chance that the user will encounter an error message similar to the one below.

$ sudo fail2ban-client set ssh unbanip 58.218.211.198
ERROR  NOK: ('IP 58.218.211.198 is not banned',)
IP 58.218.211.198 is not banned
# Example using Fail2ban 0.10.2
$ sudo fail2ban-client set sshd unbanip 58.218.211.198
ERROR  NOK: ('IP 58.218.211.198 is not banned',)
IP 58.218.211.198 is not banned

The IP can be still unbanned by the following:

$ sudo sed --in-place '/58.218.211.198,ssh/d' /etc/fail2ban/ip.blacklist
$ sudo fail2ban-client reload
$ sudo iptables -L -n | grep 58.218.211.198

Let’s say deleting the IP address from the blacklist file and reloading fail2ban didn’t work like what I experienced recently. The IP address that I was trying to unban kept coming back. I had to find another way to unban it using the iptables command. Here’s what I did to unban the IP address.

$ sudo iptables -L -n --line-numbers | grep 58.218.211.198
655    DROP       all  --  58.218.211.198       0.0.0.0/0
$ sudo iptables -D fail2ban-ssh 655
$ sudo iptables -L -n --line-numbers | grep 58.218.211.198
$ sudo fail2ban-client reload
$ sudo iptables -L -n --line-numbers | grep 58.218.211.198
# Example using 0.10.2
$ sudo iptables -L -n --line-numbers | grep 58.218.211.198
655    DROP       all  --  58.218.211.198       0.0.0.0/0
$ sudo iptables -D f2b-sshd 655
$ sudo iptables -L -n --line-numbers | grep 58.218.211.198
$ sudo fail2ban-client reload
$ sudo iptables -L -n --line-numbers | grep 58.218.211.198

Though, this seems to be a very rare occasion since I tried unbanning another IP address using the method in the manually unbanning section and it worked just fine.

Thoughts

This is exactly the software I was looking for. It is automated which means I no longer need to check auth.log and block it on my Ubiquiti EdgeRouter Lite. I did transfer the rules from my PA-200 to my new router/firewall, however. Though, I am still getting used to the creation of firewall rules because it is not as intuitive as creating rules on Cisco ASA or Palo Alto Networks firewall. While this software automatically blocks failed attempts, it does not protect from weak passwords. It is still recommended to use strong passwords.

UPDATE: I no longer use the EdgeRouter to protect my DMZ – I now use pfSense. The PA-200 will eventually be used on some user traffic but will be limited. Mostly, to learn more about how to configure it.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

Want to learn more about Linux System Administration?

UNIX and Linux System Administration Handbook, 4th Edition

References

List based permanent bans with fail2ban
How To Protect SSH with fail2ban on Debian 7
How to unban an IP properly with fail2ban

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Cleaning Up Boot Partition

08/19/2013 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

It seems like every single week I have to update my Ubuntu 12.04 Server edition. I configured it to do automatic updates now. Though, I still have to do it manually most of the time. At one point, my boot partition ran out of space and all updates kept failing. I didn’t know at the time why it was failing so I ignored for a few weeks. When I finally paid attention to the issue. I found out that my boot partition was almost full. Apparently, my Linux had warned me every time I logged in to the server since it was on the message of the day page but I normally do not pay to it so it was ignored.

How Did It Get So Big

Well, apparently, Ubuntu holds on to the old kernel when you do sudo apt-get dist-upgrade so whenever it gets updated, it decides to keep the old ones on the boot partition. I didn’t know it at the time so I was really surprised that the boot partition got to a point that I couldn’t update my Ubuntu box. Below is the /etc/motd output that I get whenever I SSH’d to the box. When I was getting an error message that I didn’t have space, I was really confused because if you look at the usage of section of the MOTD it said 43.2% of 6.62GB. But, what I didn’t know at the time was that value was only for the root partition and not for the boot.

login as: networkjutsu
networkjutsu@192.168.1.222's password:
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-52-generic x86_64)
 * Documentation:  https://help.ubuntu.com/
  System information as of Mon Aug 19 19:33:43 PDT 2013
  System load:  0.68              Processes:           76
  Usage of /:   43.2% of 6.62GB   Users logged in:     1
  Memory usage: 32%               IP address for eth0: 192.168.1.222
  Swap usage:   0%
 The warning will show up here if I am not mistaken talking about /boot partition.
  Graph this data and manage this system at https://landscape.canonical.com/
Last login: Mon Aug 19 19:33:15 2013

Checking Hard Disk Space?

To check hard disk space on Ubuntu Linux, type df -ah on the CLI. Below is the output of my Ubuntu box.

networkjutsu@ubuntu:~$ df -ah
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu-root  6.7G  2.9G  3.5G  46% /
proc                        0     0     0    - /proc
sysfs                       0     0     0    - /sys
none                        0     0     0    - /sys/fs/fuse/connections
none                        0     0     0    - /sys/kernel/debug
none                        0     0     0    - /sys/kernel/security
udev                      79M  4.0K   79M   1% /dev
devpts                      0     0     0    - /dev/pts
tmpfs                     36M  244K   35M   1% /run
none                     5.0M     0  5.0M   0% /run/lock
none                      88M     0   88M   0% /run/shm
/dev/sda1                228M  140M   76M  65% /boot

As you can see above, this particular box has used up 65% of the boot partition. This needs to be cleaned up or I’ll eventually run into the same issue as mentioned earlier.

Boot Partition Contents

At the time, I was curious what are taking up my boot partition so I issued ls -l command and found a similar output as shown below.

networkjutsu@ubuntu:~$ ls -hl /boot
total 136M
-rw-r--r-- 1 root root 773K Apr 10  2012 abi-3.2.0-23-generic
-rw-r--r-- 1 root root 775K Nov 15  2012 abi-3.2.0-34-generic
-rw-r--r-- 1 root root 775K Jan  8  2013 abi-3.2.0-36-generic
-rw-r--r-- 1 root root 775K Jan 24  2013 abi-3.2.0-37-generic
-rw-r--r-- 1 root root 777K Jun 18 11:20 abi-3.2.0-49-generic
-rw-r--r-- 1 root root 777K Jul 26 10:08 abi-3.2.0-52-generic
-rw-r--r-- 1 root root 137K Apr 10  2012 config-3.2.0-23-generic
-rw-r--r-- 1 root root 138K Nov 15  2012 config-3.2.0-34-generic
-rw-r--r-- 1 root root 138K Jan  8  2013 config-3.2.0-36-generic
-rw-r--r-- 1 root root 138K Jan 24  2013 config-3.2.0-37-generic
-rw-r--r-- 1 root root 138K Jun 18 11:20 config-3.2.0-49-generic
-rw-r--r-- 1 root root 138K Jul 26 10:08 config-3.2.0-52-generic
drwxr-xr-x 3 root root 5.0K Aug 19 19:31 grub
-rw-r--r-- 1 root root  15M Dec 15  2012 initrd.img-3.2.0-23-generic
-rw-r--r-- 1 root root  15M Dec 15  2012 initrd.img-3.2.0-34-generic
-rw-r--r-- 1 root root  15M Feb  7  2013 initrd.img-3.2.0-36-generic
-rw-r--r-- 1 root root  15M Feb  7  2013 initrd.img-3.2.0-37-generic
-rw-r--r-- 1 root root  15M Jul 19 20:33 initrd.img-3.2.0-49-generic
-rw-r--r-- 1 root root  15M Aug 19 19:32 initrd.img-3.2.0-52-generic
drwxr-xr-x 2 root root  12K Dec 15  2012 lost+found
-rw-r--r-- 1 root root 173K Nov 27  2011 memtest86+.bin
-rw-r--r-- 1 root root 175K Nov 27  2011 memtest86+_multiboot.bin
-rw------- 1 root root 2.8M Apr 10  2012 System.map-3.2.0-23-generic
-rw------- 1 root root 2.8M Nov 15  2012 System.map-3.2.0-34-generic
-rw------- 1 root root 2.8M Jan  8  2013 System.map-3.2.0-36-generic
-rw------- 1 root root 2.8M Jan 24  2013 System.map-3.2.0-37-generic
-rw------- 1 root root 2.8M Jun 18 11:20 System.map-3.2.0-49-generic
-rw------- 1 root root 2.8M Jul 26 10:08 System.map-3.2.0-52-generic
-rw------- 1 root root 4.8M Apr 10  2012 vmlinuz-3.2.0-23-generic
-rw------- 1 root root 4.8M Nov 15  2012 vmlinuz-3.2.0-34-generic
-rw------- 1 root root 4.8M Jan  8  2013 vmlinuz-3.2.0-36-generic
-rw------- 1 root root 4.8M Jan 24  2013 vmlinuz-3.2.0-37-generic
-rw------- 1 root root 4.8M Jun 18 11:20 vmlinuz-3.2.0-49-generic
-rw------- 1 root root 4.8M Jul 26 10:08 vmlinuz-3.2.0-52-generic

As shown, there are a lot of older kernel version (vmlinuz) still on the boot partition. In theory, rm command should work but it’s not as clean as running the dpkg command.

Cleaning It Up

As already mentioned, the quickest and probably the best way to clean this up is to issue the command shown below. There are a lot of ways to do this but I prefer this command since it’s only one line of command. When I first had to clean up the boot partition, I actually used several commands to clean it up. This one is just one line of command and you’re good to go. I’ve included the whole output just to show a similar screen that you’ll see.

networkjutsu@ubuntu:~$ dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge 

[sudo]
password for networkjutsu:
Reading package lists…
Done
Building dependency tree Reading state information…
Done
The following packages will be REMOVED:
linux-headers-3.2.0-23*
linux-headers-3.2.0-23-generic*
linux-headers-3.2.0-34*
linux-headers-3.2.0-34-generic*
linux-headers-3.2.0-36*
linux-headers-3.2.0-36-generic*
linux-headers-3.2.0-37*
linux-headers-3.2.0-37-generic*
linux-headers-3.2.0-49*
linux-headers-3.2.0-49-generic*
linux-image-3.2.0-23-generic*
linux-image-3.2.0-34-generic*
linux-image-3.2.0-36-generic*
linux-image-3.2.0-37-generic*
linux-image-3.2.0-49-generic*
0 upgraded, 0 newly installed, 15 to remove and 0 not upgraded. After this operation, 1,084 MB disk space will be freed. (Reading database … 180798 files and directories currently installed.)
Removing linux-headers-3.2.0-23-generic …
Removing linux-headers-3.2.0-23 …
Removing linux-headers-3.2.0-34-generic …
Removing linux-headers-3.2.0-34 …
Removing linux-headers-3.2.0-36-generic …
Removing linux-headers-3.2.0-36 …
Removing linux-headers-3.2.0-37-generic …
Removing linux-headers-3.2.0-37 …
Removing linux-headers-3.2.0-49-generic …
Removing linux-headers-3.2.0-49 …
Removing linux-image-3.2.0-23-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-23-generic /boot/vmlinuz-3.2.0-23-generic update-initramfs:
Deleting /boot/initrd.img-3.2.0-23-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-23-generic /boot/vmlinuz-3.2.0-23-generic Generating grub.cfg …
Found linux image: /boot/vmlinuz-3.2.0-52-generic
Found initrd image: /boot/initrd.img-3.2.0-52-generic
Found linux image: /boot/vmlinuz-3.2.0-49-generic
Found initrd image: /boot/initrd.img-3.2.0-49-generic
Found linux image: /boot/vmlinuz-3.2.0-37-generic
Found initrd image: /boot/initrd.img-3.2.0-37-generic
Found linux image: /boot/vmlinuz-3.2.0-36-generic
Found initrd image: /boot/initrd.img-3.2.0-36-generic
Found linux image: /boot/vmlinuz-3.2.0-34-generic
Found initrd image: /boot/initrd.img-3.2.0-34-generic
Found memtest86+ image: /memtest86+.bin done
Purging configuration files for linux-image-3.2.0-23-generic …
Examining /etc/kernel/postrm.d . run-parts:
executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-23-generic /boot/vmlinuz-3.2.0-23-generic run-parts:
executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-23-generic /boot/vmlinuz-3.2.0-23-generic
Removing linux-image-3.2.0-34-generic …
Examining /etc/kernel/postrm.d . run-parts:
executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-34-generic /boot/vmlinuz-3.2.0-34-generic update-initramfs:
Deleting /boot/initrd.img-3.2.0-34-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-34-generic /boot/vmlinuz-3.2.0-34-generic
Generating grub.cfg …
Found linux image: /boot/vmlinuz-3.2.0-52-generic
Found initrd image: /boot/initrd.img-3.2.0-52-generic
Found linux image: /boot/vmlinuz-3.2.0-49-generic
Found initrd image: /boot/initrd.img-3.2.0-49-generic
Found linux image: /boot/vmlinuz-3.2.0-37-generic
Found initrd image: /boot/initrd.img-3.2.0-37-generic
Found linux image: /boot/vmlinuz-3.2.0-36-generic
Found initrd image: /boot/initrd.img-3.2.0-36-generic
Found memtest86+ image: /memtest86+.bin done
Purging configuration files for linux-image-3.2.0-34-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-34-generic /boot/vmlinuz-3.2.0-34-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-34-generic /boot/vmlinuz-3.2.0-34-generic
Removing linux-image-3.2.0-36-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-36-generic /boot/vmlinuz-3.2.0-36-generic update-initramfs:
Deleting /boot/initrd.img-3.2.0-36-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-36-generic /boot/vmlinuz-3.2.0-36-generic
Generating grub.cfg …
Found linux image: /boot/vmlinuz-3.2.0-52-generic
Found initrd image: /boot/initrd.img-3.2.0-52-generic
Found linux image: /boot/vmlinuz-3.2.0-49-generic
Found initrd image: /boot/initrd.img-3.2.0-49-generic
Found linux image: /boot/vmlinuz-3.2.0-37-generic
Found initrd image: /boot/initrd.img-3.2.0-37-generic
Found memtest86+ image: /memtest86+.bin done
Purging configuration files for linux-image-3.2.0-36-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-36-generic /boot/vmlinuz-3.2.0-36-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-36-generic /boot/vmlinuz-3.2.0-36-generic
Removing linux-image-3.2.0-37-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-37-generic /boot/vmlinuz-3.2.0-37-generic update-initramfs:
Deleting /boot/initrd.img-3.2.0-37-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-37-generic /boot/vmlinuz-3.2.0-37-generic Generating grub.cfg …
Found linux image: /boot/vmlinuz-3.2.0-52-generic Found initrd image: /boot/initrd.img-3.2.0-52-generic
Found linux image: /boot/vmlinuz-3.2.0-49-generic Found initrd image: /boot/initrd.img-3.2.0-49-generic
Found memtest86+ image: /memtest86+.bin done Purging configuration files for linux-image-3.2.0-37-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-37-generic /boot/vmlinuz-3.2.0-37-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-37-generic /boot/vmlinuz-3.2.0-37-generic Removing linux-image-3.2.0-49-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-49-generic /boot/vmlinuz-3.2.0-49-generic update-initramfs:
Deleting /boot/initrd.img-3.2.0-49-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-49-generic /boot/vmlinuz-3.2.0-49-generic
Generating grub.cfg …
Found linux image: /boot/vmlinuz-3.2.0-52-generic
Found initrd image: /boot/initrd.img-3.2.0-52-generic
Found memtest86+ image: /memtest86+.bin done
The link /vmlinuz.old is a damaged link
Removing symbolic link vmlinuz.old you may need to re-run your boot loader[grub] The link /initrd.img.old is a damaged link
Removing symbolic link initrd.img.old you may need to re-run your boot loader[grub]
Purging configuration files for linux-image-3.2.0-49-generic …
Examining /etc/kernel/postrm.d . run-parts: executing /etc/kernel/postrm.d/initramfs-tools 3.2.0-49-generic /boot/vmlinuz-3.2.0-49-generic run-parts: executing /etc/kernel/postrm.d/zz-update-grub 3.2.0-49-generic /boot/vmlinuz-3.2.0-49-generic

Verification

To verify that the command worked, issue both commands. Though, the first command will be fine.

networkjutsu@ubuntu:~$ df -ah
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu-root  6.7G  1.7G  4.7G  27% /
proc                        0     0     0    - /proc
sysfs                       0     0     0    - /sys
none                        0     0     0    - /sys/fs/fuse/connections
none                        0     0     0    - /sys/kernel/debug
none                        0     0     0    - /sys/kernel/security
udev                      79M   12K   79M   1% /dev
devpts                      0     0     0    - /dev/pts
tmpfs                     36M  244K   35M   1% /run
none                     5.0M     0  5.0M   0% /run/lock
none                      88M     0   88M   0% /run/shm
/dev/sda1                228M   27M  189M  13% /boot
networkjutsu@ubuntu:~$ ls -al /boot
total 23544
drwxr-xr-x  4 root root     2048 Aug 19 20:18 .
drwxr-xr-x 23 root root     4096 Aug 19 20:18 ..
-rw-r--r--  1 root root   795365 Jul 26 10:08 abi-3.2.0-52-generic
-rw-r--r--  1 root root   140629 Jul 26 10:08 config-3.2.0-52-generic
drwxr-xr-x  3 root root     5120 Aug 19 20:18 grub
-rw-r--r--  1 root root 14819736 Aug 19 19:32 initrd.img-3.2.0-52-generic
drwxr-xr-x  2 root root    12288 Dec 15  2012 lost+found
-rw-r--r--  1 root root   176764 Nov 27  2011 memtest86+.bin
-rw-r--r--  1 root root   178944 Nov 27  2011 memtest86+_multiboot.bin
-rw-------  1 root root  2893555 Jul 26 10:08 System.map-3.2.0-52-generic
-rw-------  1 root root  4978224 Jul 26 10:08 vmlinuz-3.2.0-52-generic

Thoughts

I am actually surprised that Ubuntu chose to keep the older kernel when doing an update. I wish in the future release that they will include the uninstall of the older kernel so I won’t have to do this every time I am running out of boot partition space. Yes, I can redo the partition but it will eventually fill up anyway so it’s a wasted effort in my opinion.

Reference

Ubuntu Forum

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Building Your Own Proxy Server

02/18/2013 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

What is a proxy server? According to Wiki page, it is a server that acts as an intermediary for requests from clients seeking resources from other servers. There are many types of proxy servers out there and in this tutorial it will only show one type of it which allows someone to bypass most, if not all, of the poorly designed and/or implemented web content filtering services.

There are many ways to filter websites to avoid employees going to adult, gambling, social networking sites, and etc. The ones that I know of are the following: OpenDNS, Blue Coat, and Websense. While I haven’t been on a network were OpenDNS has been implemented, I’ve been on networks with Blue Coat and Websense appliances. With this tutorial, someone will most likely be successful in bypassing these web content filtering appliances if they are not properly configured or if a firewall is allowing outbound SSH traffic. In organizations where they allow any websites to be accessed, this tutorial can also hide HTTP/HTTPS traffic by encapsulating all of user’s traffic inside SSH, which is encrypted.

What you need

  • Extra computer or a new Virtual Machine (VM)
  • Know how to install and update Linux with OpenSSH installed. This tutorial will use Ubuntu Server edition.
  • Know how to use text editor on Linux. This tutorial will use VI editor.
  • Know how to configure port forwarding on your router
  • Account from DynDNS or other free Dynamic DNS
  • PuTTY (Windows) or Terminal
  • Firefox installed

Steps

Step one is to download, install and update Ubuntu Linux Server. In this tutorial, I used Ubuntu Server 12.04 LTS. My original installation still works with 16.04.

Step two is to install Squid package. To install the package, please follow the command below. If you want to learn more about Squid, then you might want to check out this book. I have no experience with the book so please read through the reviews.

networkjutsu@ubuntu:~$ sudo apt-get install squid
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  linux-headers-3.2.0-23-generic linux-headers-3.2.0-23 linux-headers-3.2.0-34
  linux-headers-3.2.0-36 linux-headers-3.2.0-34-generic
  linux-headers-3.2.0-36-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  libltdl7 squid-langpack squid3 squid3-common
Suggested packages:
  squidclient squid-cgi smbclient
The following NEW packages will be installed:
  libltdl7 squid squid-langpack squid3 squid3-common
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,012 kB of archives.
After this operation, 7,122 kB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise/main libltdl7 amd64 2.4.2-1ubuntu1 [37.6 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ precise/main squid-langpack all 20111114-1 [307 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main squid3-common all 3.1.19-1ubuntu3.12.04.2 [123 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main squid3 amd64 3.1.19-1ubuntu3.12.04.2 [1,539 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe squid amd64 3.1.19-1ubuntu3.12.04.2 [6,254 B]
Fetched 2,012 kB in 2s (753 kB/s)
Selecting previously unselected package libltdl7.
(Reading database ... 128431 files and directories currently installed.)
Unpacking libltdl7 (from .../libltdl7_2.4.2-1ubuntu1_amd64.deb) ...
Selecting previously unselected package squid-langpack.
Unpacking squid-langpack (from .../squid-langpack_20111114-1_all.deb) ...
Selecting previously unselected package squid3-common.
Unpacking squid3-common (from .../squid3-common_3.1.19-1ubuntu3.12.04.2_all.deb) ...
Selecting previously unselected package squid3.
Unpacking squid3 (from .../squid3_3.1.19-1ubuntu3.12.04.2_amd64.deb) ...
Selecting previously unselected package squid.
Unpacking squid (from .../squid_3.1.19-1ubuntu3.12.04.2_amd64.deb) ...
Processing triggers for man-db ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Processing triggers for ufw ...
Setting up libltdl7 (2.4.2-1ubuntu1) ...
Setting up squid-langpack (20111114-1) ...
Setting up squid3-common (3.1.19-1ubuntu3.12.04.2) ...
Setting up squid3 (3.1.19-1ubuntu3.12.04.2) ...
Creating Squid HTTP proxy 3.x spool directory structure
2013/02/18 11:32:01| Creating Swap Directories
squid3 start/running, process 1894
Setting up squid (3.1.19-1ubuntu3.12.04.2) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place

Step three is to configure the Ubuntu box to use static IP address or do address reservation on the home router. Since there are a lot of home routers available on the market, this tutorial will only cover assigning a static IP on Ubuntu box, which is shown below.

networkjutsu@ubuntu:~$ sudo vi /etc/network/interfaces

A new screen will show and will look like the one shown below.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp

Change the configuration to something like this:

# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.1.10
        netmask 255.255.255.0
        network 192.168.1.0
        broadcast 192.168.1.255
        gateway 192.168.1.1
        dns-nameservers 192.168.1.1

Make sure to save and exit out of the text editor. Once saved and back to Linux prompt, make sure to restart the networking services to take effect of the new IP information.

networkjutsu@ubuntu:~$ sudo /etc/init.d/networking restart
 * Running /etc/init.d/networking restart is deprecated because it may not enable again some interfaces
 * Reconfiguring network interfaces...

Since this was done remotely, the SSH session was terminated once the changes took effect.

Step four is to configure port forwarding on the home router. Again, since there are many home routers available on the market, it is difficult to compile each and every model out there to include in this tutorial.

Step five is to sign up to a Dynamic DNS service like DynDNS. This is useful for folks that are using dynamic IP address for home Internet connection. How to configure the router with DynDNS service is out of the scope of this tutorial.

Step six is to verify that the Ubuntu box is reachable via another computer on a different ISP. This is the perfect time to use the office network to connect to the Ubuntu box. If the SSH session was successful, then that means the firewall is not blocking TCP port 22 (SSH) connection outbound. If the SSH wasn’t successful, this can mean two things: port forwarding wasn’t configured correctly or the firewall is blocking SSH. Try another computer on a different ISP with no firewall blocking SSH sessions.

Step seven is to change or add SSH port to port 80 (HTTP) or 443 (HTTPS). This step can be skipped if step six was successful using the default port of SSH. To change or add port what SSH service is listening on please follow the commands below.

networkjutsu@ubuntu:~$ sudo vi /etc/ssh/sshd_config

A new screen will show up and will look like this:

# Package generated configuration file
# See the sshd_config(5) manpage for details
# What ports, IPs and protocols we listen for
Port 22

Change the configuration like the one shown below. Delete Port 22 line if desired. I’ve added port 80 and 443 in this configuration but either one is fine. Save and exit the text editor and reboot the server. I tried doing sudo service ssh restart or sudo /etc/init.d/ssh restart but didn’t work. Please feel free to let me know another way to do this other than rebooting the server.

# Package generated configuration file
# See the sshd_config(5) manpage for details
# What ports, IPs and protocols we listen for
Port 22
Port 80
Port 443

Step eight is to reconfigure the port forwarding reflecting the new port that was configured earlier.

Step nine is to try connecting to the Ubuntu box from the office network using port 80 or 443, depends on how it was configured on Ubuntu box. This should be successful unless the deep packet inspection on Blue Coat or Websense has been turned on. Another one that will prevent this connection to be successful is when the organization has Palo Alto Networks firewall installed or any appliance that is capable of doing application layer filtering. That being said, this tutorial is useless for this type of network.

Assuming that step nine is successful, then step ten is to create a new SSH session but this time with different parameters. If Windows and PuTTY is being used, please follow the guide below. Please go to page four if Apple OS X is being used.

Windows and PuTTY

Enter the host name or IP address of the Ubuntu box and the port number as normal.

On the left side of the PuTTY (category section), expand SSH menu which is under Connection menu and click Tunnels option. A screenshot has been included below for your reference.

Tunnels

Once there, we need to add forwarded ports both local and remote. Screenshots has been included below for your reference.

Local ports

Local

Remote ports

Remote

Since this SSH session will be used a lot, it is a good idea to save it so all the parameters will be populated when we use it again. Please see below for a screenshot.

Saved Session

Apple OS X and Terminal

With Apple OS X and Terminal, it is really simple compared to Windows and PuTTY combination. Please see below on how to connect using Terminal on OS X.

ssh networkjutsu@networkjutsu.com -L 3128:localhost:3128

Since this is a long command, we should create an alias to shorten the command we need to issue when connecting to the Ubuntu box. To create an alias, we need to edit the .bash_profile which is on our home directory.

NetworkJutsu-MacBook-Pro:~ NetworkJutsu$ ls -ah
.                       .bash_profile           Documents
..                      .cups                   Downloads
.CFUserTextEncoding     .filezilla              Library
.DS_Store               .putty                  Movies
.Trash                  .ssh                    Music
.anyconnect             .viminfo                Pictures
.bash_history           Desktop                 Public
NetworkJutsu-MacBook-Pro:~ NetworkJutsu$ sudo vi .bash_profile

A new window will show up that will look like the one below.

~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
".bash_profile" 2L, 120C

Enter the line shown below then save and exit out of the editor.

alias networkjutsu='ssh networkjutsu@networkjutsu.com -L 3128:localhost:3128'

By creating an alias, we save keystrokes every time we connect to our Ubuntu box as shown below. The command above is assuming that we’re using the standard SSH port which is 22. If we’re using port 80 or 443, we need to add -p 80 (or 443) in the command.

NetworkJutsu-MacBook-Pro:~ NetworkJutsu$ networkjutsu
networkjutsu@networkjutsu.com's password:
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-37-generic x86_64)
 * Documentation:  https://help.ubuntu.com/
  System information as of Mon Feb 18 13:44:53 PST 2013
  System load:  0.0               Processes:           74
  Usage of /:   47.1% of 6.62GB   Users logged in:     0
  Memory usage: 25%               IP address for eth0: 192.168.1.10
  Swap usage:   32%
  => /boot is using 91.2% of 227MB
  Graph this data and manage this system at https://landscape.canonical.com/
0 packages can be updated.
0 updates are security updates.
Last login: Mon Feb 18 11:58:59 2013 from networkjutsu.com

Browser settings

Most organizations lock down their Windows machines so most likely the Internet Explorer’s Internet Options will be grayed out or parts of it will be. Having said that, we need another browser that will support changing out settings that is not tied with Windows GPO. Google Chrome does not do the job since it is tied to Internet Explorer’s Internet Options. Another popular browser is Firefox which does the job that we need it to do. While I know there are other browsers out there, this tutorial was only done on Firefox. Please feel free to test other browser of your choice. See the picture below for the configuration details to use our Ubuntu proxy server.

Firefox

With all steps are successfully performed, we can now start browsing to virtually all websites that we want to access from anywhere. While the tutorial mentioned only bypassing web content filtering appliances, this can also serve as our protection when we are in an open-authentication wireless network like the ones that you connect to local coffee shop, restaurants, and etc. With open-authentication, the wireless traffic are not encrypted so someone who knows what they’re doing can definitely decrypt your SSL sessions when connecting to your bank and/or email. To prevent that from happening, we can use this same method to tunnel all our HTTP and/or HTTPS traffic to our Ubuntu box. It won’t be as fast but at least our traffic is secure.

Thoughts

Tutorial such as this can easily be found on the web so I encourage network engineers who are also responsible for the network security or network security engineers out there to really find ways to mitigate ways to bypass the security in place. As more and more savvy users entering the workforce, it is becoming harder to really prevent users accessing what they shouldn’t be able to access while on organization’s network. As newer products or technologies gets introduced, always be open minded what these can do for the organization. Fortinet, Check Point, Palo Alto Networks, Cisco (with their ASA-CX), and etc they can definitely help with this type of situation. Yes, I understand that money can be tight since our economy hasn’t recovered fully yet but there are other ways to at least mitigate this type of bypass as briefly mentioned in the tutorial – deep packet inspection of the HTTP/HTTPS traffic on Blue Coat or Websense appliance and outbound firewall rules.

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email
  • Go to page 1
  • Go to page 2
  • Go to Next Page »

Footer

WORK WITH US

Schedule a free consultation now!

LET’S TALK

Copyright © 2011–2023 · NetworkJutsu · All Rights Reserved · Privacy Policy · Terms of Use