• Skip to main content
  • Skip to footer

NetworkJutsu

Networking & Security Services | San Francisco Bay Area

  • Blog
  • Services
  • Testimonials
  • About
    • About Us
    • Terms of Use
    • Privacy Policy
  • Contact Us

IOS

Securing Cisco IOS SSH server

12/16/2017 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Back in 2011, I wrote a post on how to enable SSH on Cisco routers and switches. Unfortunately, it didn’t contain any of the advanced configurations that will harden Cisco IOS SSH server. To be fair, there were older IOS software versions that didn’t include advanced SSH commands that I will cover here. With this post, I’d like to share at least the minimum advanced SSH configuration that network engineers should consider adding to their template.

SSH Encryption Algorithms

If you’re a macOS 10.13.2 user and you use it to connect to Cisco routers and switches, you may have seen this error message already.

Mac mini:~ networkjutsu$ ssh router01
Unable to negotiate with 192.168.100.200 port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

The issue here is that OpenSSH has deprecated the weaker ciphers in the default SSH configuration of the newest version of macOS. Unfortunately, older Cisco IOS software uses AES 3DES-CBC for the SSH server, by default. Below is an example of a Cisco router running an older version of IOS which uses default SSH configuration.

router01>sh ssh
Connection Version Mode Encryption  Hmac	     State	               Username
0          2.0     IN   3des-cbc    hmac-sha1    Session started       networkjutsu
0          2.0     OUT  3des-cbc    hmac-sha1    Session started       networkjutsu
%No SSHv1 server connections running.

There are two options to get rid of the error message. One of the options is by configuring the client side to accept the legacy ciphers. The right course of action, in my opinion, is to change SSH server configurations. However, we still need to be able to connect to our Cisco IOS devices to correct the issue.

SSH client options

A quick fix here is to keep using compatible ciphers that the client would accept. There are three options that one could use for this workaround. Technically, they are all doing the same thing but just different approach.

Option #1

With this option, the user just needs to specify the cipher and KEX algorithms in the SSH command when connecting to an SSH server. One could create an alias to include all the necessary command flags for shorter keystrokes.

Mac mini:~ networkjutsu$ ssh -oKexAlgorithms=diffie-hellman-group1-sha1 -c 3des-cbc router01
Password:
router01>

Option #2

With this option, the user does not need to create an alias or type the whole command shown above. The .ssh/config file is a user-specific configuration file. OpenSSH receives its configuration from this file when the command issued doesn’t include command flags.

Mac mini:~ networkjutsu$ cat .ssh/config
# ***
# *** General settings (these apply to all connections)
# ***
HostkeyAlgorithms +ssh-dss
KexAlgorithms +diffie-hellman-group1-sha1
Ciphers +3des-cbc

Option #3

With this option, all users are affected by this configuration file. However, the command issued and user-specific configuration file take precedence over the global configuration file.

Mac mini:~ networkjutsu$ cat /etc/ssh/ssh_config
HostkeyAlgorithms +ssh-dss
KexAlgorithms +diffie-hellman-group1-sha1
Ciphers +3des-cbc

SSH server options

As mentioned earlier, the server side option is the correct course of action. However, one still needs to connect the Cisco IOS devices to fix the issue. That said, the SSH client workaround still plays an important role.

SSH encryption algorithm

The command shown below is used to change SSH encryption key algorithm used on a Cisco IOS device. If one gets an error message, then the command is not available in that IOS version.

router01(config)#ip ssh server algorithm encryption ?
  3des-cbc    Three-key 3DES in CBC mode
  aes128-cbc  AES with 128-bit key in CBC mode
  aes128-ctr  AES with 128-bit key in CTR mode
  aes192-cbc  AES with 192-bit key in CBC mode
  aes192-ctr  AES with 192-bit key in CTR mode
  aes256-cbc  AES with 256-bit key in CBC mode
  aes256-ctr  AES with 256-bit key in CTR mode
router01(config)#ip ssh server algorithm encryption aes256-ctr

In this particular IOS version, the SSH server supports the encryption algorithms: AES-CTR, AES-CBC, and 3DES. According to this thread, use EAX or GCM, if available. If not, the author said to use CTR over CBC mode. By specifying the encryption algorithm, we’re telling Cisco IOS to only offer the AES-256-CTR mode to any clients that try to connect to it.

Below shows the verbose output of a Cisco IOS device using default SSH configuration.

Mac mini:~ networkjutsu$ ssh -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug2: peer server KEXINIT proposal
debug2: ciphers ctos: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

Below shows the verbose output of a Cisco IOS device using the SSH configuration mentioned above.

Mac-mini:~ networkjutsu$ ss -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug2: peer server KEXINIT proposal
debug2: ciphers ctos: aes256-ctr
debug2: ciphers stoc: aes256-ctr

SSH MAC algorithm

To change the default SSH MAC algorithm used on a Cisco IOS device, use the command below.

router01(config)#ip ssh server algorithm mac ?
  hmac-sha1     HMAC-SHA1 (digest length = key length = 160 bits)
  hmac-sha1-96  HMAC-SHA1-96 (digest length = 96 bits, key length = 160 bits)
router01(config)#ip ssh server algorithm mac hmac-sha1

UPDATE: Newer IOS supports higher than SHA1.

router01(config)#ip ssh server algorithm mac ?
  hmac-sha1      HMAC-SHA1 (digest length = key length = 160 bits)
  hmac-sha1-96   HMAC-SHA1-96 (digest length = 96 bits, key length = 160 bits)
  hmac-sha2-256  HMAC-SHA2-256 (digest length = 256 bits, key length = 256 bits)
  hmac-sha2-512  HMAC-SHA2-512 (digest length = 512 bits, key length = 512 bits)
router01(config)#ip ssh server algorithm mac hmac-sha2-512

In this particular IOS version, the SSH server supports two Message Authentication Code (MAC) algorithms: HMAC-SHA1 and HMAC-SHA1-96. The difference between the two algorithms is the digest length. The HMAC-SHA1-96 is a truncated message digest. From my limited understanding, the HMAC-SHA1-96 is the weakened version of HMAC-SHA1 due to the shortened message digest.

Below shows the verbose output of a Cisco IOS device using default SSH configuration.

Mac-mini:~ networkjutsu$ ss -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug2: MACs ctos: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
debug2: MACs stoc: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96

Below shows the verbose output of a Cisco IOS device using the SSH configuration mentioned above.

Mac-mini:~ networkjutsu$ ss -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug2: peer server KEXINIT proposal
debug2: MACs ctos: hmac-sha1
debug2: MACs stoc: hmac-sha1

UPDATE: Configured with SHA2

Mac-mini:~ networkjutsu$ ss -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug2: MACs ctos: hmac-sha2-512
debug2: MACs stoc: hmac-sha2-512

Key Exchange Algorithm

If my memory serves me right, even before macOS High Sierra, OpenSSH also deprecated the use of Diffie-Hellman key exchange with SHA-1. That said, users that tried to connect to Cisco IOS devices with default SSH configurations were greeted by an error message, like the one below.

Mac mini:~ networkjutsu$ ssh router01
Unable to negotiate with 192.168.100.200 port 22: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1

The real issue is that most of the Cisco IOS versions use 1024-bit key size for Diffie-Hellman used for key exchange, by default. Though, there are old Cisco IOS versions that use 768-bit DH key size, by default. Prior the year of 2016, 1024-bit key size is adequate. However, NIST’s recommendation is to use 2048-bit key size or higher. Furthermore, the authors of the LogJam paper believes that it may be possible for a nation-state to break 1024-bit groups. Therefore, the authors recommend disabling DH Group 1.

router01(config)#sh ip ssh
<-- Output omitted for brevity -->
Minimum expected Diffie Hellman key size : 1024 bits
router01(config)#ip ssh dh min size ?
  1024  Diffie Group 1 1024-bit key
  2048  Diffie Group 14 2048-bit key
  4096  Diffie Group 16 4096-bit key
router01(config)#ip ssh dh min size 4096

Below shows the verbose output of a Cisco IOS device using the SSH configuration mentioned above.

Mac-mini:~ networkjutsu$ ss -vvv router01
OpenSSH_7.6p1, LibreSSL 2.6.2
<-- Output omitted for brevity -->
debug1: kex: algorithm: diffie-hellman-group-exchange-sha1
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes256-ctr MAC: hmac-sha1 compression: none
debug1: kex: client->server cipher: aes256-ctr MAC: hmac-sha1 compression: none

Note: Changing the DH key size to 4096 value may break some applications that connect to Cisco IOS devices. For example, HPE Opsware Network Automation (now Micro Focus) uses a Java-based SSH client that is incompatible with SSH servers that use higher than 2048-bit DH key.

Additional SSH configuration

The commands covered here deserves consideration since they increase the level of protection to Cisco IOS SSH server.

RSA keys

As covered in this post, I used 4096-bit modulus in the second example. Cisco IOS users should consider generating higher than NIST’s recommendation of the 2048-bit modulus. Generating higher than the recommended value may take a minute or two (depending on the platform). Additionally, it may take few seconds to get the prompt when connecting to a Cisco IOS device. That said, make sure to take the two facts into consideration before using higher than the recommended value. In theory, newer Cisco platforms could handle the higher values without a significant impact on performance.

router01(config)#crypto key gen rsa mod ?
  <360-4096>  size of the key modulus [360-4096]
router01(config)#crypto key gen rsa modulus 4096 label SSH_KEY
The name for the keys will be: SSH_KEY
% The key modulus size is 4096 bits
% Generating 4096 bit RSA keys, keys will be non-exportable...
[OK] (elapsed time was 103 seconds)

If you’re confused about the difference between RSA and DH mentioned here, then I recommend you to read this article. The article did a great job explaining the SSH connection process. If you just want to know the difference between the RSA and DH, then skip to the Negotiating Encryption for the Session section.

SSH authentication timeout

There is no reason to have a high authentication timeout, so it is recommended to lower the value to 60 seconds or less. This particular router has the SSH authentication timeout set to 120 seconds. We’ll change it to 30 seconds.

router01#sh ip ssh
SSH Enabled - version 2.0
Authentication methods:publickey,keyboard-interactive,password
Authentication timeout: 120 secs; Authentication retries: 3
<-- Output omitted for brevity -->
router01#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
router01(config)#ip ssh time-out 30
router01(config)#do sh ip ssh
SSH Enabled - version 2.0
Authentication methods:publickey,keyboard-interactive,password
Authentication timeout: 30 secs; Authentication retries: 3
<-- Output omitted for brevity -->

Line VTY

There four Cisco IOS features under VTY configuration that deserves consideration because they provide an increased level of protection to networking devices.

SSH transport protocols

As mentioned in this post, by default, Cisco IOS still allows telnet connection when the user doesn’t disable it. To disable, please issue the command below. If you only need 5 vty lines, I suggest disabling the remaining vty lines.

router01(config)#line vty 0 4
router01(config-line)#transport input ssh
router01(config)#line vty ?
  <0-98>  First Line number
router01(config)#line vty 5 98
router01(config-line)#transport input none

SSH ACL

Creating and applying ACL to SSH is best practice, so I decided to cover it here, even though this is considered very basic security.

router01(config)#access-list 1 permit 172.16.0.0 0.0.0.63
router01(config)#access-list 1 permit 192.168.100.0 0.0.0.255
router01(config)#line vty 0 4
router01(config-line)#access-class 1 in

Session timeout

I think this is one of the controversial settings that require some discussions with the networking team. The STIG recommends to set it to 10 minutes or less. By default, Cisco IOS uses 10 minutes for this setting. Please feel free to change it to something else that follows your security policy or suggested setting by the networking team.

router01#sh run all | sec line vty
line vty 0 4
 motd-banner
 exec-banner
 exec-timeout 10 0
<-- Output omitted for brevity -->
router01(config)#line vty 0 4
router01(config-line)#exec-timeout ?
  <0-35791>  Timeout in minutes
router01(config-line)#exec-timeout 5 ?
  <0-2147483>  Timeout in seconds
router01(config-line)#exec-timeout 5 0

Final Words

All of the configurations covered here are what I’d say minimum security standard for all Cisco IOS devices. My advice for my fellow network engineers looking to secure network devices against management plane attacks must consider including this in their configuration template. Though, this blog post is just a small part of protecting the management plane. That said, I urge my fellow network engineers to research more about other settings that protect the management plane.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

NetworkJutsu provides networking and network security consulting services for startups, a more established small and medium-sized business (SMB), or large business throughout the San Francisco Bay Area.

Want to learn more about securing Cisco IOS?

CCNP Security Secure 642-637 Official Cert Guide

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Adding Two-Factor Authentication to TACACS+

06/02/2017 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Several months ago, I covered how to add two-factor authentication (2FA) to FreeRADIUS using Google Authenticator. Today, I will cover the TACACS+ version of it.

Related: What is multi-factor authentication (MFA)?

I’ve written a blog post on how to build tac_plus server using Ubuntu. The guide was written in 2011, while it’s an old blog post, the instructions are still valid using Ubuntu Server 16.04. Please use that guide on how to build one, then use this guide to add multi-factor authentication (MFA) to TACACS+.

Related: Deploying TACACS+ on a Docker container

Installing Google Authenticator PAM

It is super easy to install Google Authenticator on Ubuntu. Below is the command we need to install Google Authenticator PAM on Ubuntu.

$ sudo apt-get install libpam-google-authenticator -y

Configure tac_plus

As mentioned earlier, the instructions in my old blog post are still valid. We’re going to use only some of them in this post for the purpose of demonstration only.

By default, the /etc/tacacs+/tac_plus.conf file looks like this:

accounting file = /var/log/tac_plus.acct
key = testing123
user = DEFAULT {
       login = PAM
       service = ppp protocol = ip {}
}

Let’s change the key and user information fields to look something like this:

accounting file = /var/log/tac_plus.acct
key = tacacskey1234
user = tacacsuser {
        member = Administrators
}
group = Administrators {
        default service = permit
        login = PAM
        enable = file /etc/passwd
}

Restart TACACS+ daemon

Since we made a change to our tac_plus config file, we need to restart the service for our changes to take effect. Issue the command below.

$ sudo /etc/init.d/tacacs_plus restart
[ ok ] Restarting tacacs_plus (via systemctl): tacacs_plus.service.

An alternative command is shown below.

$ sudo service tacacs+ restart

Generating Google Authenticator Secret Key

This step is covered in my old blog post so head over there and skip to the generating the secret key section. Alternatively, we could use the same secret key(s) from another system with Google Authenticator. However, this is not the recommended practice.

To get the secret key from another system, just copy and paste the ~/.google_authenticator file of each user, like the one below.

tacacsuser@tacplus:~$ more .google_authenticator
UXQLCMOLT2QLSMVE
" RATE_LIMIT 3 30 1436015893
" DISALLOW_REUSE 39787632
" TOTP_AUTH
55312114
13740459
80118802
81859009
79311140

If you copy and paste it to a file, make sure that the permission is set to read only.

tacacsuser@tacplus:~$ ls -l .google_authenticator
-rw-rw-r-- 1 tacacsuser tacacsuser 129 May 29 17:54 .google_authenticator
tacacsuser@tacplus:~$ chmod 400 .google_authenticator
tacacsuser@tacplus:~$ ls -l .google_authenticator
-r-------- 1 tacacsuser tacacsuser 129 May 29 17:54 .google_authenticator

Configuring TACACS+ PAM

Since we instructed tac_plus to use PAM, we now need to create a file called /etc/pam.d/tac_plus, so PAM knows what to do. The file should look like:

$ more /etc/pam.d/tac_plus
auth requisite pam_google_authenticator.so forward_pass
auth required pam_unix.so use_first_pass

IOS configuration

Before we can verify that our tac_plus config is working, let’s configure a CSR1000V router running IOS-XE.

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#aaa new-model
R1(config)#aaa authentication login default group tacacs+ enable
R1(config)#aaa authentication enable default group tacacs+ enable
R1(config)#aaa session-id common
R1(config)#tacacs-server host 192.168.250.250
 Warning: The cli will be deprecated soon
 'tacacs-server host 192.168.250.250'
 Please move to 'tacacs server <name>' CLI
R1(config)#tacacs-server directed-request
R1(config)#tacacs-server key tacacskey1234
R1(config)#end
R1#

While the configuration above still works, it is a good idea to move towards the new way of doing things. Here’s the new way of configuring TACACS+:

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#aaa new-model
R1(config)#aaa authentication login default group tacacs+ enable
R1(config)#aaa authentication enable default group tacacs+ enable
R1(config)#aaa session-id common
R1(config)#tacacs server tac_plus
R1(config-server-tacacs)# address ipv4 192.168.250.250
R1(config-server-tacacs)# key tacacskey1234
R1(config-server-tacacs)#end
R1#

Verification

This CSR1000V router is using version 15.4(2)S image. The output may vary depending on what platform and the IOS version.

$ ssh tacacsuser@192.168.250.250
Password & verification code:
R1>en
Password:
R1#

For completeness sake, I will list the passwords entered in the example above. The tacacsuser account is a valid account on the Ubuntu server running TACACS+ daemon. For example, the tacacsuser account has a password of tacacsuserpassword1234. Next, the verification code is the six-digit number displayed on Google Authenticator app. For example, the six-digit number is 567 890. With this example, the user will enter tacacsuserpassword1234567890 in the password & verification prompt.

For entering the privileged EXEC mode, we’ll again use tacacsuser’s password. If we look at the tac_plus config file, the enable = file /etc/passwd is what we defined.

Final Words

I am quite biased towards TACACS+. One of the reasons why is because of the command authorization piece. TACACS+ authentication and authorization are completely separate. That said, we could assign different command authorization level for the user or group.

With RADIUS, it combines authentication and authorization. Once the user authenticates successfully, the access-accept packet sent by RADIUS server to the device contain authorization information as well. If we configure the device similar to the example here, then the user will have full access.

To overcome RADIUS’ drawback, we could configure the device to use a local enable secret. This password is then shared only with the necessary user(s) or group(s). The issue with this approach is that the password is then a shared password. A lot of information security professionals do not like shared passwords because it is insecure.

Having said all that, I think it’s better to use TACACS+, especially with the Cisco-centric environment. While a lot of vendors support TACACS+, there might be some limitations on the authorization piece.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

You might also like to read

How to build tac_plus server
TACACS+ (tac_plus daemon) ACL
How to configure AAA on Cisco router/switches
Enabling AAA on Cisco ASA
Deploying TACACS+ on a Docker container

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Implementing Wired 802.1X

06/29/2015 By Andrew Roderos 2 Comments

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

I first learned about 802.1X when I was studying for one of the CCNP exams, BCMSN exam (SWITCH equivalent), at Ohlone College. At the time, I assumed that the short material covered in the book was all of it. Of course, that was a bad assumption in my part. That’s probably a normal assumption of someone who at the time just finished Cisco Network Academy Program CCNA 1 to 4 and newly minted CCNA with no professional experience.

What is 802.1X?

Essentially, 802.1X is a security feature that provides a mechanism to authenticate devices before it can access network resources. While it’s a good idea to have this security feature implemented, I’ve worked for companies who didn’t have this feature or similar implemented or it’s on their roadmap. It’s a shame that it wasn’t on their roadmap a long time ago since it was ratified in 2001. Then again, implementing technologies have its challenges.

How it works?

802.1X_wired_protocols
Image from Wikipedia

While there are other sources that will explain this in detail, this post includes a very short description on how it works. Basically, when a device connects to the wired network, the authenticator (switch) will send an EAP message to the supplicant (computer). If the computer has a supplicant, it will send an EAP response to the authenticator. The authenticator will then send a RADIUS message to the authentication server (RADIUS server). The authentication server will then challenge the supplicant to verify its identity. Once verified, the device will then be able to connect to organization’s network resources.

Environment

Every organization has their own unique implementation of technologies, so gather what you can and go from there. For example, in this scenario the requirements were to have two sets of RADIUS servers: one for switch-based authentication and the other for port-based authentication. This seems to be an uncommon setup so it required some research to split the two sets of RADIUS servers. My initial assumption was that it wasn’t possible. That assumption is only correct in older code, but with IOS 15.x the feature is supported.

This is a multivendor organization so LLDP is used instead of CDP, which is disabled globally by default due to the switch template configuration.

A lot of users are using Apple notebooks and/or desktops and most of these users are running VMware Fusion to run Windows and/or Linux.

IP phones are ubiquitous so this requires a great deal of attention. If my memory serves me right, the 802.1X topic in BCMSN didn’t cover how to implement it with IP phones so Cisco’s documentation and Google were my friend during my research.

A lot of devices do not have supplicant and there are instances where PXE boot is needed.

In addition, there were some locations that need WoL (Wake on LAN) feature so that needs an attention as well.

Configuration

As mentioned earlier, the requirement is to have two separate RADIUS servers for both switch-based and port-based authentication. That said, let’s take a look on how to do this. But first, let me show you how it was done prior to IOS 15.x code. This command still works in 15.0(2), but you’ll receive a warning saying that it will soon be deprecated.

Old format

radius-server host 192.168.1.1 auth-port 1812 acct-port 1813
radius-server host 192.168.1.2 auth-port 1812 acct-port 1813
radius-server retransmit 1
radius-server timeout 2
radius-server key 7 hashkeyhere

Since the requirement is to split the RADIUS servers, we need to use the new format of specifying the RADIUS servers which will be needed when we create the AAA groups.

New format

radius server switch-auth1
 address ipv4 192.168.1.1 auth-port 1812 acct-port 1813
 timeout 2
 retransmit 1
 key 7 hashkeyhere
radius server switch-auth2
 address ipv4 192.168.1.2 auth-port 1812 acct-port 1813
 timeout 2
 retransmit 1
 key 7 hashkeyhere
radius server dot1x-auth1
 address ipv4 192.168.1.3 auth-port 1812 acct-port 1813
 timeout 2
 retransmit 1
 key 7 hashkeyhere
radius server dot1x-auth2
 address ipv4 192.168.1.4 auth-port 1812 acct-port 1813
 timeout 2
 retransmit 1
 key 7 hashkeyhere

Enable AAA

Once enabled, authentication method for 802.1X needs to be defined. I included the one for the switch-based authentication with the port-based authentication for completeness sake. RADIUS accounting is turned on as well since it is listed as best practice in Cisco’s deployment guide.

aaa new-model
aaa group server radius switch-auth
 server name switch-auth1
 server name switch-auth2
aaa group server radius dot1x-auth
 server name dot1x-auth1
 server name dot1x-auth2
aaa authentication login default group switch-auth enable
aaa authentication dot1x default group dot1x-auth
aaa accounting dot1x default start-stop group dot1x-auth
aaa authorization network default group dot1x-auth

Enable 802.1X

Issue the command below.

dot1x system-auth-control

Configure switch ports

Next step is to configure each switch port that will use 802.1X. This command will automatically include dot1x pae authenticator in the running configuration so don’t be alarmed if you see it there. This is to ensure that dot1x authentication still works on legacy configurations without manual intervention. NOTE: It seems to be that the IOS that I was using automatically included the dot1x pae authenticator command. That said, please make sure to add the command if you do not see it.

interface range g1/0/1 - 48
 ! Make sure that the ports should at least have switchport mode access or it won't take the commands.
 authentication port-control auto
 dot1x pae authenticator

In the IOS 12.x, this would’ve been a different command. The command in the old world is dot1x port-control auto.

Technically, the commands above are all we need to configure for the 802.1X to work. However, the environment in this scenario requires more things from us that we still need to address.

VoIP phones

Let’s address the IP phones first since it’s ubiquitous within the enterprise environment. By default, the interfaces are set to be single-host mode. This means only one MAC is allowed in the data VLAN. This mode technically allows another MAC address but on the voice VLAN and only if CDP is supported. Since CDP is disabled on all of the switches deployed in this scenario, this needs to be enabled. I included the single-host mode command below since it won’t show up in the running configuration because it is the default configuration.

cdp run
interface g1/0/1 - 48
 authentication host-mode single-host

While this configuration works, there are few things that we need to keep in mind. The single-host mode means only single MAC can be authenticated on a switch port. If a different MAC address is detected on a port after an endpoint has authenticated then a security violation is triggered on the port. This will cause the port to be in errdisabled state and will require a manual intervention unless errdisable recovery is configured.

Since the computers are daisy chained on the back of the Cisco phones, there are technically two MAC addresses that will be seen on the port. As mentioned earlier, the single-host mode ignores the MAC address seen in the voice VLAN so this should work. It does work, however, once you shut the port down and enable it again, and phone or switch reboots, the switch port will see two MAC addresses on the data VLAN.

Now, you’re probably wondering why would the switch see two MAC addresses in the data VLAN when the IP Phone should only show up in the voice VLAN especially when the boot process is described in books like this. But, I’ve seen this happened in all three organizations I’ve worked for where the phone’s MAC address shows up in both data and voice VLAN, as shown below. If you do a quick search, you’ll see more people are seeing the same thing so it appears that this is the default behavior.

switch#sh mac add int g1/0/1
          Mac Address Table
-------------------------------------------
Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
  10    0004.f2f0.4d98    DYNAMIC     Gi1/0/1
  20    0004.f2f0.4d98    DYNAMIC     Gi1/0/1
Total Mac Addresses for this criterion: 2

As you can imagine, this could turn to an operational nightmare especially when you have facilities people going in and out of the closet to do some work and they occasionally bump into the power cord of the switch by accident. The solution is to change the host mode to something that will not cause a security violation. One option is to use the Multi-domain authentication (MDA), which is shown below.

interface range g1/0/1- 48
 authentication host-mode multi-domain

MDA vs Multi-Auth

Multi-domain authentication (MDA) allows one MAC address on both data and voice VLAN. It is kind of similar with the single-host mode but this mode requires the device in the voice VLAN to authenticate. Initial testing looks like it’s working as expected. I didn’t see the same behavior where the phone’s MAC address shows up on both VLANs when I bounced the port.

MDA does not address the fact that the environment in our scenario will have users running VMware Fusion on their computer(s). When the user configures the VM with a network type of bridged mode, which means the switch will see two MAC addresses, then that will result in a security violation. This needs to be addressed so there has to be another mode that we could use. Fortunately, there is and it is called multiple authentication.

interface range g1/0/1 - 48
 authentication host-mode multi-auth

The difference between MDA and multiple authentication is that it allows multiple MAC addresses in the data VLAN, however, all devices must be authenticated to access the network resources.

As mentioned, there is a way to automatically recover from a security violation, by default it is set to five minutes. Before I show you the command for it, let’s think about the fact that the port will be in the errdisabled state once a security violation occurs. That means, the phones will be out of commission too. This is going to be frustrating for the users so we need to find a solution that only errdisable the VLAN where the security violation occurred. Fortunately, the switch has the voice aware 802.1X security feature and is shown below with the errdisable recovery.

errdisable detect cause security-violation shutdown vlan
errdisable recovery cause security-violation

MAC Authentication Bypass

The devices that do not support 802.1X feature still needs to access network resources so we need to find a way to let them in without disabling the port-based authentication where these devices are connected to. Cisco supports fallback mechanisms when a device fails to authenticate using 802.1X. A great option for devices that do not support 802.1X is the MAC Authentication Bypass (MAB).

With MAB, the MAC address is entered to the RADIUS server and when the device fails to authenticate using the 802.1X then the switch will fallback to MAB. The switch will then forward a message, with the MAC address of the device, to the RADIUS server. RADIUS server will then check its database to see if the MAC address is in its list. If it is, then the RADIUS server will signal the switch to allow access to the network. To enable MAB, issue the command below.

interface range g1/0/1 - 48
 mab

Another version of this command is shown below. If this command is used, the IOS will change it to mab in the running and startup config.

interface range g1/0/1 - 48
 dot1x mac-auth-bypass

While this fallback mechanism works, Cisco Catalyst switches have default values which delays the transition of a non-802.1X compliant from unauthorized to authenticated for 90 seconds. This might cause some issues with DHCP or PXE clients so it is recommended to tweak the default values to make it faster for the non-802.1X compliant devices to access network resources.

The 90 seconds is the combination of the dot1x max-reauth-req and dot1x timeout tx-period values. The default value for the former is two and the latter is 30 seconds. Multiply both values and the result is 60 seconds. You’re probably thinking where’s the other 30 out of the 90 seconds? Well, that was the initial request for the device to authenticate and when it fails the switch will then send a request. It would keep sending up to the configured max-reauth-req values when there’s no response from the device. It is recommended to test what’s best for your network since there are really no recommended values. For our scenario, let’s configure them with a value of one and 10 seconds.

interface range g1/0/1 - 48
 dot1x max-reauth-req 1
 dot1x timeout tx-period timer 10

The last thing that we need to address is the WoL feature that some people use in the environment. By default, traffic through the unauthorized port is blocked in both directions and the magic packet, WoL packet sent by the server, never gets to the sleeping computer.

To support the WoL feature in 802.1X environment, we’ll need to configure the switch to allow outbound traffic to the unauthorized port but still control the incoming traffic. The command to do this is shown below.

interface range g1/0/1 - 48
 authentication control-direction in

Other considerations

Not every scenario is covered here so I recommend you to read Cisco’s configuration and deployment guide about 802.1X. For example, what if all RADIUS servers that handles the port-based authentication are unreachable? That would mean, unauthorized ports trying to move to authenticated ports will not work. Configuring critical VLAN both for data and voice may be necessary for this environment.

For partners’ devices, how would you like to handle their access to network resources? Would you allow them by implementing a Guest VLAN feature?

If you opt for using EAP-TLS, how would you manage the deployment of the certificates to all devices including mobile? This might frustrate users and may also overwhelm the desktop support staff if not handled properly.

What if your organization use non-Cisco phones? What will happen to the devices behind the phones once it gets authenticated and gets removed from the port? Does it support EAPoL Logoff/Proxy EAPoL Logoff? This is not an issue with Cisco phones with CDP since it supports CDP Enhancement for Second Port Disconnect. With this feature, when the user disconnects from the phone’s port, the phone will signal the Catalyst switch to move the data VLAN from authenticated to unauthorized state.

How do you want to authenticate the phones? Do you want to use EAP-MD5, MIC (Manufacturer Installed Certificate), or LSC (Locally Significant Certificate)?

If you do allow MAB fallback mechanism, how do you combat the possibility of unauthorized users spoofing MAC addresses that are in your RADIUS’s MAC address database? If the organization is big enough, how do you manage adding MAC addresses to the database? How do you maintain the database properly without leaving temporary entries?

Thoughts

Deploying 802.1X definitely has its challenges. This could be the reason why some organizations choose to not have some type of port-based authentication because of it may affect the availability of network resources. When it comes to deployment, I believe proper planning and testing is needed to make it a smooth deployment. Few things that could be used to make it a smooth deployment are the following: monitor mode, low impact mode, and closed mode, which is covered in this Cisco Live! presentation. Some might just opt for the lab testing then move to pilot phase, which is doable in my opinion.

Are you ready to improve your network security?

Let us answer more questions by contacting us. We’re here to listen and provide solutions that are right for you.

ENGAGE US

References

CCNP SWITCH
Wired 802.1X Deployment Guide
Catalyst 2960X Configuration Guide

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

NetFlow-Lite on Catalyst 2960-X

04/19/2015 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Several months ago, sFlow became instrumental in figuring out the issue with HP switches that we inherited. Just to give you an idea of what the issue was, the HP switches would sporadically drop off the network but the user data traffic was still flowing. Good thing it was only the management traffic that was dropping and not user traffic. With the help of sFlow collector, I was able to correlate the timestamps of when several HP switches went down and I found out that MLD (Multicast Listener Discovery) was the culprit. Tried to search the web for some answers but no luck. I upgraded the code of the switches but still no luck. Finally, I decided to contact HP Tech Support since they offer a lifetime warranty on hardware and software. When the tech support asked for the config, he saw that igmp querier was turned on and when we turned it off the problem never came back. Since we’ve been replacing the HP switches with Cisco Catalyst switches, I wanted to replicate some level of the sFlow functionality. Luckily, the Catalyst 2960-X supports NetFlow-Lite.

What is NetFlow-Lite?

Cisco defines it as shown below. If you want to read more about NetFlow-Lite, please read this. To me, it’s a way for a network professional to see some visibility of what’s on the wire and gather statistics.

NetFlow-Lite collects packets randomly, classifies them into flows, and measures flow statistics as they pass through the switch. It is a true flow-based traffic-monitoring mechanism that conserves valuable forwarding bandwidth when exporting flow-based data for analysis and reporting.

Prior to sFlow and NetFlow-Lite, I was somewhat exposed with NetFlow but it was very limited implementation. That NetFlow implementation was good enough for what we used it for. Besides, the traffic generated by devices and/or computers on the network were very specific to the business applications and the computers were locked down tight so it was not needed at all. The places where we needed application visibility had protocol analyzers deployed so there was not a whole lot of push to deploy NetFlow.

NetFlow-Lite is not available in all Catalyst switches, I believe it was first supported on Catalyst 4948 platform and now being supported on newer Catalyst switches. The NetFlow-Lite requires the FPGA (Field-Programmable Gate Array) that contains the logic to implement NetFlow engine. Without it, then there won’t be support of NetFlow-Lite. Hence, no support on older platforms.

NetFlow-Lite Configuration

If you want to know what the commands do, please visit the configuration guide here.

flow record netflow
 match datalink mac source address input
 match datalink mac destination address input
 match ipv4 protocol
 match ipv4 source address
 match ipv4 destination address
 match ipv6 protocol
 match ipv6 source address
 match ipv6 destination address
 match transport source-port
 match transport destination-port
 collect transport tcp flags
 collect interface input
 collect flow sampler
 collect counter bytes long
 collect counter packets long
 collect timestamp sys-uptime first
 collect timestamp sys-uptime last
!
flow exporter collector
 description To NetFlow Collector
 destination 192.168.1.100
 source Vlan100
 transport udp 9985
 template data timeout 60
 option interface-table
!
flow monitor netflow
 record netflow
 exporter collector
 cache timeout active 30
!
sampler netflow
 mode random 1 out-of 32
!
!
interface range Gi1/0/1 - 48
 ip flow monitor netflow sampler netflow input
!
interface range Te1/0/1, TeX/0/1
 ip flow monitor netflow sampler netflow input

NetFlow/sFlow Collector

There are many vendors out there that sell flow collector software. Vendors out there like inMon (sFlow creator), Plixer, ntop, SolarWinds, etc. Make sure that they support NetFlow v9 or IPFIX since that’s the format that NetFlow-Lite can export to. Most of these vendors have trial software that you could use to give you a demo of their product. I am sure they’ll be happy to do a webinar so that they could introduce you to their product before starting to play with their software.

Thoughts

While NetFlow-Lite gave us some visibility, I noticed that sFlow provided more information so it is still better than not having any visibility at all. If your switches are capable of doing NetFlow-Lite, I suggest you do some trial to see if it’s going to be helpful for your environment. For us, it’s definitely helpful to have visibility so it is still being used. Another pretty cool feature that I find it very convenient is the fact that it could tell you the switch and port number of the device you’re looking for. While it’s not quite of a big deal to just log in to routers and switches to trace the device you’re looking for, it’s rather inconvenient to do so, especially if you implement two-factor for your switch-based authentication.

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

Same Data and Voice VLAN

12/24/2014 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Yes, it’s not the best practice to put both data and voice traffic in the same VLAN and subnet, but I’ve recently encountered it in production and caused us some head scratching scenario. While it was probably best to redesign the whole thing since it doesn’t follow best practices, it was not going to fly in this case.

Configuration

The stack’s configuration was copied from an existing Catalyst 3750-X production switch. For the most part, it’s a basic configuration that you will encounter in a lot of production switches. Some of the configurations are the following: AAA, BPDU Guard, PortFast, Storm Control, Access and Voice VLAN (same VLAN number), etc. The 3750-X stack’s configuration was not changed for more than two years. There were no issues or complaints from the clients so it was pretty much safe to copy the configuration to the new Catalyst 2960-X stack. This stack will be the new location of the same users of the 3750-X stack – just moving to a new location.

Issue

The desktop support guy started hooking up Avaya phones and desktops to the switch and noticed that all desktops and phones connected to switch 1 (two in a stack) were not communicating to the network. The desktops and phones weren’t getting IP addresses, but once he connects to switch 2 everything started to work. The interface configurations on both switches were configured the same but for whatever reason it wasn’t working. I rebooted switch 1 and all the desktops and phones connected to it started communicating to the network.

Several hours later, the desktop support guy said that everything connected to switch 2 stopped working. I decided to reboot the whole stack at this point. Before I rebooted the stack, I noticed that one of the switches was in EX4 and the other one was in EX5. Normally, there would be a version mismatch error when there are two different IOS installed and shouldn’t join the stack, but on this particular instance both switches were in the stack. I decided to upgrade the EX4 switch to EX5 since that is the latest code anyway and we’re making that as a standard IOS.

After upgrading and rebooting the stack, I still noticed that computers and phones on switch 2 were not communicating. At that point, our client was now panicking because they were not compliant with their contract to their client. That said, we decided to replace the whole stack with new switches. The switch replacement worked for a while but the issue came back which we kind of suspected anyway – we were not fully convinced that it would resolve the issue. Since we’ve been using the Catalyst 2960-X with EX5 code for several months now, we know it works just fine with our standard configuration. The only thing different with this stack and others was the same voice and data VLAN. While the switch is solid, we’ve hit bugs starting with EX1 all the way to EX4. That said, I begun to suspect that we might be hitting a bug in EX5, so I decided to open a TAC case while it was on the working state.

Troubleshooting

A Cisco TAC engineer contacted via e-mail me to troubleshoot the issue since it was working at the time I opened the case. As usual, the first thing I had to give them was show tech output. I still do not know why I do not include that by default when opening a case.  Anyway, as mentioned, the issue came back and I engaged the TAC engineer as soon as I was informed it stopped working.

The TAC engineer started checking some stuff up to try diagnose the issue. The first thing he mentioned is the voice and data VLAN being the same and it’s not a recommended practice. I agreed with him but I also reminded him that we have this deployed in two locations and were working fine using Catalyst 3750-X. He even asked me to connect to those switches so he could see it in his own eyes.

Upon issuing few show commands, we finally found some clue on what the problem was. There was no spanning tree instance on the interfaces where the phones and desktops were connected to. It acted as if there was nothing connected to the port even though the switch sees something connected to those ports. He decided to take out the switchport voice vlan vlan-id  and the desktop started pinging and spanning tree instance on the port showed forwarding.

While the configuration worked for desktops, I still needed the phones to work. Having only switchport access vlan vlan-id on the port won’t work with the phones even if I separated phones and desktop connections. That said, we needed a way to configure the port for the phones to also work. At the time, I only knew two ways of configuring a port with phones and data connected to the switch: configuring access vlan and voice vlan commands on the interface and configuring the interface as a trunk with native VLAN set for data. Essentially, they will act the same thing so we didn’t want to do that.

Upon him talking to his colleague(s) and probably done his research, he came back and said we’re going to try the switchport voice vlan dot1p command. That solved the issue! While he explained what the command did, I was not quite happy with his explanation because I felt it was incomplete, so I decided to read the configuration guide and was quite disappointed it didn’t have more information about it.

If you do a search, you will find plenty of information about the command, which I hoped the configuration guide would have. From what I’ve gathered both online and from the TAC engineer, if configured, the switch will instruct the phone to use VLAN 0 for its voice traffic and also mark its voice traffic with their proper CoS values, 5 or 3 depending on what traffic it is. Once the frame is received, the voice traffic with VLAN 0 will be accepted and drop the voice and data traffic in the same VLAN configured on the port – access vlan command.

Thoughts

While the dot1p configuration worked, I still would’ve preferred to have two different VLANs for both data and voice which is the standard configuration anyway in our environment. But, for whatever reason, they decided to design it this way and it’s going to stay that way. Maybe when it breaks that’s when they are going to decide to separate it, but at this point we had to choose the battle we were going to fight.

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email
  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Interim pages omitted …
  • Go to page 8
  • Go to Next Page »

Footer

WORK WITH US

Schedule a free consultation now!

LET’S TALK

Copyright © 2011–2023 · NetworkJutsu · All Rights Reserved · Privacy Policy · Terms of Use