There are several ways to troubleshoot DHCP issues. One could verify that DHCP scope is good, verify if the DHCP relay configuration is configured correctly, etc. Today, this blog post will cover a possible solution for DHCP issues that one might encounter. Though, this might be unlikely to happen in some networks. Of course, that depends on the environment.
Symptom
There were several IP phones spread throughout the enterprise that was working before but then stopped getting IP addresses. Other phones on the same VLAN were getting IP addresses just fine though. Several buildings were affected by this issue and when the phones were replaced, they were still not getting IP addresses. If they were brought back to the tech’s office, the phones did get IP addresses but the topology is not the same, so the only thing that test tells us was that the phones were not the issue.
Troubleshooting
Few steps were already taken so either one would verify everything again or save some time and continue where it was left off before the escalation. For example, DHCP scope was checked already to make sure they were correct and there were available IP addresses left to give out, voice VLAN was set, VLAN is on the switch’s VLAN database, DHCP relay configurations were correct, etc. One interesting piece of information was that the DHCP server never saw DHCP Discover and/or DHCP Request from the phones that were having issues. Majority of the time (at least in my experience), the issue was the relay configuration. This time, it was not. This makes sense since only a few phones were affected and not the whole voice VLAN.
So far, this tells us that somewhere in the network is dropping packets. Before busting out the Wireshark or Ethanalyzer, if available, to track where the packet is being dropped, one possible location where it is being dropped is the gateway. In this scenario, the gateway is a pair of Nexus 7000 running HSRP. With Nexus switches, CoPP (Control Plane Policing) is one of the configurations that will be set during the initial configuration.
In this scenario, the CoPP is set to strict. This can be verified by several ways and one way is by using show run | i “copp profile” command. To verify if the CoPP is being violated, one has to issue the command below.
N7K-enable# show policy-map interface control-plane class copp-system-p-class-normal-dhcp Control Plane service-policy input: copp-system-p-policy-strict class-map copp-system-p-class-normal-dhcp (match-any) match access-group name copp-system-p-acl-dhcp match redirect dhcp-snoop set cos 1 police cir 680 kbps , bc 250 ms module 3 : conformed 3430912611 bytes; action: transmit violated 1331596081 bytes; action: drop module 4 : conformed 6375239885 bytes; action: transmit violated 2201866339 bytes; action: drop
As shown above, the CoPP for DHCP is being violated and packets are being dropped. There are some instances that the nodes will recover since it will send DHCP Discover again, but there might be times where a custom CoPP is needed. To verify that this policy is the right one for DHCP, issue the commands below.
N7K-enable# sh run copp all | sec normal-dhcp class-map type control-plane match-any copp-system-p-class-normal-dhcp match access-group name copp-system-p-acl-dhcp match access-group name copp-system-p-acl-dhcp match redirect dhcp-snoop N7K-enable# show ip access-lists copp-system-p-acl-dhcp IP access list copp-system-p-acl-dhcp 10 permit udp any eq bootpc any 20 permit udp any neq bootps any eq bootps
Custom CoPP
To customize the CoPP to fit in one’s environment, the recommended is to double the rate and monitor for issues. If problem persists, continue to modify the rate until the issue is gone. Make sure to monitor the CPU as well because this might be impacted with the changes on the CoPP since we’re allowing more packets to go through the control plane.
The first step in customizing the CoPP is to copy the profile.
N7K-enable# copp copy profile strict suffix custom-copp
Once the CoPP profile has been copied, the value(s) that need(s) modification can now be done.
N7K-enable(config)# policy-map type control-plane copp-policy-strict-custom-copp N7K-enable(config-pmap)# class copp-class-normal-dhcp-custom-copp N7K-enable(config-pmap-c)# police cir 2048 kbps bc 1250 ms conform transmit violate drop
The configuration is well above the recommended rate, but to eliminate the possibility of the CoPP being the issue, it might be a good idea to increase it high enough so that no packets will be dropped.
The last step is to now assign the policy to the control plane and some verification.
N7K-enable(config)# control-plane N7K-enable(config-cp)# service-policy input copp-policy-strict-custom-copp N7K-enable(config-cp)# exit N7K-enable (config)# exit N7K-enable# copy run start N7K-enable# show copp status Last Config Operation: service-policy input copp-policy-strict-custom-copp Last Config Operation Timestamp: 06:04:03 UTC Jan 1 2015 Last Config Operation Status: Success Policy-map attached to the control-plane: copp-policy-strict-custom-copp N7K-enable# show policy-map interface control-plane class copp-class-normal-dhcp-custom-copp Control Plane service-policy input: copp-policy-strict-custom-copp class-map copp-class-normal-dhcp-custom-copp (match-any) match access-group name copp-acl-dhcp-custom-copp match redirect dhcp-snoop set cos 1 police cir 2048 kbps , bc 1250 ms module 3 : conformed 980075838 bytes; action: transmit violated 0 bytes; action: drop module 4 : conformed 275544063 bytes; action: transmit violated 0 bytes; action: drop
Thoughts
This may be an uncommon solution for DHCP issues but once a similar symptom is being experienced, then it does not hurt to do some verification that the CoPP is being hit. There are still a lot things I have to learn in Nexus platform since it is quite different from the Catalyst platform. This is one of those things that is different between Nexus and Catalyst.
Want to learn more about NX-OS?
NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures
Disclosure
NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.