DHCP bad_address This address is already in use

Answered DHCP bad_address This address is already in use

  • Wednesday, June 13, 2012 10:17 PM
     
     

    Hello, I have a 80/20 split scope for some Cisco Wireless Access Points.  The scope serves 203 addresses 163 from one server and 41 from the other server.  We have been installing these Access Point just fine until the last week when we reached 96 Access Points and that is when I noticed we are seeing bad_address This address is already in use in the Address Leases folder on both servers in the Address Leases folder for the scope the scope In Use statistics is 100%.  I have tried just deleting the bad_addresses out of the Address Leases folder but they just show up again after a few minutes.  If I ping one of the bad addresses it times out so I have a couple of questions:

    How do I go about cleaning up the bad addresses and resolve the situation?

    How do I expand the IP range of a scope that is split?  Do I have to delete one scope and then modify the source scope then re-add the second scope or is there a better way?

    Environment

    2 Windows 2008 R2 Enterprise Servers

    3 Cisco Wireless Controllers

    Need to install 200 Cisco 1142n APs

    Thanks in advance for any advice:)

All Replies

  • Thursday, June 14, 2012 1:57 PM
     
     
    Why are you doing DHCP on more than one server?  Why not just use one (or cluster the two together) and use one scope?  Typically there is some DHCP overlap somewhere when you get the BAD_ADDRESS.  Are you sure none of your controllers are acting as a DHCP server?  Are these controllers in the same VLAN and are you priming the APs before deploying them?  I'm really going to need to see how your infrastructure is laid out to really help point you in the right direction.
  • Thursday, June 14, 2012 2:11 PM
     
     Answered

    Hello Issatia,

    First, the 80/20 design is really antiquated and meant for scenario like a main/remote office connected with a slow link.  The remote office would have 80% and the main would have 20%.  We can go into more details about this, but for scenarios where you just want to build a redundant DHCP infrastructure where the WAN connections are not of concern, a 50/50 design is more appropriate and much easier to manage.

    I do not recommend a clustered solution for DHCP as clustering services adds an extra layer of complexity and cost (shard storage for the DB), and a 50/50 works just as well, and more cost effective.

    You create the same scopes on both servers, and just add the appropriate exclusions to ensure there is no overlap of available IPs.

    With regard to BAD ADDRESS leases, they remain in effect for 1 hour, auto removed, but of course will return when the DHCP server tries to assign a lease for an IP that is in use on the network.

    This could be caused by several reasons...

    1) you have overlap in your DHCP scopes

    2) there are clients on the network that are assign static IPs within the DHCP range

    3) you have another unknown DHCP server assigning addresses in the same range.

    You first need to determine the cause before you can fix the issue with the correct solution.


    Guides and tutorials, visit ITGeared.com.

    itgeared.com facebook twitter youtube

  • Thursday, June 14, 2012 3:47 PM
     
     

    Russ,

    We were just splitting the scope between to servers in an attempt at redundancy and an attempt at load balancing I guess.  I double checked the controllers and their was no dhcp enabled on that VLAN.  Yes, all controllers and APs are on the same vlan91 and they are basically the only devices using the vlan.  We have primed one or two APs before deploying them but most of the time we plug them in for the first time at the site.   We have about 30 other scopes on the server and no issues except with this particular Scope Range and BAD_ADDRESS.  

    I decided to expand the vlan from 255.255.255.0 to 255.255.252.0 to give us room for future growth.  I then deleted the second scope and then the original scope (I didn't see anyway to change the netmask with out starting over).  I recreated the scope with an expanded range of 10.91.0.50-10.91.3.254 and a lease of 45 days last time it was unlimited and only on one server to see if I can weed out all the BAD_ADDRESSES and since doing that I have many more BAD_ADDRESSES and seems to be growing so far I have 384 addresses in use and I just created it about a 30 minutes ago.  The In Use statistics seems to be growing by about 10 every minute.  Really strange, I think I am going to sniff the traffic and see if I can figure out what is going on.  

    Thanks

  • Thursday, June 14, 2012 3:52 PM
     
     

    Jorge,

    Thanks for your recommendations.

    1- I don't think we have an overlapping DHCP scope because this is the only DHCP scope on this vlan91 but I am going to sniff the vlan to make sure.

    2 - All the statics I have tried to make in the first 48 IP addresses outside of the range of 10.91.0.50 - 10.91.3.254 (expanded to the end recently)

    3 - Not that I know of but going to sniff the traffic and see if I can figure it out.

    I decided to expand the vlan from 255.255.255.0 to 255.255.252.0 to give us room for future growth.  I then deleted the second scope and then the original scope (I didn't see anyway to change the netmask with out starting over).  I recreated the scope with an expanded range of 10.91.0.50-10.91.3.254 and a lease of 45 days last time it was unlimited and only on one server to see if I can weed out all the BAD_ADDRESSES and since doing that I have many more BAD_ADDRESSES and seems to be growing so far I have 384 addresses in use and I just created it about a 30 minutes ago.  The In Use statistics seems to be growing by about 10 every minute.  Really strange, I think I am going to sniff the traffic and see if I can figure out what is going on.  

    Thanks

  • Thursday, June 14, 2012 3:56 PM
     
     Answered

    Ugh, I wouldnt put that many addresses into one subnet, I know we have faster equipment these days and faster links, but broadcast storms could be a killer.  I know this is off topic for the DHCP question, but why are you not using Layer 3 for your access points?  The only downside to L3 on the WLC is that you have to prime every AP before it is deployed, but as long as the AP has connectivity back to the WLC it was primed to, it can use a local scope.

    Are your wireless clients connecting to the same VLAN (91)? You could have someone or multiple someones snagging addresses or putting statics on their devices over the wireless causing your bad address issue.  You can require DHCP on the SSID to help mitigate this.

  • Thursday, June 14, 2012 6:08 PM
     
     

    Thanks, for the update and you are right I too would like smaller subnets and on a side note when I inherited the network the the netmask was 255.0.0.0 with many all of the 35 subnets gateways at 10.5.255.254 for example so I thought I have been making big strides in making them much smaller with 255.255.248.0 being our bigger subnets.  

    I sniffed the vlan91 and every DHCP offer seemed to pointing to the correct DHCP server but the arp replies often have dulicate use of IP 10.91.2.85 for example and if I look at the duplicate IP records the mac addresses are different so it doesn't seem like it is one bad nic causing the problem.  I am not the best at reading wireshark logs but to me it looks like an arp problem.  So I tried clearing the arp-cache on the router restarting the DHCP server, all Wireless controllers, shutting down the vlan91 on the router and bringing the vlan back up and with no luck the scope Address Leases folder is now full of mostly BAD_ADDRESS and now some of the original APs that were working are starting to go down.

  • Thursday, June 14, 2012 8:15 PM
     
     
    well, I would make note of the two MAC addresses, see which one is on the device I know is supposed to get the address and then track both of them down to see what they really are.  This can happen when you delete a scope and a device still has time on its lease, but the DHCP server has no idea that the ip is already spoken for.
  • Thursday, June 14, 2012 10:31 PM
     
     

    What do you think I should do when I find one?  I think I have found two APs by mac address in this building that are contending for the same IP address and I know which one should have the address.  Should I try to exclude it or something?

    BTW, some of the APs are starting to show up on the DHCP server Address Leases folder correctly about 64 of them so far and still a lot of BAD_ADDRESSES 909 of them.

  • Friday, June 15, 2012 1:35 PM
     
     
    Wow, 909... what type of networking gear do you have?  Is it managed or unmanaged?  Do you know how to view the arp and mac tables on the networking gear?  I have the most experience with Cisco, and on a Cisco router/L3 switch you can use 'show ip arp | i <ip address> to see the mac for an IP that is showing up as a bad address.  Then you can use 'show mac dynamic <MAC Address>' to see what port the mac address is appearing on.  If the port is a link to another switch/site, then go to the next switch and repeat the process until you find the access port that the device is on.  Then you should be able to trace the device down from there unless it is one of your APs, then you can look in the WLC or WCS (if you are using it) and search for the MAC to see which AP it is.  One of the problems is, if your original scope was set to unlimited before, then these may not even check the DHCP server until the device is restarted/rebooted.  You maybe able to lower your lease times and ride the storm out, drop it down to a couple of hours and wait until the bad addresses go away completely, it may take a while though.  What kind of devices get addresses out of this scope in question?  Is it just APs or is it APs and clients connected to the APs as well?  If the latter is the case, you could restart all of your APs to try and force all clients to re-register with DHCP.
  • Thursday, June 21, 2012 6:32 PM
     
     Answered
    Thanks for all the help I ended up calling Cisco TAC and it ended up being the models of WLC we were using had a 48 AP limit per ap-manager port.  Once I added either a physical port or LAG setup then all the MS DHCP errors went away.