IP Phone DHCP problem

[Simon_Turnbull]

Hi Michael,

We are running Nortel CS1KE with Succession 5 and have mixed phones of 2004P2, 1140 and 1150.

We are using full DHCP to configure all of the phones and this has been working fine for several months. The last couple of days however, the vast majority of phones are dropping the connection and displaying ‘DHCP Server unreachable’

We can ping the phone from the DHCP server fine. Rebooting the phones does not make any difference, nor does deleting the lease from DHCP server or rebooting the DHCP server. After an hour or so the phones come back of their own accord??

I have never seen anything like this, and it does not effect all of the phones.

Has anyone experienced this, or any ideas on a possible cause/solution?

Many thanks, Simon

[Michael McNamara]

Hi Simon and welcome to the forums!

A few questions are necessary to help you along the path.

 Are you using any VLAN assignment with your IP telephony?
 Or are you assigning the voice VLAN via DHCP?
 What product are you using for your DHCP server?
 What is the lease time of your DHCP leases?

Generally speaking once the IP phone boots it won't look to the DHCP server again until 1/2 the lease time has elapsed. I'm currently using a DHCP lease time of 3 months which helps keep the traffic down.

I've personally seen issues when our DHCP server (Lucent-Alcatel VitalQIP) accidentally assigns an IP telephone an IP address that doesn't have DHCP option 128 assigned. In that case the IP phone gets an IP address but doesn't get the S1 with respect to the Call Server. The phone will just display "DHCP Server Unavailable" even though the server is available only the proper DHCP options are not being returned to the IP phone.

The quickest way to troubleshoot the problem is to setup a port mirror or span port. And grab a packet capture of the traffic in question. Analyzing the packet capture will help determine if the problem is with 1) DHCP server, 2) network , 3) IP phone.

Good Luck!

[Simon_Turnbull]

Hi Michael, thanks for the quick response.

We're running Windows 2003 Server for DHCP. We use DHCP to push out all the phone config so we have 191 in the data scope (VLAN1) set as 'VLAN-A:310'

And then we have 128 configured in the voice vlan (VLAN 310) as 'Nortel-i2004-A,192.168.35.12:4100,1,1;192.168.31.11:4100,1,1.'

The lease time for the phones is 8 days. I've got a wireshark capture of what happens when a phone normally boots so at the moment I'm waiting for the fault to re-occur to get a failed attempt.

But any ideas on this would be great. The phones have been running fine for 2/3 months and this has only just started happening this week, not effecting every phone?

There is nothing in the logs on the DHCP server or SIG server to point to anything obvious

[Simon_Turnbull]

Just a quick update on this.

The phones have not gone off of their own accord yet, but when rebooting my 1140 this morning it displayed the DHCP Server unreachable.

I have ran another wireshark scan and I see the phone receiving an IP address from the data scope, and option 191 is applied to put the phone into VLAN 310 (voice)

However, it does not get an IP address from the voice scope and end up with the same data IP again??

After removing the ip helper-address from the network switch, and adding it back on again, my phone reconnected after 15mins, however it is using DHCP option 157 to get the config details, and not 128?

When i check in local diagnostics and DHCP Information, all three options show as not provided: Nortel-i2004-A, Nortel-i2004-B and VLAN-A

I'm not sure what this normally show as I've not checked it before but something is not right?

[Michael McNamara]

I would highly recommend you increase the DHCP lease time as soon as possible. I would recommend 3 months or longer... no need for all that traffic, essentially the phone will renew it's DHCP lease every 4 days (half the DHCP lease time).

The phones are "going off on their own" most likely when they are trying to renew their DHCP lease (every 4 days based on your 8 day DHCP lease time). If you had a DHCP lease time of 3 months (90 days) the phone wouldn't bother communicating with the DHCP server until 45 days had passed, or if it was restarted.

What firmware version are you running on the IP phones themselves?

Have you made any DHCP server changes? Someone didn't provision some Nortel-i2004-B scopes did they? You can't have Nortel-i2004-A and Nortel-i2004-B scopes in the same subnet, the phones get confused when they make the initial request for Nortel-i2004-B and then fall back to Nortel-i2004-A.

Have you made any switch configuration changes? Has someone accidentally bridged two different VLANs together and the IP phone is getting an IP address without the proper DHCP options intermittently outside of the expected IP network.

I'm guessing you're probably going to have some type of problem like that. Ultimately the packet trace will probably reveal the problem and help you identify the culprit.

Good Luck!

[Simon_Turnbull]

Thanks Michael. I have upped the lease time to 93 days for the IP phones as you suggested.

We are running C6T for the 1140 phones and DCN for the 2004P2’s. There hasn’t been any change to the network or the DHCP server which is why we’re struggling. I’ve managed to get another Wireshark capture of a failed boot of a phone.

It acquires a DHCP address from the data VLAN and is assigned VLAN 310 for the voice VLAN via option 191. However, when the phone then releases the data IP address there is no other communication between the phone and DHCP phone and the boot obviously fails.

As this is happening on over 150 phones, they can’t be all faulty?

We’re trying to setup a test environment to try and shed any further light on what is happening.

[Simon_Turnbull]

Another question Michael, we're running HP 2626PoE switch in our test enviroment.

If we connect an 1140 and tag both the DATA and VOICE VLANs as opposed to having the DATA UNTAGGED it works? However, the i2004P2 will not work under either scenario, only disabling the DATA VLAN gets a connection?

What is the correct way to have the VLANs tagged on the LAN switches?

[Michael McNamara]

Hi Simon,

If you want to attach the packet trace here I would be happy to look at it.

You can configure the switch port and IP phone a few different ways. In the examples on my blog I configure the voice VLAN as tagged and then the data VLAN is untagged. This requires that I configure the switch port as UnTagPVIDOnly, and make the data VLAN the PVID (default VLAN for that port). What ends up happening is that voice packets are tagged 802.1q with the voice VLAN ID between the switch and the phone while packets from the PC port on the phone are untagged and are bridged to the PVID. When packets are destined to the PC port on the phone the switch will strip the PVID (802.1q tag) from those packets leaving them untagged.

You can certainly tag both the voice and data VLANs, you just need to configure the phone with that information.

If it was working something has changed.

Good Luck!

[Simon_Turnbull]

Hi Michael,

I have uploaded a failed and success boot of a phone from our live enviroment on the following link, newbielink:http://www.4shared.com/dir/12534292/7f047319/sharing.html [nonactive], along with the filter for the MAC address of the phone we used.

Our test enviroment is not giving back the same results as live though?

A 1140\1150 phone will boot correctly and be configured via DHCP when the port has both the DATA and VOICE VLANs set a TAGGED. I2004P2 does not work, only gets a data IP address.

Having the DATA untagged and VOICE tagged none of the phones connect?

There is no consistency between the live and test enviroments or even between the 11XX and I2004 phones.

Touch wood the problem has not re-occurred, but on manually booting a phone it happens again.

[Michael McNamara]

I'll look over your packet trace.

How is the phone configured? Can you go through the phone configuration and write down every option.

There's an answer here somewhere.

Cheers!

[Michael McNamara]

I looked over your packet traces and unfortunately there is a problem with the trace.

I don't see any 802.1q VLAN tags in the packet trace.  You can add the column in WireShark by going to Edit -> Preferences -> Columns and add a column for '802.1Q VLAN id'.

You might want to give this a look over, you need to guarantee that the device that is performing the packet trace is not stripping out the 802.1q tags.

http://wiki.wireshark.org/CaptureSetup/VLAN

Without the VLAN tags it's really impossible to figure out what's going on.

Cheers!

[Simon_Turnbull]

Hi Michael,

I have added that column to the Wireshark trace and no data is passed into it? However, when both the data and voice VLANs are tagged, the 1140/50 phones get the correct address and parameters passed in via DHCP.

The 2004P2 phones will not connect at all in the test environment.

We have the following setup:

Network switch
VLAN1 (default VLAN) - data
VLAN310 (voice) - voice

All ports are UNTAGGED for VLAN1
All ports are TAGGED for VLAN310
ip helper-address for BOTH VLANs set as DHCP server (10.201.30.119)

DHCP Server
Data scope - option 191 set as: VLAN-A:310
voice scope - option 128 set as: Nortel-i2004-A,192.168.35.12:4100,1,3;192.168.31.11:4100,1,5.

1150 Phone
LLDP - ticked
DHCP - yes
Enable Voice 802.1Q - ticked
VoiceVLAN - auto
Enable PC Port - ticked
Enable Data 802.1Q - unticked
PCPort Untag All - ticked
Cached IP - unticked

[Michael McNamara]

Disable (turn off/uncheck) LLDP on the phone and retest.

[Simon_Turnbull]

Still not working with the data VLAN set as untagged?

[Michael McNamara]

I'm now suspecting your Ethernet switch... If you take a PC and connect it directly to the port you were using to test the IP phone, does the PC work properly?  Can you get an IP address and communicate, etc?

If you can get a good packet trace that will definitely tell us what's going on... if you are using a laptop to perform the packet trace you need to make sure that the Ethernet driver isn't stripping the 802.1Q VLAN tags from the frames before passing them up the IP stack.