a few busted 5330 phones No longer requests DHCP and send Incorrect CDP duplex

[contra]

Hi many 5330 phones and 4 3300 controllers

We have a flat /12 network for all of our voice, the controllers and the scopes for the phones.

we use cisco swiches with a helper address on each SVI pointing to a 2003DHCP server prociding data VLAN addresses.

we use option 128 on these servers to reach the DHCP servers on the mitel controllers.

We have had a number of Ip phones that have lately been plugged into a different data vlan, and about 5 percent of them can no longer get a dhcp address.

The firmware hasnt changed and many of the phoens that have been moved still work fine. All host facing ports are configured the same way.  We use CDP.

After taking a wireshark capture of a working phone and a non working phone, the differences are:

Working:
sends 4 0x4545 frames
sends 2 EAPOL start Frames
sends 3 CDP frames
    sends a duplex value of FULL
Sends a DHCP Discover frame
Gets a DHCP Offer from the SVI on behalf of the win 2003 DHCP server
Sends a DHCP Request for the offer
Gets an Ack from SVI (Yo layer 3!)
sends Gratuatios arp for offered address (This a client side check for duplciate ip's addresses im guessing)
sends dhcp release grab the voice vlan adressess etc etc.....
Phone works

Non working phone:
sends 4 0x4545 frames
sends 2 EAPOL start Frames
sends 3 CDP frames
    sends a duplex value of HALF!!!!!!!!!!!!!
Sends a DHCP Discover frame
Gets a DHCP Offer from the SVI on behalf of the win 2003 DHCP server
DOES NOT Send a DHCP Request for the offer..........
Sends a DHCP Discover frame
Gets a DHCP Offer from the SVI on behalf of the win 2003 DHCP server
DOES NOT Send a DHCP Request for the offer..........

doesnt work

Now I have tried Manually setting Ip address, Vlans prioritys, as well as manually setting the duplex on the phone.

Both phones have identical firmware and identical hardware setups.

Both captures were taken on the same port.

the only difference is that the working phone sends a full duplex capability, and the non working one sends half.  The working phone sends a request for its offer, and the non-working one ignores the offer and tries a discover again.

we are using minet not sip

I have tried a factory reset

Anyway to manually force the firmware onto it again ?

 or factory reflash the device using my own tftp server ? Ive seen instructions on the web for sip, not minet.

We have lost aabout 10 phones to this "Disease"

[brantn]

Well I was going to say a factory reset would solve it but since you tried that obviously through the tools menu then I would then suggest to either static a phone or set into teleworker mode by holding the 7 key on boot then entering the ip of the controller. This should force it to the tftp server and get rid of sticky paramaters. Typically a factory reset will clear them.

[contra]

Hi thanks brantn, I have tried setting a static address with ICP server, gateway and all the same settings on my working phone and it doesnt work. I just tried teleworker pointing it to the tftp server / controller and Its still stuck.

Either the fact that its half duplex (even when I set it to H/Full manually) I think that because I cannot ping it statically it just cannot communicate at layer3.  The switch sees a mac address etc and broadcast are working as we can get as far as getting a response from the server, but the phone just ignores it.  There is a fundamental fault with it now. 

Is there no way for me to manually force some firmware onto this phone from a tftp server on my pc ? probably not as ive got no ip connectivity..... How would a mitel engineer fix this ? is there a hidden com port or something ?  Would anyone like to take a look at the wireshark capture?

[brantn]

And if you static the phone and plug it directly into your pc can you ping it? You can set the tftp server to your local machine but sounds like it might be hardware related at this point. Did they possibly get unplugged during a flash cycle? Mitel would replace the ROM inside the phone or reflash.

[contra]

Ill take it too my house and try as there are no cross over cables at my work, plus Ill be able to do some port mirroring on my ccie lab, as I dont have access to the network kit here.

Thank you for your suggestions

[ralph]

Would you have any opportunity to plug a phone on to a network that has a lower software rev?
Here's my thought: the phone will boot off a a different switch, pickup an older firmware load and boot.
Then you may be able to plug into your existing switch and that will force a reload of the firmware.

OTH is it possible that there is an arp cache on the data switches?   A data switch reboot would solve that.

Ralph

[contra]

Are you referreng to a part of the network that gives a tftp address of a controller that has a lower version of the fiirmware ? If this is the case I dont think we have one. As far as arp cache is concerned, I have tried this in 3 different buildings on about 5 different switches, all our arp caches on the switches.  Both DHCP and Static IP addresses cannot communicate with the network, I am not sure that even if I had access to an older firmware that I could get it onto the phone.  I was hoping for an up up down down left right left right B A start that reveals a button that can revert to a working firmware.  I would love to know how Mitel would fix this, at a lower level, some kind of ROMMON mode .....