MX3 III randomly reboots & displays License Error?

[jalfield]

Hi all,

New to this forum & hoping I might be able to get some insight/assistance with an issue I'm experiencing at my hotel. Quick backstory on our property:

We have a Mitel MX3 III 3300 ICP (dated 2016 03) controlling approximately +/- 300 phones (5212/5312/5340).

A few weeks ago, our phone system rebooted itself without warning. Once it came back online, all of the phones began displaying a License Error message. I tried to connect to the ICP via IE as I always have before (using IE 9 emulation as the system does not seem to like any version newer than this); but, after entering my credentials on the log in page, the browser would time out while waiting and never allow me to access the menu. Our vendor was similarly unable to access the system remotely. The phones throughout the building would then start acting strangely - some would display the date/time of the reboot and never update, others would be stuck with either the "Applications Loading" message or "Please wait while updates..." screen, all while still displaying the License Error. Additionally, the guest names associated with each room via our PMS provider seem to revert back to the occupants that were in-house at the time of the original failure/reboot event.

The only way I found that I could gain access to the ICP was by removing the LAN network connection and rebooting the controller. Once it had come up fully (after 5-10 minutes) I could connect my laptop directly to the LAN port and access the User Interface (with no delay at all) in the manner mentioned above. Upon reconnecting the LAN cable from our building's core switch, the phones throughout the building would reconnect and the License Error would be cleared; but then I would experience the same time-out/unresponsiveness when trying to access the interface again. After a day or so, the License Error would return to all of the phones.

Just this morning, the ICP apparently rebooted itself again. I'm having a hard time making headway with our service vendor as they seem to only be interested in replacing our entire phone system rather than repairing our current ICP. I've been informed that our phones are no longer supported by Mitel and neither is our outdated ICP.

That's why I thought I would reach out to the professionals here for any thoughts/advice/insight. Any help would be greatly appreciated.

Thanks!

[jalfield]

UPDATE: I just had to reboot the ICP in order to make a change to call forwarding on an extension (fun fact: when the system is acting up, the SuperKey functions time out as well, just like the User Interface). Before reconnecting the ICP's LAN port to our network switch, I exported an updated log of events/errors. It's mostly lost on me; but I'd be happy to share it with anyone who feels it might be a useful tool in seeing what might be causing our problems...

Thanks again!

[acejavelin]

I would really suggest getting your vendor involved here... this system's software is past EOL... this could be database corruption, hardware failure, some kind some kind of network problem on the voice VLAN, or a failure of the controller itself (like the E2T, which controls network functions).

I would highly suggest getting the system re-enlisted with Software Assurance, and upgrade it to the current release. Doing this involves restoring the database which also eliminates the database corruption issue (bad data get's ignored, never restored... old Mitel proverb). Once the software and database are considered "clean", you can dig deeper and involve tech support if necessary (tech support will not assist when you don't have software assurance or are more than 2 revs out from current release).

The MXe III is still a solid platform, although it is starting to show it's age, for ~300 phones it should serve you well for some time.

[ZuluAlpha]

The license error could be due to having the outdated root cert as well which was a big issue a couple of years ago. Mitel does have a fix for that.

5300 model phones are still supported and work fine, but don't expect anything new to come out for them. As for the reboots, without access to the logs it's just hard to say.

[jalfield]

@acejavelin & @ZuluAlpha - thanks for the insights & suggestions! Our current vendor is seemingly taking the hardline of "your equipment is outdated and no longer supported by Mitel and needs to be replaced"; so I may have to shop around for an alternate service provider. In the interim, I'm happy to email or DM the log that I pulled from the ICP on Tuesday for your review. At an initial (and untrained glance), there seem to be a large number of Warning alerts containing "OS (Message) warning: Receiver's mailbox is full..." Not sure if that points to anything or if this is a normal condition.

Thanks again!

[ZuluAlpha]

I would check the logs just before and after a reboot. I had an "issue" with one of our systems doing that and the cause was "power_on_reset" indicating it lost power. Turns out it wasn't the PBX but rather a UPS that was giving up the ghost.

[johnp]

This almost sounds like there is a duplicate ip address or loop somewhere in the network