False Alarms in Cluster

[Jordan]

Hi all,

I have a bit of a strange issue that I'm sure I've seen before but can't remember how I fixed it (if I even did fix it!)...

We have a cluster of 2 controllers which when logged into show the group alarm status as Major. If I log into cluster element 1, the alarm status of 1 is clear, and 2 is major. If I then log into cluster element 2 the status of 1 is major and 2 is clear. Whichever element I log into always shows the other as having the alarm.

Any ideas what could be causing this and how to stop the false reporting of alarms? There is nothing wrong and no service impacting issues, but the alarm needs to be cleared.

Cheers,
Jordan

[sarond]

What is the actual alarm showing? Not the group alarm, the Alarm Details.

[Jordan]

Hi Sarond, that's the issue - the alarm details page has no alarms on either controller

[sarond]

From memory I think disabling SNMP and re-enabling can refresh the alarm state.
System Properties/System Administration/SNMP Configuration

[Jordan]

So, I've done that and it does clear the group alarm status on both controllers! But only for about 30 seconds  after that I get the original issue back again and again, neither controller has anything on the alarm details page.

[sarond]

Does anything show if you ALARMS in the maintenance commands

[Jordan]

Both show no alarm

[sarond]

OK, I'm at a bit of a loss now.

I'll let others chime in.

[Jordan]

Appreciate the help sarond, I thought you had the answer when I saw it go clear!

If anyone else has any ideas I'd appreciate it!

[ZuluAlpha]

I typically resolve this how sarond states, but when you disable SNMP Trap Configuration, wait about 60 seconds before you re-enable it.

[Jordan]

Just tried again doing SNMP and SNMP Trap config waiting 60 seconds but still no luck.

Also for what it's worth we've tried a sync between the controllers and that didn't help either.

[lundah]

Just to be clear and cover all the bases, you're using the same SNMP read-only string on both controller and they're in the same Administrative Group, right? Is there a firewall between the controllers?

I see this all the time on larger (10+ nodes) clusters, but for only 2 nodes that seems odd.

[Jordan]

Hi Lundah....So, yesterday when the others were helping I had noticed that the read only string was different and correct that, then followed their advice which didn't work.

Just went to sanity check everything again before I replied to you and saw the read only string had changed back to what it was originally. Switched that back to what it needed to be, disabled the admin group, disabled SNMP, waited 60 seconds, enabled SNMP and enabled the admin group and now both nodes are showing as clear!

One thing I can't remember is if at the top of the web page it normally says "Group alarms clear" or something? Both nodes only say "Node '[node name]' Alarm Status: No Alarm". Is that right and it only changes to group if there's an alarm on one?

Thanks for all the help everyone!

[august]

Major Alarm but System has no Alarms

Document ID HT1346  Version 5.0 Status Published  Published Date 09/06/2017
Products MiVoice Business,3300 ICP
Summary
Major Alarm but System has no Alarms
Question
Major Alarm but System has no Alarms
Answer
 

System is in a cluster and has a Major Alarm at top of ESM window and the Admin Group Alarm Summary shows a cluster element is in alarm, however Alarm Details show no alarms.

Solution

In the SNMP Configuration form

1. Verify "Enable SNMP Agent" is set to Yes

2. Verify all cluster elements have the same names in the Read only and Read/Write Community

NOTE: if "Accept Requests From All Managers" is set to YES - There is no need to configure IP addresses in the "Accept Requests from the following Managers" area of the form.

3. If all is configured properly - and there is still an active Admin Group alarm - Change the "Enable SNMP Agent" to "No", save, and then change back to "Yes" and save - the alarm should clear within a few minutes.