Author Topic: Unexpected Alarm Status - Clear Alarm  (Read 3029 times)

Offline dg

  • Full Member
  • ***
  • Posts: 113
  • Country: us
  • Karma: +2/-1
    • View Profile
Unexpected Alarm Status - Clear Alarm
« on: February 07, 2018, 12:12:48 PM »
Hi folks,
I just noticed I have a "Major" alarm showing in the main heading from January 26, 2018. When I click that to see the detail, it says Backup Failure, but no other details.
I don't have any backups scheduled, nor did anyone manually attempt any backup. At one time early last year, I had started a backup schedule, and it did fail, but have subsequently deleted that task from the scheduler, and the scheduler shows no tasks.

First of all, any ideas why it would tell me that error in the first place? and second, how does one clear it?

Thanks in advance for suggestions,
-dg


Offline sunspark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 980
  • Country: mx
  • Karma: +16/-1
    • View Profile
Re: Unexpected Alarm Status - Clear Alarm
« Reply #1 on: February 07, 2018, 03:35:22 PM »
Backup Failure alarm indicates a system backup failure(s) on the local node. Details are stored
in the /db/spyLog/alarms/backupAlarm file, which is renamed and can still be viewed after the
alarm is cleared. The directory is removed only during an upgrade or full install.
Only one alarm (always Major) is generated for any number of backup failures. The alarm
persists after a system reboot; it clears only after a system restore OR once the Audit Failure
alarm (if present) and other issues are cleared and the backup process completes successfully


Database Verification included in the backup process prevents the system from backing up
incomplete or corrupted data. If the verification fails, an Audit Failure alarm is raised and the
backup process fails. A backup failure (for any reason) results in a Backup Failure alarm. A
successful backup cannot be completed until both alarms are cleared. To clear the alarms,
perform a system restore. You can also monitor the current database condition by Scheduling
Database Audits or executing the DATABASE AUDIT command.

Offline acejavelin

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4064
  • Country: us
  • Karma: +129/-0
  • High-tech, heavy metal redneck!
    • View Profile
    • Like what I do and wanna help out? Send me a donation!
Re: Unexpected Alarm Status - Clear Alarm
« Reply #2 on: February 07, 2018, 09:49:41 PM »
We see this from time to time... Often it is related to a scheduled backup and the FTP server being unavailable for whatever reason, but other times it just seems random. If you can't find an obvious reason for it I usually do the following:

1) Perform a manual backup, if it completes normally, schedule a forced reboot... preferably when your schedule is open the next morning. Actually, I have never had this fail but better safe then sorry.
2) If the backup fails when done manually, stop and start the embedded VM services from the ssh shell with iPVM_Stop and wait for the confirmation, followed by a iPVM_Start and wait for the confirmation. Then return to step #1

98% of the time this will correct the issue... if not, if this is a stand-alone controller reboot and try again, if that fails a call to tech support might be in order. If this is a clustered controller, perform a full network sync and try the above steps, still fails, time to open a ticket.

On extremely rare occasions, ok once I think... we had to revert to a previous good backup, a restore operation corrects all database corruption issues, because if the data is corrupt it is skipped and not restored. It is often a good idea to connect to the serial port when doing a restore and capture the output so you can see if anything is skipped and check it manually after the restore.

Offline dg

  • Full Member
  • ***
  • Posts: 113
  • Country: us
  • Karma: +2/-1
    • View Profile
Re: Unexpected Alarm Status - Clear Alarm
« Reply #3 on: February 13, 2018, 10:16:57 AM »
Okay, thanks for the info - quite helpful.
I just ran a manual backup to my local PC and the alarm cleared out. Perhaps a silly question though -
Does the system just run a DB verification and backup on its own by default on a set schedule? If so, on what schedule? I'm not finding any settings for this anywhere.

Thanks in advance,
-dg

Offline Dogbreath

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 389
  • Country: gb
  • Karma: +18/-0
    • View Profile
Re: Unexpected Alarm Status - Clear Alarm
« Reply #4 on: February 16, 2018, 09:54:14 AM »
Maintenance and Diags > Scheduler.

Presumably you set it to annual when you were testing it early last year :-)


 

Sitemap 1 2 3 4 5 6 7 8 9 10