Symptoms

There are backups found on an OACI backup node that are present on the file system, but are missing in OACI Control Panel and Database. How to remove such backups properly and preserve consistency?

Resolution

Use the attached sync_backups.sh script to be run on OACI Instance Manager node. There is also a 'dry' version of the script in attachment without backup removal step sync_backups_dry.sh.

Prerequisites:

  • Install postgresql client packages:

    # yum install postgresql96
  • Configure SSH passwordless access between IM and all backup nodes, registered in OACI. This step allows the script to be run with no interaction. If SSH keys are not placed on the backup nodes, the script will ask for the password to be input during its execution.

To run the script:

# bash sync_backups.sh

The script algorithm:

  1. Connect to the IM database, get the list of backup nodes and their IPs. For each backup node follow the next steps.
  2. Get the list of backups located on this backup node per IM database.
  3. Get the list of full backups located on the backup node per prlctl output.
  4. Get the list of incremental backups located on the backup node per prlctl output.
  5. For each full backup from the backup node list:
    • if this full backup is not present in IM DB list, delete it from the node and go to the next full backup
    • if this full backup is present in IM DB list, follow the next step.
  6. For each incremental backup, belonging to the current full backup, starting from the last in the chain:
    • if this incremental backup is not present in IM DB list, delete it from the node and go to the next incremental backup
    • if this incremental backup is present in IM DB list, stop here and go the next full backup

Internal content

Link on internal Article