Backup Policies

From OSNEXUS Online Documentation Site
Jump to: navigation, search


Within QuantaStor you can create backup policies where data from any NFS or CIFS share on your network can be automatically backed up for you to your QuantaStor system. To create a Backup Policy simply right-click on the Network Share where you want the data to be backed up to and choose the 'Create Backup Policy...' option from the pop-up menu. Backup policies will do a CIFS/NFS mount of the specified NAS share on your network locally to the system in order to access the data to be archived. When the backup starts it creates a Backup Job object which you will see in the web interface and you can see the progress of any given Backup Job by monitoring it in the Backup Jobs tab in the center-pane of the web interface after you select the Network Share to which the backup policy is attached.

Right-click on a share, then select Create Backup Policy...

Creating Backup Policies

Backup policies in QuantaStor support heavy parallelism so that very large NAS filers with 100m+ files can be easily scanned for changes. The default level of parallelism is 32 concurrent scan+copy threads but this can be reduced or increased to 64 concurrent threads.

Navigation: Storage Management --> Network Shares --> Network Share --> Create Backup Policy... (rightclick)

Network Share

This is where you indicate where you want the data to be backed up to on your QuantaStor systems. With QuantaStor backup policies your data is copied from a NAS share on the network to a Network Share on your QuantaStor system.

Policy Name

This is a friendly name for your Backup Policy. If you are going to have multiple policies doing backups to the same Network Share then each policy will be associated with a directory with the name of the policy. For example, if your share is called "media-backups" and you have a policy called "project1" and a policy called "project2" then there will be sub-directories under the "media-backups" share for "project1" and "project2". In order to support multiple policies per Network Share you must select, in the Advanced Settings tab, the option which says 'Store files to policy specific subdirectory on destination'. If that is not selected then only one policy can be associated with the network share and the backups will go into the root of the share to form an exact mirror copy.

Backup Policies use parallelized scanning and copy techniques to efficiently backup (copy) or migrate (move) data between Network Shares and remote data sources. Long Term Retention Rules only apply to ZFS and CephFS share types.
Create Backup Policy - Adv Set.jpg

Selecting the Backup Source

Under the Policy Settings tab the section which says Hostname / IP Address: enter the IP address of the NAS filer or server which is sharing the NAS folder you want to backup. For NFS or SMB shares you should enter the IP address and press the Scan button. If NFS shares are found they'll show up in the CIFS/NFS Export: list. For CIFS share backups you'll need to enter the network path to the share in a special format starting with double forward slashes like so: //username%password@ipaddress. For example, you might scan for shares on a filer located at using the SMB credentials of 'admin' and password 'password123' using this path: //admin%password123@ In AD environments you can also include the domain in the SMB path like so //DOMAIN/username%password@ipaddress.

Create Bkup Policy Settings.jpg

Policy Type

You can indicate that you want the backup policy to backup everything by selecting 'Backup All Files' or you can do a 'Sliding Window Backup'. For backing up data from huge filers with 100m+ files it is sometimes useful to only backup and maintain a sliding window of the most recently modified or created files. If you set the Retention Period to 60 days then all files that have been created or modified within the last 60 days will be retained. Files that are older than that will be purged from the backup folder.

Be careful with the Backup All Files mode. If you have a Purge Policy enabled it will remove any files from the network share which were not found on the source NAS share that's being backed up. If you attached such a backup policy to an existing share which has data on it, the purge policy will remove any data/files that exists in your QuantaStor Network Share which is not on the source NAS share on the remote filer. So use caution with this as Backup All Files really means maintain a mirror copy of the remote NAS share.

Purge Policy

Backup policies may run many times per day to quickly backup new and modified files. A scan to determine what needs purging is typically less important so it is more efficient to run it nightly rather than with each and every backup job. For the Sliding Window policies the purge phase will throw out any files that are older than the retention period. For the Backup All Files policies there is a comparison that is done and any files that are no longer present in the NAS source share are removed from the backup. The Purge Policy can also be set to 'Never Purge Files From Backup' which will backup files to your Network Share but never remove them.

Create Bkup Policy - Retention.jpg

Backup Logs

If you select 'Maintain a log of each backup' from the Advanced Settings tab then a backup log file will be written out after each backup. Backup logs can be found on your QuantaStor system in the /var/log/backup-log/POLICY-NAME directory. The purge process produces a log with the .purgelog suffix and the backup process produces a log with the .changelog suffix.

Deleting Backup Policies

To delete an existing Backup Policy simply right-click on the policy and choose the delete option from the menu. Deleting a backup policy does not delete any data, just the policy itself and any Backup Jobs associated with it.

Data Mover Utility / pwalk

pwalk is a open source command line utility extended by OSNEXUS and included with QuantaStor and used for concurrent file copy/movement and scanning, more information is available here.