OSNEXUS Online Documentation Site - Recent changes [en]

Guides:Index

Qadmin — Thu, 08 Oct 2026 21:00:07 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 21:00, 8 October 2026
Line 6:		Line 6:
	* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.		* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[Guides:Enterprise Shared Storage for Proxmox VE Clusters\|Enterprise Shared Storage for Proxmox VE Clusters]]''' — Shared storage for a Proxmox VE cluster is a block or file store that every node can reach, so virtual machines and containers can migrate between nodes and restart elsewhere when a node fails.
	* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).		* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
	* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''		* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''

Guides:Enterprise Shared Storage for Proxmox VE Clusters

Qadmin — Thu, 08 Oct 2026 21:00:03 GMT

osn-seo-utilities: proxmox-shared-storage @ 2579ba917a3f (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 3, 2026''

Shared storage for a Proxmox VE cluster is a block or file store that every node can reach, so virtual machines and containers can migrate between nodes and restart elsewhere when a node fails. QuantaStor provides that storage from a central appliance or HA cluster, and its open-source Proxmox VE storage plugin adds a native <code><nowiki>quantastor</nowiki></code> storage type that provisions one iSCSI volume per Proxmox disk, with snapshots, templates, clones, resize and migration driven from the standard Proxmox tools. This guide covers the shared storage options, how the plugin works, and a walkthrough from storage pool to first VM disk.

== Why it matters ==

Proxmox VE runs well on local disks, but local disks tie each guest to one node. Live migration, HA restart and a single place to manage capacity all need storage every node can see. Teams moving off VMware hit the same issue in reverse: a vSAN cluster combined compute and storage in the same hosts, and the replacement hypervisor needs somewhere to put the data.

External shared storage separates those concerns. Compute nodes can be added, upgraded or reinstalled without touching the data, and capacity grows by adding drives to the storage tier rather than buying matching hypervisor hosts. The storage tier also brings its own data services: snapshots, replication to a second site, encryption and role-based administration.

== How it works ==

Proxmox VE has several built-in ways to consume shared storage (see the Proxmox [https://pve.proxmox.com/wiki/Storage storage documentation]). QuantaStor can serve the common ones:

{| class="wikitable"
! Option !! What Proxmox sees !! QuantaStor side !! Best fit
|-
| QuantaStor storage plugin (iSCSI) || A <code><nowiki>quantastor</nowiki></code> storage; one block device per disk || Scale-up [[Storage Pools|storage pool]], one [[Storage Volumes|storage volume]] per disk || VM and LXC disks with per-disk snapshots and clones
|-
| NFS || A shared directory for disk images, ISOs and backups || [[NFS Configuration|NFS share]] on a storage pool || ISO libraries, backups, file-based disk images
|-
| Ceph RBD || Proxmox's own RBD storage type || [[Scale-out Block Setup (ceph)|Scale-out block storage]] on a Ceph cluster || Scale-out deployments across many storage nodes
|}

For the RBD option, Proxmox creates images directly with its own Ceph client. QuantaStor recognizes block devices that were created outside its own management by Proxmox, OpenStack or Kubernetes and cleans up their volume objects automatically, so the grid view stays consistent with the cluster. Ceph block storage needs its own Ceph pool, separate from any pool that holds a CephFS file system, and the web manager only offers compatible pools when you create block storage.

The rest of this guide focuses on the plugin, because it gives Proxmox the most direct control over each disk. It separates management traffic from storage traffic:

* '''Control plane.''' Create, delete, snapshot, clone, resize and access-control operations are REST calls to the [[QuantaStor REST API Reference Guide|QuantaStor REST API]] over HTTPS on port 8153, authenticated with HTTP Basic credentials.
* '''Data plane.''' Guest I/O runs over iSCSI on port 3260. The plugin drives the standard <code><nowiki>open-iscsi</nowiki></code> initiator with <code><nowiki>iscsiadm</nowiki></code>: it discovers the target, logs in and waits for the device before handing it to Proxmox.

Each storage volume is exported through its own iSCSI target at LUN 0, and the plugin addresses it by a stable <code><nowiki>/dev/disk/by-path/...-lun-0</nowiki></code> path rather than a <code><nowiki>/dev/sdX</nowiki></code> name. QEMU attaches the raw block device for VM disks; for containers, Proxmox formats the same device with ext4 and mounts it as the root filesystem.

On first activation, the plugin registers each Proxmox node as a [[Hosts and Host Groups|host]] using the initiator IQN from <code><nowiki>/etc/iscsi/initiatorname.iscsi</nowiki></code>. Before a node uses a volume, the plugin assigns that host to the volume's access control list, and it removes the assignment when the volume is released. A volume is visible only to the nodes actively using it.

Volume names follow the Proxmox convention, so mapping a disk to its volume needs no lookup table:

{| class="wikitable"
! Type !! Name pattern !! Example
|-
| VM or container disk || <code><nowiki>vm-<vmid>-disk-<N></nowiki></code> || <code><nowiki>vm-100-disk-0</nowiki></code>
|-
| Template base disk || <code><nowiki>base-<vmid>-disk-<N></nowiki></code> || <code><nowiki>base-100-disk-0</nowiki></code>
|-
| Template snapshot || <code><nowiki>template-base-<vmid>-disk-<N></nowiki></code> || <code><nowiki>template-base-100-disk-0</nowiki></code>
|-
| Snapshot || <code><nowiki><volume>_<snapshot></nowiki></code> || <code><nowiki>vm-100-disk-0_snap1</nowiki></code>
|}

The plugin does not patch or replace Proxmox source files, so Proxmox package upgrades do not break it. It is distributed as a Debian package from the [https://github.com/OSNEXUS/pve-quantastor-plugin pve-quantastor-plugin repository].

== Design and sizing ==

Plan the cluster around these requirements from the [[Proxmox Storage Plugin|plugin reference]]:

{| class="wikitable"
! Item !! Requirement
|-
| Proxmox VE || 9.1.x and 9.2.x, which can be mixed in one cluster. 8.4.x and older do not load the plugin, so keep 8.x nodes out of the cluster.
|-
| Storage || A storage pool to provision from, on a current QuantaStor release
|-
| Network || Every Proxmox node reaches the appliance on TCP 8153 (REST API) and TCP 3260 (iSCSI)
|-
| QuantaStor configuration || None beyond the pool: the plugin creates volumes, host entries and ACL assignments
|}

For production clusters, put the pool in a [[Create Storage Pool High-Availability Group|high-availability failover group]] so a storage controller failure doesn't take every VM down with it. The plugin's configuration options cover HA pools and SSL; see the plugin reference for the exact settings. The grid below shows scale-up ZFS pools next to scale-out Ceph block, file and object pools on the same grid, with the HA group controls in the toolbar.

[[File:Guide-proxmox-shared-storage-proxmox-storage-pools.png|frame|center|The Storage Pools grid: scale-up (ZFS) pools that can back the Proxmox plugin, scale-out block pools for Ceph RBD, and the Storage Pool HA Resource Group toolbar used to make a pool highly available]]

Keep the iSCSI data path on a dedicated storage network where possible. The REST API traffic is light, but guest I/O follows every disk.

== Setting it up ==

=== 1. Create a least-privilege API user ===

The <code><nowiki>admin</nowiki></code> account works, but credentials stored on the Proxmox cluster should carry only what the plugin needs. Create a [[Role Create|role]] under '''Security → Role → Create''', then a [[User Add|user]] with that role under '''Security → User → Add'''. The role needs view access to everything plus these write operations:

* '''Storage Volume''': create, delete, modify, resize, createSnapshot, deleteSnapshot, rollback, clone
* '''Storage Volume ACL''': add, remove
* '''Host''': add, addInitiator, modify, remove, assignVolume, unassignVolume

[[File:Guide-proxmox-shared-storage-proxmox-create-role-dialog.png|frame|center|The Create Role dialog: filter the Object Type list, tick the rows the plugin needs, then use Apply Permission Scope to Selected to give them System scope]]

Or create both in one pass with the [[QuantaStor CLI Command Reference|QuantaStor CLI]]:

<pre>qs role-add --name=pve-plugin-role --desc="Role for the Proxmox VE storage plugin" \
--permissions="*:view:system,\
StorageVolume:create:system,StorageVolume:delete:system,StorageVolume:modify:system,\
StorageVolume:resize:system,StorageVolume:createSnapshot:system,StorageVolume:deleteSnapshot:system,\
StorageVolume:rollback:system,StorageVolume:clone:system,\
StorageVolumeAcl:add:system,StorageVolumeAcl:remove:system,\
Host:add:system,Host:addInitiator:system,Host:modify:system,Host:remove:system,\
Host:assignVolume:system,Host:unassignVolume:system"

qs user-add --name=pve-plugin --password=<password> --role=pve-plugin-role</pre>

=== 2. Check connectivity from each Proxmox node ===

Before installing anything, confirm every node can reach both ports on the appliance (192.0.2.10 here):

<pre>nc -zv 192.0.2.10 8153
nc -zv 192.0.2.10 3260
cat /etc/iscsi/initiatorname.iscsi</pre>

=== 3. Install the plugin and add the storage ===

Install the <code><nowiki>pve-storage-quantastor</nowiki></code> package from the [https://github.com/OSNEXUS/pve-quantastor-plugin/releases releases page] on every node in the cluster. The <code><nowiki>quantastor</nowiki></code> type then appears under '''Datacenter → Storage → Add''' in the Proxmox web interface, alongside the built-in types. Enter the appliance address, the storage pool and the API user created in step 1. The plugin reference lists every configuration option, including the HA pool and SSL settings.

== Operating and testing ==

Once the storage is added, it works with the standard <code><nowiki>pvesm</nowiki></code>, <code><nowiki>qm</nowiki></code> and <code><nowiki>pct</nowiki></code> tooling. A quick check from any node:

<pre>pvesm status
qm create 9001 --name test-vm --memory 2048 --scsi0 <storage-id>:8
iscsiadm -m session</pre>

The new disk appears in QuantaStor as a storage volume named <code><nowiki>vm-9001-disk-0</nowiki></code>, and the node that owns the VM appears as a host assigned to it. Migrate the VM to another node and the assignment moves with it: the volume is released from the old node and assigned to the new one.

Snapshots, templates, clones, resize and migration all run through the plugin; the Usage section of the [[Proxmox Storage Plugin|plugin reference]] describes how each behaves. Each Proxmox disk is a regular storage volume, so the volume-level tools on the QuantaStor side also apply: [[Snapshot Schedules|snapshot schedules]] for point-in-time copies outside Proxmox, and [[Remote-replication (DR)|remote replication]] to a second site for disaster recovery. For the replication design, see [[Guides:Asynchronous Remote Replication and Disaster Recovery Failover|Asynchronous Remote Replication and Disaster Recovery Failover]].

== FAQ ==

=== Which Proxmox VE versions does the QuantaStor plugin support? ===

Proxmox VE 9.1.x and 9.2.x, and both can run in the same cluster. On 9.2 the plugin logs a cosmetic message about implementing an older storage API. Proxmox VE 8.4 and older do not load the plugin, so any storage operation routed to an 8.x node fails.

=== Does the plugin work for LXC containers as well as VMs? ===

Yes. Container root filesystems run on QuantaStor volumes: Proxmox formats the iSCSI block device with ext4 and mounts it as the container root, using the same provisioning and access control as VM disks.

=== Do I need to configure iSCSI targets or host entries on QuantaStor first? ===

No. Apart from the storage pool, the plugin creates everything itself: a volume and dedicated target per disk, a host entry per Proxmox node from its initiator IQN, and the ACL assignments that make a volume visible only to the node using it.

=== Can QuantaStor replace vSAN after moving from VMware to Proxmox? ===

It replaces vSAN's role as shared storage with an external tier. Proxmox nodes use QuantaStor over iSCSI through the plugin, over NFS, or over Ceph RBD for scale-out designs, and the storage tier adds HA pools, snapshots and remote replication of its own. Compute and storage then scale independently instead of in matched hypervisor hosts.

=== Should I use the plugin, NFS or Ceph RBD? ===

Use the plugin for VM and container disks when you want one volume per disk with storage-side snapshots, clones and per-node access control. Use NFS for ISO libraries, backups and file-based images. Choose Ceph RBD when the deployment needs to scale out across many storage nodes.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-proxmox-shared-storage-proxmox-create-role-dialog.png

Qadmin — Thu, 08 Oct 2026 21:00:03 GMT

Qadmin uploaded File:Guide-proxmox-shared-storage-proxmox-create-role-dialog.png osn-seo-utilities: screenshot for Guides (proxmox-shared-storage)

New page

osn-seo-utilities: screenshot for Guides (proxmox-shared-storage)

File:Guide-proxmox-shared-storage-proxmox-storage-pools.png

Qadmin — Thu, 08 Oct 2026 21:00:03 GMT

Qadmin uploaded File:Guide-proxmox-shared-storage-proxmox-storage-pools.png osn-seo-utilities: screenshot for Guides (proxmox-shared-storage)

New page

osn-seo-utilities: screenshot for Guides (proxmox-shared-storage)

Windows MPIO for Failover Clusters and Hyper-V

Qadmin — Thu, 08 Oct 2026 11:12:54 GMT

Add custom path recovery (UseCustomPathRecoveryTime Enabled, 40 s) to the recommended MPIO settings; iSCSI notes; validated over FC and iSCSI

← Older revision		Revision as of 11:12, 8 October 2026
Line 3:		Line 3:
	This page describes the recommended Microsoft Multipath I/O (MPIO) configuration for Windows Server hosts that use QuantaStor storage volumes as shared cluster disks, including Windows Server Failover Clustering, Cluster Shared Volumes, Hyper-V clusters and SQL Server Failover Cluster Instances. It applies to Fibre Channel and iSCSI access to a QuantaStor HA pool, and it is the client-side companion to [[Clustered SCSI-3 Persistent Reservations]].		This page describes the recommended Microsoft Multipath I/O (MPIO) configuration for Windows Server hosts that use QuantaStor storage volumes as shared cluster disks, including Windows Server Failover Clustering, Cluster Shared Volumes, Hyper-V clusters and SQL Server Failover Cluster Instances. It applies to Fibre Channel and iSCSI access to a QuantaStor HA pool, and it is the client-side companion to [[Clustered SCSI-3 Persistent Reservations]].

	'''Summary.''' Install MPIO, let the Microsoft DSM claim QuantaStor devices, and apply the recommended MPIO timer settings below on every cluster node. With this configuration, an HA failover or an appliance reboot is a brief, transparent pause for the cluster: cluster disks stay online, Cluster Shared Volumes keep running, and Hyper-V virtual machines continue without interruption. The configuration was validated on Windows Server 2022 Hyper-V clusters connected ~~over Fibre Channel~~ to a QuantaStor HA pair.		'''Summary.''' Install MPIO, let the Microsoft DSM claim QuantaStor devices, and apply the recommended MPIO timer settings below on every cluster node. With this configuration, an HA failover or an appliance reboot is a brief, transparent pause for the cluster: cluster disks stay online, Cluster Shared Volumes keep running, and Hyper-V virtual machines continue without interruption. The configuration was validated on Windows Server 2022 Hyper-V clusters connected to a QuantaStor HA pair over both Fibre Channel and iSCSI, including a live migration of the cluster's storage from iSCSI to Fibre Channel.

	{\| class="wikitable"		{\| class="wikitable"
Line 24:		Line 24:

	During an HA failover, the pool moves from one QuantaStor appliance to its partner. For a short interval, while the receiving appliance imports the pool, I/O is held rather than completed. With Fibre Channel and ALUA, QuantaStor keeps the standby paths through the partner appliance present throughout, so Windows sees the paths change state and continues on the new active paths, typically within well under a minute.		During an HA failover, the pool moves from one QuantaStor appliance to its partner. For a short interval, while the receiving appliance imports the pool, I/O is held rather than completed. With Fibre Channel and ALUA, QuantaStor keeps the standby paths through the partner appliance present throughout, so Windows sees the paths change state and continues on the new active paths, typically within well under a minute.

			With iSCSI connected through the pool's HA virtual interface, each disk has a single session that moves with the virtual interface: the session reconnects to the partner appliance once it has imported the pool. MPIO's custom path recovery returns that path to service as soon as the session is back, so the disk continues on the same path.

	MPIO decides how long to hold I/O and keep a disk while its paths change. The Windows defaults are tuned for general-purpose storage and allow only a short window (20 seconds before an unreachable disk is released, 60 seconds per I/O). The recommended settings extend that window so it comfortably covers an HA failover, including a slower one, for example when a busy pool takes longer to export. The cluster then experiences the failover as a short pause, and Cluster Shared Volumes and virtual machines carry on as normal.		MPIO decides how long to hold I/O and keep a disk while its paths change. The Windows defaults are tuned for general-purpose storage and allow only a short window (20 seconds before an unreachable disk is released, 60 seconds per I/O). The recommended settings extend that window so it comfortably covers an HA failover, including a slower one, for example when a busy pool takes longer to export. The cluster then experiences the failover as a short pause, and Cluster Shared Volumes and virtual machines carry on as normal.
Line 68:		Line 70:
	\|-		\|-
	\| RetryInterval \|\| 1 \|\| 1 \|\| Seconds between those retries.		\| RetryInterval \|\| 1 \|\| 1 \|\| Seconds between those retries.
			\|-
			\| UseCustomPathRecoveryTime \|\| Disabled \|\| '''Enabled''' \|\| MPIO retries a path that has become unavailable, so the path returns to service on its own once it is reachable again, for example when an iSCSI session reconnects after a failover.
			\|-
			\| CustomPathRecoveryTime \|\| 40 \|\| 40 \|\| Seconds between those path recovery attempts.
	\|}		\|}

Line 73:		Line 79:

	<pre style="font-size: smaller">		<pre style="font-size: smaller">
	Set-MPIOSetting -NewPathVerificationState Enabled -NewPathVerificationPeriod 30 -NewPDORemovePeriod 240 -NewRetryCount 3 -NewRetryInterval 1 -NewDiskTimeout 100		Set-MPIOSetting -NewPathVerificationState Enabled -NewPathVerificationPeriod 30 -NewPDORemovePeriod 240 -NewRetryCount 3 -NewRetryInterval 1 -NewDiskTimeout 100 -CustomPathRecovery Enabled -NewPathRecoveryInterval 40
	</pre>		</pre>

Line 88:		Line 94:
	RetryCount : 3		RetryCount : 3
	RetryInterval : 1		RetryInterval : 1
	UseCustomPathRecoveryTime : ~~Disabled~~		UseCustomPathRecoveryTime : Enabled
	CustomPathRecoveryTime : 40		CustomPathRecoveryTime : 40
	DiskTimeoutValue : 100		DiskTimeoutValue : 100
Line 96:		Line 102:

	The Fibre Channel HBA driver parameters can stay at the vendor defaults; this configuration was validated with the QLogic Windows driver at its defaults.		The Fibre Channel HBA driver parameters can stay at the vendor defaults; this configuration was validated with the QLogic Windows driver at its defaults.

			The Microsoft iSCSI Initiator settings can also stay at their defaults. The initiator's {{Code\|1=LinkDownTime}} (15 seconds by default) is how long it waits for a session to reconnect before it reports the path as unavailable; during a failover that is longer than this, custom path recovery is what brings the path back when the session reconnects. The configuration was validated with the iSCSI Initiator at its defaults.

	== Paths and load balancing ==		== Paths and load balancing ==
Line 109:		Line 117:

	A host with two HBA ports zoned to both appliances of an HA pair shows four paths per disk: two Active/Optimized and two Standby. After a failover the two sets swap roles, and the path count stays the same.		A host with two HBA ports zoned to both appliances of an HA pair shows four paths per disk: two Active/Optimized and two Standby. After a failover the two sets swap roles, and the path count stays the same.

			An iSCSI disk connected through a pool's HA virtual interface shows one Active/Optimized path per session. It has no Standby path, because the session itself follows the virtual interface to the partner appliance during a failover.

	== After maintenance on a QuantaStor appliance ==		== After maintenance on a QuantaStor appliance ==
Line 128:		Line 138:
	== Troubleshooting ==		== Troubleshooting ==

	'''A cluster disk did not stay online through a failover.''' Run {{Code\|1=Get-MPIOSetting}} on every node and confirm the recommended values are active; they take effect after a reboot. Also confirm that [[Clustered SCSI-3 Persistent Reservations]] is Enabled on the pool's HA group. To bring the disk back, run {{Code\|1=Update-HostStorageCache}} and a {{Code\|1=diskpart}} rescan on each node, then bring the cluster disk online in Failover Cluster Manager ({{Code\|1=Start-ClusterResource}}).		'''A cluster disk did not stay online through a failover.''' Run {{Code\|1=Get-MPIOSetting}} on every node and confirm the recommended values are active, including {{Code\|1=UseCustomPathRecoveryTime : Enabled}}; they take effect after a reboot. Also confirm that [[Clustered SCSI-3 Persistent Reservations]] is Enabled on the pool's HA group. To bring the disk back, run {{Code\|1=Update-HostStorageCache}} and a {{Code\|1=diskpart}} rescan on each node, then bring the cluster disk online in Failover Cluster Manager ({{Code\|1=Start-ClusterResource}}).

			'''A node no longer lists a disk after a failover, or the disk has no disk number.''' Rescan on that node as described above. If the disk is still missing, drain the node, restart it, and resume it; the disk returns with its paths and the node rejoins its Cluster Shared Volumes in direct mode. Then confirm {{Code\|1=UseCustomPathRecoveryTime : Enabled}} on every node.

	'''A disk shows fewer paths than expected, or Standby paths only.''' Rescan as described in [[#After maintenance on a QuantaStor appliance\|After maintenance on a QuantaStor appliance]]. Also check that both of the host's HBA ports are zoned to both appliances, and that both of its WWPNs are listed under the host in QuantaStor.		'''A disk shows fewer paths than expected, or Standby paths only.''' Rescan as described in [[#After maintenance on a QuantaStor appliance\|After maintenance on a QuantaStor appliance]]. Also check that both of the host's HBA ports are zoned to both appliances, and that both of its WWPNs are listed under the host in QuantaStor.

Guides:Index

Qadmin — Thu, 08 Oct 2026 00:00:18 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 00:00, 8 October 2026
Line 4:		Line 4:

	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
			* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
	* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).		* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
Line 13:		Line 14:
	* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]		* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
	* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.		* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.
			* '''[[Snapshot Schedules\|Snapshot Schedules]]''' — [[Category:admin_guide]] A '''Snapshot Schedule''' takes point-in-time snapshots of a chosen set of [[Storage Volumes\|Storage Volumes]] and [[Network Shares\|Network Shares]] on a repeating schedule, and rotates them so each volume and share keeps a moving window of recovery points.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

Snapshot Schedules

Qadmin — Thu, 08 Oct 2026 00:00:14 GMT

osn-seo-utilities: docs-triggering-a-snapshot-schedule-on-demand-snaps @ 2a0350bcdfcb (approved in the portal)

Show changes

Ceph over ZFS

Qadmin — Thu, 08 Oct 2026 00:00:04 GMT

osn-seo-utilities: docs-ceph-over-zfs-zfs-draid-pool-reserved @ d389deda0b81 (approved in the portal)

New page

[[Category:admin_guide]]
Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks. There are a number of benefits of this architecture but the biggest is the double layer durability localizes the rebuild from failed drives so that the Ceph cluster doesn't need to absorb that load and production workload impact. The setup is simple, on each node you provision a Scale-up (ZFS) Storage Pool, ideally in a declustered RAID (dRAID) layout when you have 40+ drives per node. These pools become the backing store for Ceph OSDs, which become one-to-one mapped to Storage Volumes (zvols). So after the pool is created you'll batch create something like 16x, 24x, or 32x Storage Volumes from each pool. We call this the P:V or physical-to-virtual ratio and usually you'll want to aim for anything from 2:1 to 4:1. A 3:1 ratio means that for a node with 90x HDDs one would make that into a pool and then provision 30x (90:30 = 3:1) Storage Volumes to be turned into OSDs. This reduction also reduces the number of OSD processes running on each node and the amount of RAM you need per system. As an example with all 90x HDDs getting directly turned into OSDs we'd allocate 4GB to 6GB of RAM per OSD which amounts to 360GB to 540GB of RAM. In contrast, with a P:V of 3:1 we are producing 30x OSDs rather than 90x OSDs for roughly the same capacity. 30x OSDs at 8GB RAM per OSD is just 240GB of RAM so we're roughly cutting the RAM requirements in half. The same goes for the required CPU power.. we'd normally allocate about 1 to 1.5GHz per HDD based OSD. That's 90GHz to 135GHz if using the 90x HDDs directly as OSDs. Allocating 2GHz for each of the 30x zvol based OSDs puts us at just 60GHz. Granted, with the second filesystem (ZFS) at work underneath Ceph you'll need to reserve CPU for it and RAM for its ARC layer so the reduction can be somewhat offset but you should still see an overall reduction of 25% in CPU and RAM requirements.

{| class="wikitable"
! Section !! Purpose
|-
| [[#How it works|How it works]] || The Use Case setting and what it changes
|-
| [[#Requirements|Requirements]] || What must be in place before you start
|-
| [[#Step 1: Create a Storage Pool reserved for Ceph OSDs|Step 1]] || Reserve a ZFS pool for Ceph OSDs
|-
| [[#Step 2: Create the zvols|Step 2]] || Carve the zvols that back the OSDs
|-
| [[#Step 3: Create the OSDs|Step 3]] || Build one OSD on each zvol
|-
| [[#Operating notes|Operating notes]] || Boot order, LVM visibility and missing OSDs
|}

__TOC__

== How it works ==

A ZFS Storage Pool carries a '''Use Case''' that is set when the pool is created and cannot be changed afterwards:

{| class="wikitable"
! Use Case !! Meaning
|-
| Default || The pool provisions Storage Volumes and Network Shares as usual.
|-
| Ceph OSD || The pool is reserved for zvols that back Ceph OSDs. Every Storage Volume created in it inherits the Ceph OSD use case.
|}

A Storage Volume with the Ceph OSD use case is consumed directly by the Ceph OSD running on it. QuantaStor never presents it to hosts over iSCSI, FC or NVMe-oF: assigning one to a host fails with the message that the volume ''is reserved as a Ceph OSD backing device''.

The setup has three steps, each covered below:

# Create a ZFS Storage Pool with the Use Case set to '''Ceph OSD'''.
# Create the zvols in that pool, one per OSD you want.
# Create the OSDs from the zvols in the Ceph cluster.

The Storage Pools list has a '''Use Case''' column, so you can tell a Default pool from one reserved for Ceph OSDs at a glance.

== Requirements ==

* The pool must be a ZFS pool. QuantaStor refuses the Ceph OSD use case for any other pool type, and refuses a Ceph OSD zvol in a non-ZFS pool.
* The Storage System that owns the pool must be a member of the Ceph cluster you create the OSDs in, because that system creates the OSD on its own zvol.
* The pool must be started when you create the OSDs.
* Each zvol backs exactly one OSD. A snapshot, or a volume that is assigned to hosts, cannot back an OSD.

See [[Scale-out Block Setup (ceph)]] or [[Scale-out Object Setup (ceph)]] for creating the Ceph cluster itself.

== Step 1: Create a Storage Pool reserved for Ceph OSDs ==

{{Navigation|Storage Management → Storage Pools → Create ''(toolbar)'' → Advanced Settings ''(tab)''}}

[[File:Docs-docs-ceph-over-zfs-zfs-draid-pool-reserved-pool-use-case.png|thumb|right|800px|The Use Case list on the Advanced Settings tab of Create Storage Pool.]]

Create the pool as described on [[Storage Pools]]: name it, pick the disks on the '''General''' tab and choose a '''RAID Type'''. The DRAID layouts suit this use case because the OSD layer is designed around a DRAID pool of HDDs (see [[#How QuantaStor treats a zvol-backed OSD|How QuantaStor treats a zvol-backed OSD]]).

On the '''Advanced Settings''' tab, set '''Use Case''' to '''Ceph OSD'''. The other options on the tab (I/O Profile, Enable Compression, Enable SSD Auto Trim, Block Size Offset (ashift)) work as they do for any pool.

The Use Case cannot be changed once the pool exists, so set it here.

From the CLI, use <code>[[QuantaStor CLI Command Reference#pool-create|qs pool-create]]</code> with <code>--pool-use-case=ceph-zvol-osd</code>:

<pre style="font-size: smaller">
qs pool-create --name=osd-pool-1 --disk-list=<disk1>,<disk2>,... --raid-type=DRAID2_1S --pool-use-case=ceph-zvol-osd
</pre>

The DRAID layouts the CLI accepts are DRAID, DRAID1_1S, DRAID1_2S, DRAID1_3S, DRAID2_1S, DRAID2_2S, DRAID2_3S, DRAID3_1S, DRAID3_2S and DRAID3_3S.

== Step 2: Create the zvols ==

{{Navigation|Storage Management → Storage Volumes → Create ''(toolbar)''}}

Select the reserved pool as the '''Storage Pool''' on the '''General Settings''' tab, then set a '''Size''' for each zvol. Each zvol becomes one OSD, so the size you choose here is the OSD's capacity.

On the '''Advanced Settings''' tab:

* '''Use Case''' shows '''Ceph OSD''' and is locked, because the pool forces its use case onto every volume it holds.
* '''% Reserved''' defaults to '''100%'''. A zvol in a Ceph OSD pool is fully reserved by default because if the pool underneath runs out of space, every OSD on it starts failing writes at once. You can still lower it deliberately.
* '''Block Size''' sets the zvol's block size. When the OSD is created, QuantaStor sets Bluestore's minimum allocation size to match it, within a range of 4K to 64K.
* '''Batch Create''' creates several zvols of the same size in one step.

From the CLI, use <code>[[QuantaStor CLI Command Reference#volume-create|qs volume-create]]</code>. The CLI default for <code>--percent-reserved</code> is 0, so pass 100 explicitly to match the web interface:

<pre style="font-size: smaller">
qs volume-create --name=osd-vol --size=4T --pool=osd-pool-1 --count=6 --percent-reserved=100
</pre>

=== A single OSD zvol in a Default pool ===

You can also reserve an individual zvol in a Default pool: set '''Use Case''' to '''Ceph OSD''' on the '''Advanced Settings''' tab of Create Storage Volume, or pass <code>--volume-use-case=ceph-zvol-osd</code> to <code>qs volume-create</code>. Only that zvol is set aside for Ceph; the pool's other volumes stay available to hosts as usual. On a Ceph block pool there is nothing to choose, and the field stays at Default.

== Step 3: Create the OSDs ==

{{Navigation|Scale-out Storage Configuration → Data & Journal Devices → Create OSDs & Journals ''(toolbar)'' → Logical Devices ''(tab)''}}

The Create OSDs dialog has two tabs of available devices: '''Physical Devices''' for unused disks and '''Logical Devices''' for zvols. The Logical Devices tab lists every zvol that has the Ceph OSD use case, is healthy, is not a snapshot and does not already back an OSD. Move the zvols you want into '''Data/OSD Devices (HDDs or SSDs)''' and click '''OK'''. You can combine zvols and physical disks in one create. See [[Create Ceph OSD]] for the journal and optimization options in the rest of the dialog.

If a node has no unused physical disks but does have eligible zvols, the dialog still opens; it closes with ''No unused devices were found to create new OSDs or Journal Groups with'' only when neither tab has anything to offer.

From the CLI, use <code>[[QuantaStor CLI Command Reference#ceph-osd-multi-create|qs ceph-osd-multi-create]]</code> with <code>--volume-list</code>, which you can combine with <code>--physical-disk-list</code>:

<pre style="font-size: smaller">
qs ceph-osd-multi-create --ceph-cluster=<cluster> --volume-list=osd-vol-1,osd-vol-2,osd-vol-3
</pre>

=== How QuantaStor treats a zvol-backed OSD ===

* A zvol is new and has no partition table to clear, so QuantaStor skips the quick-format it runs on physical disks.
* For WAL/DB placement, a zvol counts as spinning media, because it is only as fast as the HDD pool behind it. Journal Group devices on SSD therefore apply to it as they do to an HDD.
* Once the OSD is up, QuantaStor renames its zvol to {{Code|1=vol-<system name>-osd.<n>}}, so the volume and the OSD it carries line up in the web interface and the CLI. If the rename fails, the OSD is unaffected and the zvol keeps its old name.

== Operating notes ==

=== Boot order ===

The OSDs cannot start until their pool is imported. At boot, QuantaStor imports every pool reserved for Ceph OSDs, and every Default pool that holds a Ceph OSD zvol, before the Ceph OSD services start. A pool that fails to import does not stop the rest of the boot; Ceph reports the OSDs on it as down.

Two cases are skipped:

* A pool with auto-start disabled is not imported, and the OSDs on it do not start.
* A pool managed by a Storage Pool HA failover group is left to the HA manager rather than imported at boot.

=== LVM visibility ===

Ceph builds each OSD as an LVM volume on its device, so LVM must be able to see the zvol. By default, the {{Code|1=global_filter}} in {{Code|1=/etc/lvm/lvm.conf}} hides every zvol from LVM so the appliance never scans a volume handed out to a client. QuantaStor adds accept rules for Ceph OSD zvols only: one rule for each pool reserved for Ceph OSDs, and one per zvol for a Ceph OSD zvol in a Default pool, so that pool's other zvols stay hidden. It updates the rules when a pool is created or started, including on another node after an HA failover, and when a Ceph OSD zvol is created or deleted. Other entries in the filter are left alone.

=== Missing OSDs ===

If the pool behind an OSD is not imported, the OSD shows as missing. QuantaStor keeps the OSD's link to its zvol, so it reconnects the OSD to the zvol once the pool is imported again.

== Related pages ==

* [[Storage Pools]]
* [[Storage Volumes]]
* [[Create Ceph OSD]]
* [[Delete Ceph OSD]]
* [[Scale-out Block Setup (ceph)]]
* [[Scale-out Object Setup (ceph)]]
* [[Designing Systems]]
* [[QuantaStor CLI Command Reference]]

----
<small>''Verified against QuantaStor 6.9.0.''</small>

+ Admin Guide Overview

Qadmin — Thu, 08 Oct 2026 00:00:04 GMT

osn-seo-utilities: list Ceph over ZFS under Scale-out Cluster Configuration (docs-ceph-over-zfs-zfs-draid-pool-reserved)

← Older revision		Revision as of 00:00, 8 October 2026
Line 146:		Line 146:
	\|-		\|-
	\| [[Scale-out Object Setup (ceph)]] \|\| Scale-out object storage over the S3-compatible REST protocols.		\| [[Scale-out Object Setup (ceph)]] \|\| Scale-out object storage over the S3-compatible REST protocols.
			\|-
			\| [[Ceph over ZFS]] \|\| Reserve a ZFS (DRAID) storage pool as the backing store for Ceph OSDs, with zvols used as the OSD devices.
	\|}		\|}

File:Docs-docs-ceph-over-zfs-zfs-draid-pool-reserved-pool-use-case.png

Qadmin — Thu, 08 Oct 2026 00:00:04 GMT

Qadmin uploaded File:Docs-docs-ceph-over-zfs-zfs-draid-pool-reserved-pool-use-case.png osn-seo-utilities: screenshot for Ceph over ZFS (docs-ceph-over-zfs-zfs-draid-pool-reserved)

New page

osn-seo-utilities: screenshot for Ceph over ZFS (docs-ceph-over-zfs-zfs-draid-pool-reserved)

Guides:Index

Qadmin — Wed, 07 Oct 2026 20:00:23 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 20:00, 7 October 2026
Line 5:		Line 5:
	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
			* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''
			* '''[[High-availability VIF Management\|High-availability VIF Management]]''' — Cluster virtual interfaces (VIFs) are floating IP addresses that move between the appliances of a site cluster so that a service address stays reachable when the appliance hosting it fails.
	* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.		* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.
	* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.		* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.

High-availability VIF Management

Qadmin — Wed, 07 Oct 2026 20:00:19 GMT

osn-seo-utilities: docs-active-directory-spn-failover-with-the-ha @ b83457fda786 (approved in the portal)

Show changes

HA Cluster Setup (JBODs)

Qadmin — Wed, 07 Oct 2026 20:00:14 GMT

osn-seo-utilities: docs-manual-planned-pool-failover @ d066481c2606 (approved in the portal)

Show changes

Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing

Qadmin — Wed, 07 Oct 2026 20:00:04 GMT

osn-seo-utilities: fc-iscsi-alua @ f86c646a4cec (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 7, 2026''

QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access). On a high-availability storage pool, the node that owns the pool serves a volume's active paths, and the partner node serves standby paths to the same logical unit, so host multipath software always knows which paths to use and switches over cleanly when the pool fails over. This guide explains how those path states behave through a failover, how to design and set up FC and iSCSI access, and what to configure on Linux, VMware and Windows hosts, including Hyper-V failover clusters with Cluster Shared Volumes.

== Why it matters ==

Most virtualization and database estates still run on block storage, and most of those hosts expect two things from the array: more than one path to every LUN, and an honest answer about which of those paths are optimized. Without ALUA, a host either sends I/O down paths that cannot serve it or waits for timeouts to learn that a path is dead. With ALUA, the array tells the host which paths are active and which are standby, and the host's multipath driver (dm-multipath with <code><nowiki>scsi_dh_alua</nowiki></code> on Linux, the in-box Microsoft DSM on Windows, the native multipathing plug-in on VMware ESXi) follows those states.

Clustered hosts add a second requirement. A Windows Server Failover Cluster, Hyper-V Cluster Shared Volumes and other shared-disk clusters arbitrate ownership with SCSI-3 persistent reservations, and those reservations must stay valid when the storage itself fails over from one controller to the other. QuantaStor covers both: ALUA path states for every FC and iSCSI LUN on an HA pool, and clustered SCSI-3 persistent reservations that survive an HA failover.

== How it works ==

=== Fibre Channel target ports ===

QuantaStor's fabric manager scans the FC transport about once a minute and creates an FC target port object for each QLogic port it finds. Any port that is target-mode capable is switched into target mode automatically unless you have explicitly set it to initiator mode, so a dedicated storage appliance needs no per-port configuration. The ports appear in the FC Ports tab of each Storage System, with State, Link State, Status and Mode columns; <code><nowiki>Link Up - F_Port</nowiki></code> in the Link State column means the port negotiated a point-to-point link to a fabric switch, which is what you want. [[Fibre Channel Target Port Management]] covers each column, target and initiator mode, LIPs and the CLI.

Zoning is your job on the switch: QuantaStor cannot configure it. Zone each host initiator WWPN with the target WWPNs on both nodes of the HA pair, so every host sees paths through both controllers.

=== Active and standby paths on an HA pool ===

Each volume on an HA pool is one logical unit with one identity, visible through the target ports of both nodes:

{| class="wikitable"
! Node !! Device behind the LUN !! ALUA state reported !! What the host does
|-
| Pool owner || The volume itself || Active || Sends I/O on these paths
|-
| Partner node || A standby placeholder device with the same identity || Standby || Holds the paths in reserve
|}

The standby device advertises the same thin-provisioning support, UNMAP limits and block geometry as the volume, so a host sees one consistent logical unit no matter which node's path it probes first. That matters for hosts that cache those properties per LUN, such as Windows trim and alignment handling, VMware VAAI UNMAP and dm-multipath discard limits.

During an HA failover the old owner moves its LUNs onto standby devices rather than removing them, and the new owner switches its target group to active only once every LUN is backed by the real volume. Hosts keep the same set of paths throughout; only the states change. A volume's relative target ID also stays the same across failovers, so hosts that track LUNs by target port identity keep their mapping.

After a node restarts, its FC target ports are enabled only once every exported volume is mapped and its ALUA state is in place. When a former pool owner reboots and comes back as the partner, its ports open as soon as its standby devices are ready. Initiators therefore never log in to a port that has no LUNs or unsettled path states behind it. Deleting an HA group removes the partner node's standby devices, ALUA device groups and FC LUN maps, so hosts do not keep standby paths to a node that can no longer serve the pool.

=== Clustered SCSI-3 persistent reservations ===

With clustered reservations enabled on the HA group, both nodes share reservation state through DLM (the Linux distributed lock manager) integrated with the SCST target driver. A reservation a cluster member takes through one node is honored on the other, and it survives a failover in either direction. Volumes enter clustered reservation mode only once they are assigned to a host, so unassigned volumes use no cluster locking resources. See [[Clustered SCSI-3 Persistent Reservations]].

== Design and sizing ==

* '''Pool layout.''' Put block volumes on a storage pool in an HA failover group ([[Create Storage Pool High-Availability Group]]). Only HA pools have a partner node to serve standby paths.
* '''Paths.''' Give each host at least two initiator ports, each zoned (FC) or routed (iSCSI) to both nodes. Two host ports and two target ports per node gives four paths per LUN to each node: active through the owner, standby through the partner.
* '''Port roles.''' Keep the appliance's FC ports in target mode. Use dual mode (<code><nowiki>qs-util enabledualmode</nowiki></code>, then a reboot) only when the appliance must also consume FC storage from another array; [[Dual-mode FC configuration]] describes the procedure.
* '''iSCSI addressing.''' Point iSCSI discovery at an HA virtual interface, which moves with the pool ([[High-availability VIF Management]]).
* '''Block size.''' Choose the volume block size at creation time; it cannot be changed later. For VMware VMFS datastores use 8K to 64K, with 64K recommended for general virtualization workloads and 8K worth considering for databases ([[VMware Configuration]], [[Storage Volumes]]).
* '''Clustered hosts.''' Enable clustered SCSI-3 reservations on any HA group that serves Hyper-V CSVs, WSFC disks or other shared-disk clusters.

== Setting it up ==

# '''Check the FC ports.''' In Storage Management, select a Storage System and open the FC Ports tab. Every port you have cabled should show Mode <code><nowiki>Target</nowiki></code>, Status <code><nowiki>Online</nowiki></code> and a <code><nowiki>Link Up - F_Port</nowiki></code> link state. Ideally you'll see the ports in dual Target+Initiator mode but if they are in just Initiator mode they'll need to be transitioned to Target mode before they can be used to present Storage Volumes as FC LUNs from QuantaStor.

# '''Create the HA group''' for the pool, choose the primary and secondary systems, and set '''Clustered SCSI3-PR Reservations''' if the pool will serve a host cluster. Enabling or disabling it later from Modify Group brings every device of the pool into line on both nodes ([[Modify Storage Pool High-Availability Group]]).

[[File:Guide-fc-iscsi-alua-fc-iscsi-alua-ha-group-pr.png|frame|center|The Modify Storage Pool High-Availability Group dialog: the Clustered SCSI3-PR Reservations setting sits below the primary and secondary system choices]]

# '''Register each host.''' In Hosts & Portal Groups, choose Add, set the operating system type, and enter the host's iSCSI IQN or FC WWPN. For FC, Select Initiator lets you pick the WWPN from a list instead of typing it. Group the members of a host cluster in a host group so they get identical assignments ([[Hosts and Host Groups]]).

[[File:Guide-fc-iscsi-alua-fc-iscsi-alua-add-host.png|frame|center|The Add Host dialog: one host entry can carry an iSCSI IQN, an FC WWPN or an NVMe-oF NQN as its initiator]]

# '''Assign volumes''' to the host or host group. An assignment maps the volume as a LUN through the target ports of both nodes, active on the owner and standby on the partner.

[[File:Guide-fc-iscsi-alua-fc-iscsi-alua-hosts.png|frame|center|The Hosts view: each registered host lists its initiator identifiers on the left and its storage volume assignments on the right]]

# '''Connect the hosts.'''
** ''Linux:'' set up the iSCSI initiator ([[ISCSI Initiator Setup]]) or rescan the FC HBAs, then configure dm-multipath with ALUA path grouping ([[Multipath IO Configuration]]).
** ''VMware ESXi:'' add the software iSCSI adapter, add the HA virtual interface address under Dynamic Discovery, and rescan storage. The volume appears as an "OSNEXUS iSCSI Disk"; match it to the EUI shown in the volume's properties in QuantaStor, then create the VMFS datastore.
** ''Windows and Hyper-V:'' enable MPIO with the in-box Microsoft DSM, claim the QuantaStor disks, bring them online on one node, and add them to the failover cluster as Cluster Shared Volumes.

== Operating and testing ==

Test the paths before production. On a Linux host, <code><nowiki>multipath -ll</nowiki></code> should show two path groups per LUN: one whose paths go to the pool owner as active, and one whose paths go to the partner as standby (Windows MPIO and the ESXi path list show the same split). Then trigger a manual failover from Storage Pool HA Groups with Manual Failover ([[Storage Pool Ha Failover Group Failover]]) and watch the two groups swap states while I/O continues.

For clustered hosts, test failover under real load. QuantaStor has been validated with two-host Windows Server 2022 Hyper-V failover clusters running Cluster Shared Volumes with SCSI-3 persistent reservations on FC HA pools, through manual and automatic failovers, live migration during a failover, and host drain and reboot.

If a LUN does not appear on a host, check in this order: zoning on the switch, the FC port's Link State, the host's initiator entry in QuantaStor, and the volume assignment. Issuing a LIP from the FC Ports tab forces the fabric to rediscover the ports.

== FAQ ==

=== Which software-defined storage platforms support Fibre Channel and iSCSI with ALUA multipathing? ===

QuantaStor does. It presents volumes over FC through QLogic HBAs in target mode and over iSCSI, and it reports active and standby ALUA path states for every LUN on an HA storage pool. Hosts use their standard multipath drivers (dm-multipath, Windows MPIO with the Microsoft DSM, VMware native multipathing) with no vendor plug-in.

=== What storage can I use for a Hyper-V failover cluster with Cluster Shared Volumes? ===

Any FC or iSCSI LUN that supports SCSI-3 persistent reservations and keeps them through a controller failover. On QuantaStor that is a volume on an HA pool whose HA group has clustered SCSI-3 reservations enabled. Two-host Windows Server 2022 Hyper-V clusters with CSVs have been validated on FC HA pools, including live migration during a failover.

=== What happens to host paths during an HA failover? ===

Nothing is removed. The old owner's paths change from active to standby, the new owner's paths change from standby to active once every LUN is backed by the volume, and the host's multipath driver moves I/O accordingly. Persistent reservations held by clustered hosts stay in place.

=== Do I need to enable target mode on each FC port? ===

No. Any target-capable QLogic port without an explicit initiator-mode setting is put into target mode automatically within about a minute. Use the Enable FC Initiator Mode action only for ports you want to use as initiators.

=== Can I change a volume's block size after creating it? ===

No. The block size is fixed when the volume is created. To change it, create a new volume with the right size and move the data, for example by live-migrating VMs to a new VMware datastore. You can grow the size of the Storage Volume at any time but the underlying block size is fixed at creation, 64K by default.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-fc-iscsi-alua-fc-iscsi-alua-hosts.png

Qadmin — Wed, 07 Oct 2026 20:00:04 GMT

Qadmin uploaded File:Guide-fc-iscsi-alua-fc-iscsi-alua-hosts.png osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

New page

osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

File:Guide-fc-iscsi-alua-fc-iscsi-alua-add-host.png

Qadmin — Wed, 07 Oct 2026 20:00:03 GMT

Qadmin uploaded File:Guide-fc-iscsi-alua-fc-iscsi-alua-add-host.png osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

New page

osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

File:Guide-fc-iscsi-alua-fc-iscsi-alua-ha-group-pr.png

Qadmin — Wed, 07 Oct 2026 20:00:03 GMT

Qadmin uploaded File:Guide-fc-iscsi-alua-fc-iscsi-alua-ha-group-pr.png osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

New page

osn-seo-utilities: screenshot for Guides (fc-iscsi-alua)

Guides:Index

Qadmin — Wed, 07 Oct 2026 05:45:21 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 05:45, 7 October 2026
Line 4:		Line 4:

	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
			* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.
	* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.		* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.
	* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]		* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
			* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
			* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

Network Ports

Qadmin — Wed, 07 Oct 2026 05:45:18 GMT

osn-seo-utilities: docs-arp-flux-protection-for-multi-homed-ha @ d54c6794ab44 (approved in the portal)

Show changes

File:Docs-arp-flux-protection-for-multi-homed-ha-arp-filtering.png

Qadmin — Wed, 07 Oct 2026 05:45:18 GMT

Qadmin uploaded File:Docs-arp-flux-protection-for-multi-homed-ha-arp-filtering.png osn-seo-utilities: screenshot for Network Ports (docs-arp-flux-protection-for-multi-homed-ha)

New page

osn-seo-utilities: screenshot for Network Ports (docs-arp-flux-protection-for-multi-homed-ha)

+ Admin Guide Overview

Qadmin — Wed, 07 Oct 2026 05:45:12 GMT

osn-seo-utilities: list ISCSI Target Configuration under Storage Provisioning (docs-iscsi-target-presentation-of-storage-volumes-t)

← Older revision		Revision as of 05:45, 7 October 2026
Line 78:		Line 78:
	\|-		\|-
	\| [[Cloud Containers / NAS Gateway]] \|\| Mapping an object-storage bucket to a Network Share so it can be accessed as files.		\| [[Cloud Containers / NAS Gateway]] \|\| Mapping an object-storage bucket to a Network Share so it can be accessed as files.
			\|-
			\| [[ISCSI Target Configuration]] \|\| How QuantaStor presents storage volumes over iSCSI - target IQNs, portals and network ports, discovery, and what an initiator sees after a volume is assigned to a host.
	\|}		\|}

ISCSI Target Configuration

Qadmin — Wed, 07 Oct 2026 05:45:12 GMT

osn-seo-utilities: docs-iscsi-target-presentation-of-storage-volumes-t @ a4968c7b29ee (approved in the portal)

New page

QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI. This page covers how the target IQN is formed, which addresses a target listens on, how assigning a volume to a host opens the target to that host's initiator, how CHAP is applied, and what the initiator sees once it logs in.

__TOC__

{| class="wikitable"
! Section !! Purpose
|-
| [[#One target per Storage Volume|One target per Storage Volume]] || The target IQN format and why every iSCSI volume is LUN 0
|-
| [[#Portals: which network ports serve iSCSI|Portals: which network ports serve iSCSI]] || The iSCSI Portal setting on a network port, TCP port 3260, HA VIFs and Portal Groups
|-
| [[#Hosts and initiator IQNs|Hosts and initiator IQNs]] || Registering a client by its initiator IQN
|-
| [[#Assigning a Storage Volume to a host|Assigning a Storage Volume to a host]] || What an assignment programs on the target
|-
| [[#Discovery and login|Discovery and login]] || What a client needs to find and log in to a target
|-
| [[#CHAP authentication|CHAP authentication]] || Per-volume, per-user and Resource Group CHAP
|-
| [[#What the initiator sees|What the initiator sees]] || Vendor and product strings, device identifiers
|-
| [[#iSCSI sessions|iSCSI sessions]] || Viewing and dropping active sessions
|-
| [[#Troubleshooting|Troubleshooting]] || A volume that does not appear on the client
|-
| [[#CLI reference|CLI reference]] || The commands used on this page
|}

== One target per Storage Volume ==

QuantaStor uses the SCST iSCSI target. Every Storage Volume gets its own iSCSI target, and assigning the volume to more hosts adds initiators to that same target rather than creating new ones. Because each volume has a unique IQN, the volume is always presented at '''LUN 0''' of its target. The LUN number you can set on a Storage Volume, labelled FC LUN, applies only to Fibre Channel; see [[Fibre Channel Target Port Management]].

A target IQN has three parts after the fixed prefix:

<pre style="font-size: smaller">
iqn.2009-10.com.osnexus:9c30f734-0d4edacd5d071f74:volume-vmwaretest.001
| | |
| | +-- Storage Volume name (underscores become dots)
| +-- first 16 hex digits of the Storage Volume ID
+-- first 8 hex digits of the Storage Pool ID
</pre>

The pool part is what ties a target to its pool. QuantaStor uses it to decide which portals a target is offered on, which is how a volume in an HA pool follows the pool's virtual interface (see below). If a volume is in a namespace, the namespace is inserted before the pool part.

The IQN is shown in the IQN row of the Storage Volume's Properties panel and in the Target IQN/WWN column of the Storage Volumes grid. It is also returned by <code>[[QuantaStor CLI Command Reference#volume-get|qs volume-get]] --volume=<volume></code>.

The IQN format is controlled by the {{Code|1=[iqn]}} section of {{Code|1=/etc/quantastor.conf}}. Leave it at the defaults: initiators that are already logged in identify the volume by its IQN, so a change affects every existing client.

== Portals: which network ports serve iSCSI ==

A portal is an IP address and TCP port on which a target accepts logins. QuantaStor's iSCSI target listens on TCP port '''3260'''. If a firewall sits between the clients and the appliance, it must allow that port; see [[Firewall Configuration]].

=== The iSCSI Portal setting on a network port ===

{{Navigation|Storage Management → Storage Systems ''(section)'' → ''select a Storage System'' → Network Ports ''(tab)'' → ''select a port'' ''(select + right-click)'' → Modify Network Port}}

Each network port has an '''iSCSI Portal''' checkbox in the Modify Network Port dialog. The dialog's tooltip describes it: ''by checking this option this interface becomes an allowed portal for the protocol.'' A target is offered only on ports that have iSCSI enabled and a valid IP address. Clear the checkbox on management or replication ports to keep block traffic off them. The checkbox is not available on the grid management virtual interface. [[Network Ports]] covers the rest of the dialog.

From the CLI, use <code>[[QuantaStor CLI Command Reference#network-port-modify|qs network-port-modify]] --port=<port> --iscsi-enable=true</code>. The Properties panel for a port shows the current setting in the iSCSI Enabled? row.

If no port on the appliance has iSCSI enabled, QuantaStor binds the targets to the loopback address only, which blocks all external iSCSI access.

=== HA pools and virtual interfaces ===

For a volume in a high-availability Storage Pool, QuantaStor offers the target '''only''' on the pool's HA virtual interfaces (VIFs), not on the appliance's physical port addresses. The VIF moves with the pool on failover, so a client that logged in through it reconnects to the node that now owns the pool. Point initiators at the VIF address, never at a node's own IP. Configure the VIF in [[High-availability VIF Management]], and enable iSCSI on it when you create it.

Volumes in a scale-out (Ceph) pool are offered on the site cluster VIFs by default; see [[Cluster VIFs]].

=== Portal Groups ===

A Portal Group restricts a set of Storage Volumes to a specific set of cluster VIFs and FC ports, for example to keep one group of clients on one storage network. Portal Groups live in the Hosts & Portal Groups section. A Portal Group belongs to one Storage Pool, and a Storage Volume can be in at most one Portal Group. Create one with <code>[[QuantaStor CLI Command Reference#portal-group-create|qs portal-group-create]] --name=<name> --pool=<pool></code>, or list the portals a pool can use with <code>[[QuantaStor CLI Command Reference#pool-portals-list|qs pool-portals-list]] --pool=<pool></code>.

Resource Group network settings narrow the allowed portals further for volumes that belong to a tenant; see [[Create Resource Group]].

== Hosts and initiator IQNs ==

[[File:Docs-iscsi-target-host-add.png|thumb|right|600px|The Add Host dialog. Choose iSCSI Initiator (IQN) and paste the client's initiator IQN.]]

{{Navigation|Storage Management → Hosts & Portal Groups ''(section)'' → Host ''(toolbar group)'' → Add ''(toolbar)''}}

QuantaStor grants access by initiator, so each client must exist as a host with its initiator IQN. Copy the IQN from the client: on Linux it is in {{Code|1=/etc/iscsi/initiatorname.iscsi}}; on VMware ESXi it is shown on the software iSCSI adapter (see [[VMware Configuration]]); on Windows it is on the Configuration tab of the iSCSI Initiator.

The Add Host dialog has these fields:

{| class="wikitable"
! Field !! Notes
|-
| '''Host Name''' || A name for the host in QuantaStor. It does not have to match the client's hostname. Letters, digits and {{Code|1=- _ .}} only.
|-
| '''Description''' || Optional.
|-
| '''Operating System Type''' || Windows, Mac OS X, Linux, Solaris, AIX, HP-UX, VMware, XenServer or Other. Windows is the default.
|-
| '''Initiator''' || One of '''iSCSI Initiator (IQN)''', '''FC Initiator WWPN''' or '''NVMeoF Initiator (NQN)'''. For iSCSI, select the first option and enter the IQN, for example {{Code|1=iqn.1991-05.com.example:iscsihost-03fo1500}}.
|}

The dialog takes one initiator. A client with several initiators, or a cluster node that should be reachable under one host entry, gets the others through '''Add Initiator''' on the Host toolbar group, or <code>[[QuantaStor CLI Command Reference#host-initiator-add|qs host-initiator-add]] --host=<host> --iqn=<iqn></code>. Only entries that begin with {{Code|1=iqn.}} are used for iSCSI access; WWPNs and NQNs are used by the FC and NVMe-oF targets.

To give a group of clients, such as the nodes of a hypervisor cluster, the same volumes, put the hosts in a Host Group and assign volumes to the group. [[Hosts and Host Groups]] covers hosts, host groups and their dialogs in full.

The CLI equivalent of the dialog is <code>[[QuantaStor CLI Command Reference#host-add|qs host-add]] --hostname=<name> --iqn=<iqn> --host-type=linux</code>.

== Assigning a Storage Volume to a host ==

[[File:Docs-iscsi-target-volume-assign.png|thumb|right|438px|The Assign/Unassign Storage Volume dialog. Tick the hosts, or switch to the Host Groups tab, then click OK.]]

{{Navigation|Storage Management → Storage Volumes ''(section)'' → ''select a Storage Volume'' → Assign ''(toolbar)''}}

The same dialog is on the right-click menu of a Storage Volume as '''Assign/Unassign Host Access...'''. To work from the other direction, select a host and use '''Assign''' on the Host toolbar group, or '''Assign Volumes...''' on its right-click menu, to choose several volumes for one host.

The dialog lists every host and host group. Ticked entries have access to the volume; clearing a tick removes it. The '''Release/free unused storage volume LUN number(s)''' checkbox concerns FC LUN numbers only and has no effect on iSCSI, where the LUN is always 0.

When you click OK, QuantaStor updates the volume's target:

# The target gets an access list (an SCST initiator group) containing the iSCSI IQNs of every assigned host and every host in an assigned host group. New initiators are added before removed ones are taken out.
# The volume's CHAP settings are applied to the target.
# The target is enabled, on the allowed portals described above.

'''A target is enabled only while it has at least one initiator.''' An unassigned volume, or one assigned only to hosts that have no iSCSI IQN, has no active iSCSI target, and no client can log in to it.

From the CLI: <code>[[QuantaStor CLI Command Reference#volume-assign|qs volume-assign]] --volume=<volume> --host-list=<host></code>, reversed with <code>[[QuantaStor CLI Command Reference#volume-unassign|qs volume-unassign]]</code>. <code>[[QuantaStor CLI Command Reference#volume-assign-list|qs volume-assign-list]] --host=<host></code> lists what a host can reach.

== Discovery and login ==

QuantaStor does not configure iSNS. Clients find targets by SendTargets discovery against a portal address:

* For a volume in a standalone pool, use the IP address of a network port with iSCSI enabled.
* For a volume in an HA pool, use the pool's HA VIF address.
* For a volume in a scale-out pool, use a site cluster VIF address.

On a Linux client with open-iscsi, discover and log in like this:

<pre style="font-size: smaller">
iscsiadm -m discovery -t sendtargets -p 10.0.20.50:3260
iscsiadm -m node -T iqn.2009-10.com.osnexus:9c30f734-0d4edacd5d071f74:vol1 -p 10.0.20.50:3260 --login
</pre>

A client discovers a target only on the portals that the target is allowed on. To reach a volume over several networks for multipathing, enable iSCSI on a port in each network and log in once per portal; see [[Multipath Configuration]]. Client-side setup for Linux and Windows is covered in [[ISCSI Initiator Setup]], and for ESXi in [[VMware Configuration]].

== CHAP authentication ==

{{Navigation|Storage Management → Storage Volumes ''(section)'' → ''select a Storage Volume'' → Modify ''(toolbar)'' → Security Settings ''(tab)''}}

CHAP is set per Storage Volume, so it applies to the volume's target and to every host assigned to it. Hosts have no CHAP settings. The Security Settings tab of the Storage Volume Create and Modify dialogs has a '''CHAP Authentication & Policy Settings''' group with three options:

{| class="wikitable"
! Option !! Credentials used
|-
| '''Use User's default CHAP user/pass if available''' || The default CHAP username and password of the QuantaStor user who owns the volume.
|-
| '''Use Resource Group default CHAP user/pass''' || The CHAP credentials of the volume's Resource Group, for multi-tenant setups.
|-
| '''Use Volume specific CHAP user/pass specified below''' || The '''CHAP Username''' and '''CHAP Password''' entered in the dialog.
|}

The username may not contain spaces. The password must be 12 to 16 alphanumeric characters. CHAP is disabled by default. Once it is enabled, the client must be configured with the same username and password before it can log in, so set the client first or expect existing sessions to fail at their next login.

From the CLI, use <code>[[QuantaStor CLI Command Reference#volume-modify|qs volume-modify]] --volume=<volume> --chap-policy=target --chap-user=<user> --chap-pass=<password></code>. The other {{Code|1=--chap-policy}} values are {{Code|1=user-defaults}}, {{Code|1=cloud-defaults}} (the Resource Group credentials) and {{Code|1=disabled}}.

QuantaStor configures one-way CHAP, where the target authenticates the initiator. Mutual CHAP is off.

== What the initiator sees ==

After login, the client sees one disk per target at LUN 0, with:

{| class="wikitable"
! Property !! Value
|-
| SCSI vendor || {{Code|1=OSNEXUS}}
|-
| SCSI product || {{Code|1=QUANTASTOR}}
|-
| Device identification || An NAA type 6 identifier, {{Code|1=62000000}} followed by a descriptor built from the Storage Volume ID and the Storage System ID, plus a T10 vendor ID and a unit serial number built from the same descriptor
|-
| Size || The Storage Volume's size. A resize is visible after the client rescans the device.
|}

The identifiers come from the Storage Volume's ID rather than its name, and they are the same on every portal the target is offered on. That is what lets a multipath driver recognise the paths through each portal as one device. The Properties panel of a Storage Volume also shows a VMware EUI identifier, which is how [[VMware Configuration]] matches a datastore device to its volume.

Per-volume iSCSI tuning, such as burst lengths and queued commands, comes from the volume's IO profile rather than from appliance-wide settings; see [[Storage Volumes]].

== iSCSI sessions ==

{{Navigation|Storage Management → Storage Volumes ''(section)'' → ''select a Storage Volume'' → Sessions ''(tab)''}}

The Sessions tab below the Storage Volumes grid lists the active iSCSI sessions for the selected volume: Storage System, Session, State, Storage Volume, Initiator IQN/WWN, Initiator IP, Target IQN/WWN, Reads, Writes and Created. It is the quickest way to confirm that a client actually logged in, and from which address.

To end a session, right-click it and choose '''Drop Session'''; see [[Session Drop]]. The CLI help recommends removing the host's assignment instead, because dropping a session does not take away the host's access.

From the CLI: <code>[[QuantaStor CLI Command Reference#volume-session-list|qs volume-session-list]] --volume=<volume></code> and <code>[[QuantaStor CLI Command Reference#volume-session-close|qs volume-session-close]] --volume=<volume> --session-list=<session></code>.

== Troubleshooting ==

'''Discovery returns no targets.'''
* Check that the volume is assigned to a host, and that the host entry has the client's exact initiator IQN. A typo in the IQN leaves the target with no matching initiator.
* Check that the address you discover against is on a port with iSCSI Portal enabled, or, for an HA pool, is the pool's VIF.
* Check that TCP port 3260 is open between the client and the appliance.

'''Discovery works but login fails.'''
* CHAP mismatch: compare the volume's CHAP settings with the client's.
* The client is using a node IP for a volume in an HA pool. Use the VIF.

'''The client logs in but sees no disk.'''
* Rescan the client's iSCSI adapter. On ESXi, rescan storage after every new assignment.

'''A volume disappears from a client.'''
* Check whether the assignment was removed, the host's initiator IQN changed, or the port's iSCSI Portal setting cleared.

== CLI reference ==

{| class="wikitable"
! Command !! Purpose
|-
| <code>[[QuantaStor CLI Command Reference#host-add|qs host-add]]</code> || Add a host with its initiator IQN
|-
| <code>[[QuantaStor CLI Command Reference#host-initiator-add|qs host-initiator-add]]</code> || Add another initiator IQN to a host
|-
| <code>[[QuantaStor CLI Command Reference#volume-assign|qs volume-assign]]</code> || Give hosts access to a Storage Volume
|-
| <code>[[QuantaStor CLI Command Reference#volume-unassign|qs volume-unassign]]</code> || Remove access
|-
| <code>[[QuantaStor CLI Command Reference#volume-assign-list|qs volume-assign-list]]</code> || List volume-to-host assignments
|-
| <code>[[QuantaStor CLI Command Reference#volume-modify|qs volume-modify]]</code> || Set the CHAP policy and credentials
|-
| <code>[[QuantaStor CLI Command Reference#network-port-modify|qs network-port-modify]]</code> || Enable or disable iSCSI on a port
|-
| <code>[[QuantaStor CLI Command Reference#portal-group-create|qs portal-group-create]]</code> || Restrict volumes to a set of VIFs and FC ports
|-
| <code>[[QuantaStor CLI Command Reference#volume-session-list|qs volume-session-list]]</code> || List active iSCSI sessions
|-
| <code>[[QuantaStor CLI Command Reference#volume-session-close|qs volume-session-close]]</code> || Close an iSCSI session
|}

== Related pages ==

* [[Hosts and Host Groups]]
* [[Storage Volumes]]
* [[Network Ports]]
* [[High-availability VIF Management]]
* [[ISCSI Initiator Setup]]
* [[Multipath Configuration]]
* [[VMware Configuration]]
* [[Fibre Channel Target Port Management]]
* [[NVMe-oF Target Configuration]]
* [[QuantaStor CLI Command Reference]]

[[Category:admin_guide]]

----
<small>''Verified against QuantaStor 6.9.0.''</small>

File:Docs-iscsi-target-volume-assign.png

Qadmin — Wed, 07 Oct 2026 05:45:12 GMT

Qadmin uploaded File:Docs-iscsi-target-volume-assign.png osn-seo-utilities: screenshot for ISCSI Target Configuration (docs-iscsi-target-presentation-of-storage-volumes-t)

New page

osn-seo-utilities: screenshot for ISCSI Target Configuration (docs-iscsi-target-presentation-of-storage-volumes-t)

File:Docs-iscsi-target-host-add.png

Qadmin — Wed, 07 Oct 2026 05:45:12 GMT

Qadmin uploaded File:Docs-iscsi-target-host-add.png osn-seo-utilities: screenshot for ISCSI Target Configuration (docs-iscsi-target-presentation-of-storage-volumes-t)

New page

osn-seo-utilities: screenshot for ISCSI Target Configuration (docs-iscsi-target-presentation-of-storage-volumes-t)

Guides:Nightly Database Analysis from iSCSI Snapshots

Qadmin — Wed, 07 Oct 2026 05:45:08 GMT

osn-seo-utilities: db-analysis-iscsi-snapshot @ cf7d5275edd5 (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 3, 2026''

Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load. QuantaStor makes that copy with a snapshot schedule that snapshots every database volume in a pool at the same instant, presents those snapshots to an analysis server over iSCSI, and rotates old copies out on its own. This guide builds the whole nightly loop with the <code><nowiki>qs</nowiki></code> CLI and a short bash script that triggers the schedule, waits for the new snapshots, swaps the old mounts for new ones and verifies them.

== Why it matters ==

Nightly reporting, data-quality checks and ad-hoc analysis all compete with production I/O when they run against the live database. Copying databases to a reporting server every night is slow and doubles the capacity bill. A snapshot avoids both: it is created near-instantly, records metadata rather than copying blocks, and consumes pool space only for blocks that change afterwards.

A snapshot schedule adds two things a hand-rolled snapshot script lacks. All ZFS members of a schedule that live in the same [[Storage Pools|storage pool]] are snapshotted by a single ZFS command, so a database spread across data, log and index volumes is captured as one crash-consistent set rather than several copies taken seconds apart. The schedule also owns rotation, so you never write cleanup code that deletes the wrong snapshot.

== How it works ==

The moving parts on the appliance side are all documented reference features:

* '''[[Snapshot Schedules]]''' take the snapshots, name them <code><nowiki><volume>_GMT<YYYYMMDD>_<HHMMSS></nowiki></code> from the UTC start time of the run, and keep only '''Max Short Term Snapshots''' of each volume in the rotating window.
* '''Triggering''' runs a schedule immediately, in addition to its timed runs. From the CLI it is <code><nowiki>qs snap-schedule-trigger</nowiki></code>, and over the [[QuantaStor REST API Reference Guide|REST API]] it is the <code><nowiki>snapshotScheduleTrigger</nowiki></code> call. Triggering a disabled schedule does nothing, so the schedule must stay enabled.
* '''Lazy cloning.''' A schedule snapshot sits in the Offline state with no iSCSI target until it is needed. Assigning it to a host materializes it: the clone is created, the target is registered and the state moves to Normal. Each volume, snapshots included, gets its own target IQN and presents at LUN 0 (see [[Storage Volumes]]).
* '''Host assignment''' is the LUN masking that lets only the analysis server see the snapshot ([[Hosts and Host Groups]]).

On the analysis server, open-iscsi logs in to the snapshot's target and the filesystem is mounted like any local disk. Because the snapshot is a writable clone (Read/Write is the default access mode for snapshots), the database engine or the filesystem can replay its journal on mount as it would after a power loss. Those writes land in the clone and never touch the production volume.

== Design and sizing ==

Put every volume of one database, or every database you want captured at the same instant, in the '''same storage pool''' and the '''same schedule'''. Atomicity holds per pool, and '''Disable Atomicity''' on the schedule's Advanced Settings tab breaks it into batches, so leave it off for this use. A schedule run also snapshots only members owned by the appliance it runs on.

The snapshots are crash-consistent. If your database needs a cleaner point, the schedule runs <code><nowiki>/var/opt/osnexus/custom/schedule-prestart.sh</nowiki></code> on the appliance at the start of every run, before any snapshot is taken, with <code><nowiki>--name=<schedule name> --id=<schedule id></nowiki></code>. It blocks the run for up to five minutes, which is enough to ask the database to checkpoint.

Retention for an analysis schedule is short. You only need the snapshot being analyzed and the one about to replace it:

{| class="wikitable"
! Setting !! Suggested value !! Reason
|-
| Max Short Term Snapshots (<code><nowiki>--max-snaps</nowiki></code>) || 3 || The mounted snapshot is never the oldest one in the window
|-
| Hourly to quarterly retention (<code><nowiki>--rc-*</nowiki></code>) || 0 || Long-term tags exempt snapshots from rotation, holding space you don't need
|-
| Schedule type || Calendar, one hour per day || The schedule must be enabled for triggers to work, so give it a harmless timed run
|-
| Offset || Different from any replication schedule on the same volumes || A member snapshotted by another schedule within the last 10 seconds is skipped
|}

Capacity cost tracks write volume, not database size: each snapshot holds the blocks overwritten since it was taken, and the mounted clone adds whatever the analysis server writes. Check '''Max Total Source Snapshots''' on the Snapshot Settings tab against pool free space before you commit.

[[File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-retention.png|frame|center|The Snapshot Settings tab: clear the long-term counts and keep Max Short Term Snapshots small, and watch the Max Total Source Snapshots figure]]

== Setting it up ==

'''1. Create the schedule.''' In the web interface, go to '''Storage Management → Schedules → Snapshot Schedule → Create''', or use the CLI. This schedule runs at 1 AM every day, keeps three snapshots per volume and no long-term copies:

<pre>qs snap-schedule-create --name=db-nightly \
--days=mon,tue,wed,thu,fri,sat,sun --hours=1am \
--volume-list=db-sales,db-finance,db-hr \
--max-snaps=3 --rc-hourly=0 --rc-dailies=0 --rc-weeklies=0 \
--rc-monthlies=0 --rc-quarterlies=0</pre>

[[File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-interval.png|frame|center|The Schedule Interval tab: tick the days and the single hour the schedule should fire on its own]]

To add another database later, use <code><nowiki>qs snap-schedule-add --schedule=db-nightly --volume-list=db-ops</nowiki></code> or the '''Update Selections''' dialog ([[Snapshot Schedule Add/Remove Volumes]]). <code><nowiki>qs snap-schedule-assoc-list --schedule=db-nightly</nowiki></code> confirms the members.

'''2. Register the analysis server.''' Read its IQN from <code><nowiki>/etc/iscsi/initiatorname.iscsi</nowiki></code> (install <code><nowiki>open-iscsi</nowiki></code> or <code><nowiki>iscsi-initiator-utils</nowiki></code> if it is missing) and add a Host with that IQN, as described in [[ISCSI Initiator Setup]]:

<pre>qs host-add --hostname=analytics1 --host-type=linux \
--iqn=iqn.1994-05.com.redhat:5a66857c49dc</pre>

[[File:Guide-db-analysis-iscsi-snapshot-db-snap-add-host.png|frame|center|The Add Host dialog: set Operating System Type to Linux and paste the server's IQN into iSCSI Initiator]]

The appliance port at 192.0.2.10 in the examples below must have iSCSI enabled, or discovery returns no portals.

'''3. Install the refresh script''' on the analysis server. It needs the <code><nowiki>qs</nowiki></code> CLI pointed at the appliance (the <code><nowiki>QS_SERVER</nowiki></code> variable or <code><nowiki>~/.qs.cnf</nowiki></code>, per the [[QuantaStor CLI Command Reference|CLI reference]]), plus <code><nowiki>jq</nowiki></code> and <code><nowiki>iscsiadm</nowiki></code>:

<pre>#!/bin/bash
set -euo pipefail # usage: refresh-db-snaps.sh [--use-existing-snaps]

SCHEDULE=db-nightly
HOST=analytics1 # Host object holding this server's IQN
PORTAL=192.0.2.10 # iSCSI-enabled port on the appliance
VOLUMES="db-sales db-finance db-hr" # source volumes; each mounts at $MNT/<volume>
MNT=/mnt/dbsnap
PART=-part1 # set to "" if the filesystem uses the whole disk
STATE=/var/lib/dbsnap # which snapshot is mounted where
WAIT=600 # seconds to wait for new snapshots

USE_EXISTING=0
[ "${1:-}" = "--use-existing-snaps" ] && USE_EXISTING=1
mkdir -p "$STATE"

latest() { # names are <volume>_GMTYYYYMMDD_HHMMSS (UTC): last sorted is newest
qs volume-list --json | jq -r '.. | objects | .name? // empty' \
| grep -E "^${1}_GMT[0-9]{8}_[0-9]{6}\$" | sort -u | tail -1 || true
}

declare -A NEW OLD
if [ "$USE_EXISTING" = 1 ]; then
for v in $VOLUMES; do NEW[$v]=$(latest "$v"); done
else
for v in $VOLUMES; do OLD[$v]=$(latest "$v"); done
qs snap-schedule-trigger --schedule="$SCHEDULE"
deadline=$((SECONDS + WAIT))
for v in $VOLUMES; do
while NEW[$v]=$(latest "$v"); [ "${NEW[$v]}" = "${OLD[$v]}" ]; do
[ "$SECONDS" -lt "$deadline" ] || { echo "no new snapshot of $v" >&2; exit 1; }
sleep 10
done
done
fi

changed=0 # remount only if some volume has a newer snapshot than the one mounted
for v in $VOLUMES; do
[ -n "${NEW[$v]}" ] || { echo "no snapshot of $v yet" >&2; exit 1; }
[ "${NEW[$v]}" = "$(cat "$STATE/$v" 2>/dev/null)" ] || changed=1
done
[ "$changed" = 1 ] || { echo "snapshots unchanged, mounts left in place"; exit 0; }

for v in $VOLUMES; do
# Detach the previous snapshot: unmount, log out, remove the assignment.
if [ -f "$STATE/$v" ]; then
old=$(cat "$STATE/$v"); oldiqn=$(cat "$STATE/$v.iqn")
if mountpoint -q "$MNT/$v"; then umount "$MNT/$v"; fi
iscsiadm -m node -T "$oldiqn" -p "$PORTAL" --logout || true
iscsiadm -m node -T "$oldiqn" -p "$PORTAL" -o delete || true
qs volume-unassign --volume="$old" --host-list="$HOST" || true
rm -f "$STATE/$v" "$STATE/$v.iqn"
fi

# Attach the new one. Assigning it materializes the lazy clone and its target.
snap=${NEW[$v]}
qs volume-assign --volume="$snap" --host-list="$HOST"
iqn=$(qs volume-get --volume="$snap" --json \
| jq -r '.. | objects | .iqn? // empty' | head -1)
iscsiadm -m discovery -t sendtargets -p "$PORTAL" >/dev/null
iscsiadm -m node -T "$iqn" -p "$PORTAL" --login
dev=/dev/disk/by-path/ip-$PORTAL:3260-iscsi-$iqn-lun-0$PART
for _ in $(seq 30); do [ -b "$dev" ] && break; sleep 1; done
mkdir -p "$MNT/$v"
mount "$dev" "$MNT/$v"

# Verify: mounted and not empty, then record it for the next run.
mountpoint -q "$MNT/$v" && [ -n "$(ls -A "$MNT/$v")" ] \
|| { echo "mount check failed for $snap" >&2; exit 1; }
echo "$snap" > "$STATE/$v"; echo "$iqn" > "$STATE/$v.iqn"
echo "$v -> $snap mounted at $MNT/$v"
done</pre>

<code><nowiki>set -e</nowiki></code> stops the script if an unmount fails, which is what you want when a report still has files open: the old snapshot stays mounted and nothing is half-swapped.

== Operating and testing ==

Choose one of two timing models:

* '''Script-driven.''' Cron runs <code><nowiki>refresh-db-snaps.sh</nowiki></code> when you want the copy, for example after the nightly ETL load finishes. The trigger makes the snapshots, and the script waits up to <code><nowiki>WAIT</nowiki></code> seconds for a new name to appear on every volume.
* '''Schedule-driven.''' The schedule's own 1 AM run makes the snapshots and cron runs <code><nowiki>refresh-db-snaps.sh --use-existing-snaps</nowiki></code> at 1:30 AM. The script compares the newest snapshot names with what it last mounted and exits without touching the mounts if nothing is new. This is also the safe way to re-run the script by hand during the day.

[[File:Guide-db-analysis-iscsi-snapshot-db-snap-host-assignments.png|frame|center|The Hosts view: the analysis server's initiator IQN, and in Storage Volume Assignments, the volumes currently assigned to it]]

To test, run the script by hand once and check four things:

# <code><nowiki>qs volume-list</nowiki></code> shows a new <code><nowiki>db-sales_GMT...</nowiki></code> snapshot for each volume, all with the same timestamp.
# <code><nowiki>qs volume-assign-list --host=analytics1</nowiki></code> lists exactly one snapshot per source volume.
# <code><nowiki>iscsiadm -m session</nowiki></code> shows one session per snapshot target.
# The database engine starts against <code><nowiki>/mnt/dbsnap</nowiki></code> and your reports return yesterday's numbers.

Run it a second time with <code><nowiki>--use-existing-snaps</nowiki></code> and confirm it reports the snapshots as unchanged. If no new snapshot ever appears, <code><nowiki>qs snap-schedule-get --schedule=db-nightly</nowiki></code> shows whether the schedule is enabled. A schedule left with no members disables itself.

Old snapshots need no cleanup code. Once the script has unassigned a snapshot ([[Remove Storage Assignment]] is the equivalent in the web interface), the schedule rotates it out when it falls outside the window. To keep one copy for an audit, put a hold on it with <code><nowiki>qs volume-hold-add --volume=<snapshot> --hold-tag=keep-for-audit</nowiki></code>. A held snapshot is excluded from schedule cleanup until the hold is released.

== FAQ ==

=== Is a snapshot taken by the schedule consistent across several database volumes? ===

Yes, for volumes in the same storage pool. The schedule snapshots all of its ZFS members in a pool with a single ZFS command, so they share one point in time and are crash-consistent with each other. Volumes in different pools, or a schedule with Disable Atomicity ticked, don't get that guarantee.

=== Does running analysis on the snapshot slow down the production database? ===

The analysis reads come from the snapshot's clone, which shares unchanged blocks with the production volume in the same pool. The analysis server's queries therefore still use the pool's disks, but none of the work runs inside the production database engine, and writes to the clone never reach the production volume. If pool I/O is a concern, schedule the refresh and heavy reports outside peak hours.

=== What does --use-existing-snaps do? ===

It skips the <code><nowiki>snap-schedule-trigger</nowiki></code> call and works with snapshots the schedule has already made. If the newest snapshot of every volume is the one already mounted, the script exits without unmounting anything. Otherwise it swaps in the newer snapshots exactly as a normal run would.

=== Can I trigger the schedule over the REST API instead of the CLI? ===

Yes. The <code><nowiki>snapshotScheduleTrigger</nowiki></code> call takes the schedule name or ID and signals the schedule manager to run it immediately, the same as <code><nowiki>qs snap-schedule-trigger</nowiki></code>. The CLI is the simpler choice for a bash script, and the API suits tools that already speak JSON-RPC.

=== How much pool space do the snapshots use? ===

Each snapshot holds only the blocks overwritten in production since it was taken, plus anything the analysis server writes to its clone. With Max Short Term Snapshots at 3 and no long-term retention, you hold roughly three days of change per volume.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-db-analysis-iscsi-snapshot-db-snap-host-assignments.png

Qadmin — Wed, 07 Oct 2026 05:45:07 GMT

Qadmin uploaded File:Guide-db-analysis-iscsi-snapshot-db-snap-host-assignments.png osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

New page

osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

File:Guide-db-analysis-iscsi-snapshot-db-snap-add-host.png

Qadmin — Wed, 07 Oct 2026 05:45:07 GMT

Qadmin uploaded File:Guide-db-analysis-iscsi-snapshot-db-snap-add-host.png osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

New page

osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-interval.png

Qadmin — Wed, 07 Oct 2026 05:45:07 GMT

Qadmin uploaded File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-interval.png osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

New page

osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-retention.png

Qadmin — Wed, 07 Oct 2026 05:45:07 GMT

Qadmin uploaded File:Guide-db-analysis-iscsi-snapshot-db-snap-schedule-retention.png osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

New page

osn-seo-utilities: screenshot for Guides (db-analysis-iscsi-snapshot)

Guides:Encryption at Rest and FIPS Mode for On-Premises Storage

Qadmin — Wed, 07 Oct 2026 05:45:04 GMT

osn-seo-utilities: fips-encryption @ 9d6065261ad2 (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 3, 2026''

Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys. QuantaStor encrypts storage pools with AES-256, either in software through Linux LUKS with AES-NI acceleration or in hardware on Opal 2 self-encrypting NVMe drives, with keys held on the appliance, protected by a passphrase or stored on a KMIP key server. A separate FIPS mode switches the appliance's cryptography to the FIPS 140-2 validated OSNEXUS Crypto Library and self-tests it at every start.

== Why it matters ==

Federal, defense, healthcare and financial buyers usually face two separate requirements that are easy to mix up:

* '''Data at rest must be encrypted.''' Lost drives, RMA returns and decommissioned servers must not expose data.
* '''The cryptography must be FIPS validated.''' Many government contracts and agency policies require cryptographic modules validated under the NIST Cryptographic Module Validation Program (CMVP), and an auditor will ask for the certificate number.

Encryption alone covers the first requirement. To cover the second, the module doing the encryption needs a certificate, and that decides which of the options below fits. The [[End-to-end Security]] overview places encryption at rest alongside the other controls agencies typically review: encryption on the wire, role-based access control, password policy and data shredding.

== How it works ==

=== Software encryption ===

[[Software Encryption|Software encryption]] uses the Linux LUKS key management system, so it works on any media type: HDD, SATA/SAS SSD or NVMe. The AES-NI instructions in modern server processors do the encryption. It is applied when the pool is created, because the underlying devices themselves are encrypted. An encrypted pool shows a lock icon in the web interface. When you grow the pool, new devices are encrypted automatically before they're added.

=== Hardware encryption (self-encrypting drives) ===

[[Hardware Encryption|Hardware encryption]] moves the work onto the drive. QuantaStor supports Opal 2 compliant NVMe self-encrypting drives (SEDs), both FIPS and non-FIPS models. For HDD-based SED encryption, the recommended approach is a Seagate Corvault system, which handles encryption in the external enclosure. [[Physical Disks/Devices]] reports each drive's SED capability and status. The older RAID-controller-based encryption is deprecated; HBA (IT) mode controllers are the current standard.

=== Key protection ===

The '''Encryption''' tab of the Create Storage Pool dialog offers three ways to protect a pool's keys:

{| class="wikitable"
! Key protection !! Where the key lives !! Behavior
|-
| Encrypt with No Passphrase || On the appliance || Pool starts automatically at boot; supported in HA groups
|-
| Encrypt with User Defined Passphrase || On the appliance, locked by the passphrase || Pool starts only after an administrator enters the passphrase; not supported in HA groups
|-
| Encrypt and Store on Key Server || On a KMIP key server || Keys are managed centrally through a key server profile
|}

KMIP is the standard protocol for talking to enterprise key managers. You add a [[Create Key Server Profile|key server profile]], then select it when you create a pool. You can also move a pool's locally stored keys onto the key server later by modifying the pool and selecting the profile.

[[File:Guide-fips-encryption-fips-encryption-create-pool-software.png|frame|center|The Encryption tab of Create Storage Pool with Software Encryption selected: AES-256 for every option, the three Key Protection choices, and the passphrase and Key Server Profile fields below them]]

=== FIPS mode ===

[[FIPS Mode|FIPS mode]] controls which cryptographic library the appliance uses. QuantaStor ships two complete OpenSSL library sets and switches between them:

{| class="wikitable"
! Mode !! OpenSSL !! Crypto library
|-
| Non-FIPS (default) || 3.5.7 || Non-FIPS build of the OSNEXUS Crypto Library
|-
| FIPS || 3.1.2 with the OpenSSL FIPS provider || OSNEXUS Crypto Library (FIPS build)
|}

In FIPS mode the appliance runs power-on self-tests and integrity checks, and the QuantaStor service won't start if they fail. The FIPS path stays on OpenSSL 3.1.2 on purpose: the FIPS provider is tied to the OpenSSL release it was built and validated against, so it is pinned separately from the general runtime.

== Design and sizing ==

=== Certified versus compliant ===

Be precise about what is validated. The '''OSNEXUS Crypto Library holds FIPS 140-2 Level 1 validation, [https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/4185 NIST CMVP certificate 4185]''', with a public [https://csrc.nist.gov/CSRC/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp4185.pdf non-proprietary security policy]. QuantaStor 6 and 7 have '''not''' been through FIPS 140-3 certification. The crypto library is maintained against current FIPS 140-3 compliant OpenSSL releases, but compliant isn't the same as certified.

Where a deployment requires FIPS certification, the documented recommendation is '''self-encrypting drives with full-disk encryption on FIPS 140-3 validated media'''. That puts the certified cryptographic boundary at the drive, where the media vendor holds the certificate. Running FIPS mode on the appliance as well is still worthwhile: it limits management cryptography to validated algorithms and self-tests them at every start, which satisfies many internal security policies.

=== Choosing an approach ===

{| class="wikitable"
! Requirement !! Recommended configuration
|-
| Encrypt at rest, no formal certification || Software encryption, no passphrase or KMIP
|-
| Certificate required for data at rest || FIPS 140-3 SED media with hardware encryption, plus FIPS mode
|-
| Keys held off the appliance under central control || Software encryption with a KMIP key server profile
|-
| Pool must not start without a person present || Software encryption with a user-defined passphrase (not with HA)
|-
| HA failover between two controllers || Software encryption without a passphrase, or SEDs
|}

=== Performance ===

Software encryption typically reduces a pool's performance by about 15%, and by as much as 30% depending on the CPU and the number of devices. Hardware encryption avoids that CPU cost, so if your media is SED Opal compliant, it's the better option. Plan software-encrypted pools with that headroom in mind, especially all-NVMe pools, where the CPU is more likely to be the bottleneck.

== Setting it up ==

=== Create an encrypted pool ===

# Go to '''Storage Management → Storage Pools → Create'''. See [[Storage Pools]] for the General tab settings.
# On the '''Encryption''' tab, select '''Software Encryption''' or '''Hardware Encryption'''.
# Under '''Key Protection''', choose no passphrase, a user-defined passphrase, or '''Encrypt and Store on Key Server''' with a key server profile.
# Finish creating the pool. Encryption can't be added after the pool is created.

=== Back up the keys ===

Immediately after you create the pool, use '''Storage Pool Encryption → Export Keys''' ([[Storage Pool Export Encryption Keys]]) and keep the backup somewhere secure. If the boot media is lost, you reinstall QuantaStor on new boot devices, run '''Import Keys''' and start the pool. The same ribbon group holds '''Re-Key Pool'''.

[[File:Guide-fips-encryption-fips-encryption-pool-encryption-ribbon.png|frame|center|The Storage Pools view with the Storage Pool Encryption group on the ribbon: Import Keys, Export Keys and Re-Key Pool sit next to the pool and HA group actions]]

=== Enable FIPS mode ===

FIPS mode is set per appliance, from the console as root. A grid has no grid-wide switch, so repeat this on every appliance (see [[Grid Configuration]]). Connect over SSH as <code><nowiki>qadmin</nowiki></code>, then run:

<pre>sudo qs-util enablefips
sudo reboot</pre>

<code><nowiki>enablefips</nowiki></code> repoints the library symlinks and restarts the service, reporting <code><nowiki>FIPS mode enabled in the storage system. Restarting QuantaStor Service</nowiki></code>. The reboot is required. Several integrity checks run only at system startup, and the FIPS state doesn't settle until they have. Restarting the service isn't a substitute.

=== HA and the rest of the security stack ===

Encrypted pools work in [[Create Storage Pool High-Availability Group|storage pool HA groups]] like unencrypted pools, provided they have no passphrase. For data on the wire, use SMB3 encryption, [[IPSec|IPsec]] and HTTPS, and CHAP authentication for iSCSI. Grid management and replication traffic use SSL/TLS 1.2 with strong ciphers; see [[Custom SSL TLS Security]]. Administrative access is covered by [[Security Configuration]] (role-based access control) and the [[Password Policy set|password policy]], which supports HIPAA and CJIS requirements.

== Operating and testing ==

Check the FIPS state after the reboot, not before:

<pre>qs system-get | grep -i fips
sudo /opt/osnexus/quantastor/bin/osn_fipscheck validate-fips
tail -f /var/log/qs/qs_crypto.log</pre>

On an appliance in FIPS mode, <code><nowiki>qs system-get</nowiki></code> reports:

<pre>FIPS State: Verified
FIPS State Detail: FIPS 140-2 security validation checks succeeded, FIPS mode enabled.</pre>

<code><nowiki>osn_fipscheck validate-fips</nowiki></code> confirms that FIPS mode is enabled and the module passed its self-tests, returning 0 on success and 1 or greater on failure. A healthy start logs <code><nowiki>Start up self-tests completed successfully</nowiki></code> and the DRBG in use (<code><nowiki>CTR-DRBG</nowiki></code>) in <code><nowiki>qs_crypto.log</nowiki></code>. In the web interface, the '''FIPS''' and '''FIPS State Detail''' fields appear in the storage system's Properties panel.

To confirm which OpenSSL the appliance actually uses, don't run <code><nowiki>openssl version</nowiki></code>, which reports the operating system's copy. Read the active library instead:

<pre>strings /opt/osnexus/common/lib/active/libcrypto.so.3 | grep -oE 'OpenSSL 3\.[0-9]+\.[0-9]+' | sort -u</pre>

Routine operations on an encrypted pool work the same as on a plain one: adding devices, replacing a faulted drive, and adding write log, cache or spare devices to a passphrase-protected pool. After a reboot, a passphrase-protected pool stays stopped until an administrator starts it with the passphrase from the web interface or the CLI. To change a passphrase, remove it and apply a new one.

When a pool is retired, delete it using a data shredding option: the 4-pass DoD 5220.22-M (section 8-306) procedure, the 4-pass NNSA Policy Letter NAP-14.1-C procedure or the US Army AR380-19 method. Shredding runs at the block level, concurrently across all disks in the pool, and the encryption keys can be shredded too.

== FAQ ==

=== Is QuantaStor FIPS 140 validated? ===

The OSNEXUS Crypto Library, which QuantaStor uses in FIPS mode, holds FIPS 140-2 Level 1 validation under NIST CMVP certificate 4185. QuantaStor 6 and 7 haven't been through FIPS 140-3 certification. Where a 140-3 certificate is required for data at rest, use FIPS 140-3 validated self-encrypting drives, where the drive vendor holds the certificate.

=== Is it suitable for US government and defense deployments? ===

It provides the controls these environments usually review: AES-256 encryption at rest, a FIPS mode that uses a validated crypto library, support for FIPS SED media, encryption on the wire with SMB3, IPsec and TLS 1.2, role-based access control, password policy that supports HIPAA and CJIS requirements, and DoD 5220.22-M, NNSA and AR380-19 data shredding. Whether a given deployment passes accreditation depends on the agency's specific control set, so map these controls to it.

=== Can I encrypt an existing pool? ===

No. Software encryption encrypts the underlying devices, so it has to be selected when the pool is created. To encrypt existing data, create a new encrypted pool and move the data onto it.

=== What happens if I lose the boot drives? ===

If you exported the pool keys, reinstall on new boot media, import the keys and start the pool. Without the key backup, a software-encrypted pool can't be recovered, so export the keys as soon as you create the pool.

=== Does encryption work with high availability? ===

Yes. Encrypted pools fail over between controllers like unencrypted pools, provided they have no passphrase. A passphrase would stop the pool from starting automatically on the surviving controller.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-fips-encryption-fips-encryption-pool-encryption-ribbon.png

Qadmin — Wed, 07 Oct 2026 05:45:04 GMT

Qadmin uploaded File:Guide-fips-encryption-fips-encryption-pool-encryption-ribbon.png osn-seo-utilities: screenshot for Guides (fips-encryption)

New page

osn-seo-utilities: screenshot for Guides (fips-encryption)

File:Guide-fips-encryption-fips-encryption-create-pool-software.png

Qadmin — Wed, 07 Oct 2026 05:45:04 GMT

Qadmin uploaded File:Guide-fips-encryption-fips-encryption-create-pool-software.png osn-seo-utilities: screenshot for Guides (fips-encryption)

New page

osn-seo-utilities: screenshot for Guides (fips-encryption)

Network Ports

Umbe — Wed, 07 Oct 2026 05:14:09 GMT

Management and grid

← Older revision		Revision as of 05:14, 7 October 2026
(One intermediate revision by the same user not shown)
Line 390:		Line 390:
	\| 443 \|\| TCP \|\| Web management interface (HTTPS) \|\| -- \|\| The address you browse to. Keep it open from the networks your administrators work from.		\| 443 \|\| TCP \|\| Web management interface (HTTPS) \|\| -- \|\| The address you browse to. Keep it open from the networks your administrators work from.
	\|-		\|-
	\| 80 \|\| TCP \|\| Web management interface (HTTP) \|\| QS Web Management \|\| Redirects to HTTPS on 443 by default. ~~Close it if your browsers~~ always ~~use~~ HTTPS.		\| 80 \|\| TCP \|\| Web management interface (HTTP) \|\| QS Web Management \|\| Redirects to HTTPS on 443 by default. This is safe to close as you'll always be using HTTPS but the redirect is there for convenience.
	\|-		\|-
	\| 8080 \|\| TCP \|\| Web management interface (HTTPS, alternate port) \|\| QS Web Management \|\| Serves the same interface over HTTPS on a second port. ~~Close~~ it unless something depends on it.		\| 8080 \|\| TCP \|\| Web management interface (HTTPS, alternate port) \|\| QS Web Management \|\| Serves the same interface over HTTPS on a second port. This is available as a backup in case a service or firewall is setup to block it. On example where this can be handy is if the object storage gateways is set to bind to 443 rather than 7480. Safe to close it unless something depends on it.
	\|-		\|-
	\| 8153 \|\| TCP \|\| [[QuantaStor REST API Reference Guide\|REST API]] (HTTPS) \|\| QS REST API \|\| Needed by scripts, automation, plugins and integrations that call the REST API. ~~Close it~~ if ~~nothing~~ does.		\| 8153 \|\| TCP \|\| [[QuantaStor REST API Reference Guide\|REST API]] (HTTPS) \|\| QS REST API \|\| Needed by scripts, automation, plugins and integrations that call the REST API. Safe to close if you're not using the QuantaStor REST API in scripts, QuantaStor Proxmox plug-in or the QuantaStor Python Client library. The 'qs' CLI does not use this interface, it goes direct into the service API on 5151.
	\|-		\|-
	\| 5151 \|\| TCP \|\| QuantaStor core service (TLS) \|\| -- \|\| Every appliance in a [[Grid Configuration\|storage grid]] talks to the others on this port, so it must be open between all grid members in both directions. The <code>qs</code> CLI also uses it when pointed at a remote system.		\| 5151 \|\| TCP \|\| QuantaStor core service (TLS) \|\| -- \|\| Every appliance in a [[Grid Configuration\|storage grid]] talks to the others on this port, so it must be open between all grid members in both directions. The <code>qs</code> CLI also uses it when pointed at a remote system.

Guides:Index

Qadmin — Wed, 07 Oct 2026 05:00:08 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 05:00, 7 October 2026
Line 5:		Line 5:
	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
	* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.		* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.
			* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

Network Ports

Qadmin — Wed, 07 Oct 2026 05:00:03 GMT

osn-seo-utilities: quantastor-network-port-usage @ 5caca23311c7 (approved in the portal)

Show changes

Clustered SCSI-3 Persistent Reservations

Qadmin — Tue, 06 Oct 2026 16:02:05 GMT

Reword the Windows MPIO links

← Older revision		Revision as of 16:02, 6 October 2026
Line 60:		Line 60:
	* '''A healthy site cluster.''' The appliances must be members of a [[Site Cluster Setup\|site cluster]] with Corosync and Pacemaker running. QuantaStor gives the Corosync cluster a name if it does not have one, because the DLM cannot start without it; this is done across the cluster under maintenance mode, and needs no action from you.		* '''A healthy site cluster.''' The appliances must be members of a [[Site Cluster Setup\|site cluster]] with Corosync and Pacemaker running. QuantaStor gives the Corosync cluster a name if it does not have one, because the DLM cannot start without it; this is done across the cluster under maintenance mode, and needs no action from you.

	The operating system on the Windows side needs nothing beyond what a failover cluster already requires: multipath I/O configured for the iSCSI or Fibre Channel paths to the QuantaStor appliances, and the cluster nodes' initiators registered as hosts on QuantaStor. For the recommended MPIO ~~setup and timer tuning~~, ~~so that~~ cluster disks ~~ride~~ through a failover ~~instead of being removed~~, see [[Windows MPIO for Failover Clusters and Hyper-V]].		The operating system on the Windows side needs nothing beyond what a failover cluster already requires: multipath I/O configured for the iSCSI or Fibre Channel paths to the QuantaStor appliances, and the cluster nodes' initiators registered as hosts on QuantaStor. For the recommended MPIO configuration, which keeps cluster disks online and Cluster Shared Volumes running smoothly through a failover, see [[Windows MPIO for Failover Clusters and Hyper-V]].

	== Enabling clustered reservations ==		== Enabling clustered reservations ==
Line 104:		Line 104:
	Add each Windows cluster node as a host with its iSCSI IQN or Fibre Channel WWPNs, put all of the nodes in one Host Group, and assign the volumes to that Host Group. [[Hosts and Host Groups#Host Groups\|Host Groups]] covers why a group assignment is the right tool for a cluster; for Fibre Channel, [[Fibre Channel Target Port Management#Assigning a storage volume over FC\|Assigning a storage volume over FC]] covers zoning and WWPNs.		Add each Windows cluster node as a host with its iSCSI IQN or Fibre Channel WWPNs, put all of the nodes in one Host Group, and assign the volumes to that Host Group. [[Hosts and Host Groups#Host Groups\|Host Groups]] covers why a group assignment is the right tool for a cluster; for Fibre Channel, [[Fibre Channel Target Port Management#Assigning a storage volume over FC\|Assigning a storage volume over FC]] covers zoning and WWPNs.

	For iSCSI, have each node log in to the pool's HA virtual interface rather than an appliance's own address, so its sessions follow the pool when it fails over. For Fibre Channel, configure multipath on each node; QuantaStor advertises the paths through the owning appliance as active and the others as standby. ~~Before going further~~, set up MPIO ~~and apply~~ the recommended ~~timer~~ settings on every node as described in [[Windows MPIO for Failover Clusters and Hyper-V]]; ~~the settings need~~ a reboot of each node.		For iSCSI, have each node log in to the pool's HA virtual interface rather than an appliance's own address, so its sessions follow the pool when it fails over. For Fibre Channel, configure multipath on each node; QuantaStor advertises the paths through the owning appliance as active and the others as standby. For the best results, set up MPIO with the recommended settings on every node as described in [[Windows MPIO for Failover Clusters and Hyper-V]]; a reboot of each node activates them.

	=== 4. Bring the disks online on one node ===		=== 4. Bring the disks online on one node ===

Windows MPIO for Failover Clusters and Hyper-V

Qadmin — Tue, 06 Oct 2026 16:02:04 GMT

Reword as the recommended configuration

Show changes

Multipath IO Configuration

Qadmin — Tue, 06 Oct 2026 15:58:54 GMT

Point to the Windows MPIO page for failover clusters and Hyper-V

← Older revision		Revision as of 15:58, 6 October 2026
Line 30:		Line 30:

	=== Configuring Microsoft MPIO ===		=== Configuring Microsoft MPIO ===
			''For Windows Server failover clusters, Cluster Shared Volumes and Hyper-V on QuantaStor HA pools, see [[Windows MPIO for Failover Clusters and Hyper-V]] for the recommended MPIO setup and timer tuning; the steps below cover only adding the QuantaStor device string.''

	The first step in configuring MPIO on Windows Server is to navigate to MPIO properties. (Note that desktop versions of Windows such as XP, Vista, and Windows 7/8 do not support MPIO, you must be using Windows Server 2003/2008/2012) The easiest way to get to the MPIO configuration screen is to type MPIO in the ''Start'' menu search bar. It can also be found in the Control Panel under "MPIO".		The first step in configuring MPIO on Windows Server is to navigate to MPIO properties. (Note that desktop versions of Windows such as XP, Vista, and Windows 7/8 do not support MPIO, you must be using Windows Server 2003/2008/2012) The easiest way to get to the MPIO configuration screen is to type MPIO in the ''Start'' menu search bar. It can also be found in the Control Panel under "MPIO".

Clustered SCSI-3 Persistent Reservations

Qadmin — Tue, 06 Oct 2026 15:58:54 GMT

Link the recommended Windows MPIO setup and tuning page

Windows MPIO for Failover Clusters and Hyper-V

Qadmin — Tue, 06 Oct 2026 15:58:54 GMT

Recommended Windows MPIO setup and timer tuning for failover clusters, CSV and Hyper-V on QuantaStor HA pools

New page

[[Category:admin_guide]]

This page covers the recommended Microsoft Multipath I/O (MPIO) setup and tuning for Windows Server hosts that use QuantaStor storage volumes as shared cluster disks: Windows Server Failover Clustering, Cluster Shared Volumes, Hyper-V clusters and SQL Server Failover Cluster Instances. It applies to Fibre Channel and iSCSI access to a QuantaStor HA pool, and it is the client-side companion to [[Clustered SCSI-3 Persistent Reservations]].

'''Summary.''' Install MPIO, let the Microsoft DSM claim QuantaStor devices, and raise three MPIO timers above their Windows defaults so that a disk survives a QuantaStor HA failover or an appliance reboot without Windows removing it. The settings were validated on Windows Server 2022 Hyper-V clusters connected over Fibre Channel to a QuantaStor HA pair.

{| class="wikitable"
! Section !! Purpose
|-
| [[#Why the defaults are not enough|Why the defaults are not enough]] || What Windows does when every path to a disk is briefly unavailable
|-
| [[#Installing MPIO and claiming QuantaStor devices|Installing MPIO and claiming QuantaStor devices]] || The MPIO feature and the QuantaStor hardware ID
|-
| [[#Recommended MPIO settings|Recommended MPIO settings]] || The timer values, and how to apply and verify them
|-
| [[#Paths and load balancing|Paths and load balancing]] || What a correctly configured disk looks like
|-
| [[#After maintenance on a QuantaStor appliance|After maintenance on a QuantaStor appliance]] || Checking that a rebooted appliance's paths are back before failing pools to it
|-
| [[#Troubleshooting|Troubleshooting]] || Disks that go offline during a failover, and missing paths
|}

== Why the defaults are not enough ==

During an HA failover the pool moves from one QuantaStor appliance to the other. For a short time no path to a volume can complete I/O: the departing appliance has stopped serving the pool and the receiving appliance has not yet finished importing it. With Fibre Channel and ALUA, QuantaStor keeps the standby paths through the partner appliance present throughout, so Windows sees the paths change state rather than disappear; the I/O stall is typically well under a minute.

Windows decides how long to wait with three MPIO settings. With the defaults, MPIO deletes the disk 20 seconds after its last path stops working (PDORemovePeriod), and each I/O times out after 60 seconds (DiskTimeoutValue). A failover that runs longer than that, for example because the pool export runs up to its configured Export Timeout, makes Windows remove the disk. For a cluster disk that means the disk goes Failed or Offline, a Cluster Shared Volume stops, and Hyper-V virtual machines lose their storage. Windows does not bring a removed disk back on its own when the pool comes back: an administrator has to rescan storage and bring the cluster disk online again.

Raising the timers makes Windows hold I/O and keep the disk while the pool moves, so the failover appears to the cluster as a pause rather than a disk failure.

== Installing MPIO and claiming QuantaStor devices ==

Do this on every cluster node, before presenting QuantaStor volumes to it.

'''1. Install the Multipath I/O feature''' (requires a reboot):

<pre style="font-size: smaller">
Install-WindowsFeature -Name Multipath-IO
</pre>

'''2. Add the QuantaStor hardware ID to the Microsoft DSM''' so that MPIO combines the paths to each QuantaStor volume into one disk:

<pre style="font-size: smaller">
New-MSDSMSupportedHW -VendorId OSNEXUS -ProductId QUANTASTOR
Get-MSDSMSupportedHW | Where-Object VendorId -match OSNEXUS
</pre>

The PowerShell cmdlet pads the vendor and product IDs to their SCSI field widths. If you use the MPIO control panel or {{Code|1=mpclaim}} instead, the string must be exactly {{Code|1=OSNEXUS QUANTASTOR}} followed by six trailing spaces, as described in [[Multipath IO Configuration#Configuring Microsoft MPIO|Configuring Microsoft MPIO]]. Without this entry each path appears as a separate disk.

'''3. Reboot''' the node so MPIO claims the devices, then confirm in Disk Management or with {{Code|1=Get-Disk}} that each QuantaStor volume appears once.

For iSCSI, also connect one session per path in the iSCSI Initiator with '''Enable multi-path''' selected, using addresses on separate subnets; see [[ISCSI Initiator Setup]] and [[Multipath IO Configuration]]. For an HA pool, connect to the pool's HA virtual interfaces rather than an appliance's own addresses, so the sessions follow the pool when it fails over.

== Recommended MPIO settings ==

{| class="wikitable"
! Setting !! Windows default !! Recommended !! Effect
|-
| PDORemovePeriod || 20 || '''240''' || Seconds MPIO keeps a disk after its last path has failed before removing it. Must be longer than the slowest failover you expect.
|-
| DiskTimeoutValue || 60 || '''100''' || Seconds before Windows times out an individual I/O. 100 is the largest value {{Code|1=Set-MPIOSetting}} accepts.
|-
| PathVerificationState || Disabled || '''Enabled''' || MPIO periodically tests every path, so a failed path is detected and a recovered path is put back into use without waiting for I/O to hit it.
|-
| PathVerificationPeriod || 30 || 30 || Seconds between path tests. The default is fine.
|-
| RetryCount || 3 || 3 || Times MPIO retries a failed I/O on a path. Leave at the default.
|-
| RetryInterval || 1 || 1 || Seconds between those retries. Leave at the default.
|}

Apply them in an elevated PowerShell on each cluster node:

<pre style="font-size: smaller">
Set-MPIOSetting -NewPathVerificationState Enabled -NewPathVerificationPeriod 30 -NewPDORemovePeriod 240 -NewRetryCount 3 -NewRetryInterval 1 -NewDiskTimeout 100
</pre>

'''Reboot the node for the settings to take effect.''' In a running cluster, drain one node at a time (Pause, Drain Roles in Failover Cluster Manager, or {{Code|1=Suspend-ClusterNode -Drain}}), reboot it, resume it, and move on to the next.

Verify with {{Code|1=Get-MPIOSetting}}, which is the authoritative view; the cmdlet stores the values through WMI, so reading the MPIO registry key directly can be misleading:

<pre style="font-size: smaller">
PS C:\> Get-MPIOSetting

PathVerificationState : Enabled
PathVerificationPeriod : 30
PDORemovePeriod : 240
RetryCount : 3
RetryInterval : 1
UseCustomPathRecoveryTime : Disabled
CustomPathRecoveryTime : 40
DiskTimeoutValue : 100
</pre>

If {{Code|1=Set-MPIOSetting}} reports a parameter error, for example a {{Code|1=-NewDiskTimeout}} above 100, none of the values in that command are applied. Check the output for errors and run {{Code|1=Get-MPIOSetting}} afterwards.

Leave the Fibre Channel HBA driver parameters at the vendor defaults; these settings were validated with the QLogic Windows driver at its defaults.

== Paths and load balancing ==

Leave the Microsoft DSM load-balance policy at its default. QuantaStor reports ALUA path states for HA pools: paths through the appliance that owns the pool are Active/Optimized and paths through the partner appliance are Standby. The Microsoft DSM's default for ALUA storage, Round Robin With Subset, sends I/O across the Active/Optimized paths only and switches to the other set when the states change.

Check a disk's paths with {{Code|1=mpclaim}}:

<pre style="font-size: smaller">
mpclaim -s -d # list MPIO disks
mpclaim -s -d 1 # paths and their states for MPIO disk 1
</pre>

A host with two HBA ports zoned to both appliances of an HA pair shows four paths per disk: two Active/Optimized and two Standby. After a failover, the states swap between the two appliances, and the number of paths stays the same.

== After maintenance on a QuantaStor appliance ==

When a QuantaStor appliance reboots or its Fibre Channel ports go down, Windows marks its paths as failed. When the appliance returns, Windows does not always add its paths back on its own. A disk then keeps running on the partner appliance's paths, but a pool failed back to the returning appliance would find that host with standby paths only.

After any appliance reboot, and before failing a pool back to it:

<ol>
<li>On every cluster node, rescan storage:
<pre style="font-size: smaller">
Update-HostStorageCache
pnputil /scan-devices
</pre>
If paths are still missing, run {{Code|1=rescan}} in {{Code|1=diskpart}}.</li>
<li>Check with {{Code|1=mpclaim -s -d <n>}} that every QuantaStor disk shows its full path count again.</li>
<li>Only then fail the pool back.</li>
</ol>

QuantaStor holds an appliance's Fibre Channel target ports disabled during startup until its LUNs are mapped and ALUA states are set, so wait until the appliance is fully up, with its Storage Pools showing a Normal state in the web interface, before rescanning.

== Troubleshooting ==

'''A cluster disk went Failed or Offline during a failover, and the disk disappeared from Disk Management.''' MPIO removed the disk because the failover outlasted PDORemovePeriod. Check {{Code|1=Get-MPIOSetting}} on every node; the values above must be in effect, which requires a reboot after setting them. To recover, run {{Code|1=Update-HostStorageCache}} and a {{Code|1=diskpart}} rescan on each node, then bring the cluster disk online in Failover Cluster Manager ({{Code|1=Start-ClusterResource}}).

'''Cluster disks went offline after a failover, but the disks are still present in Windows.''' Check that [[Clustered SCSI-3 Persistent Reservations]] is Enabled on the pool's HA group. Without it the reservations the cluster placed on the disks are lost when the pool moves, and the cluster treats the disks as failed.

'''A disk shows only Standby paths, or fewer paths than expected.''' One appliance's paths were not re-added after it rebooted. Follow [[#After maintenance on a QuantaStor appliance|After maintenance on a QuantaStor appliance]]. Also confirm that both of the host's HBA ports are zoned to both appliances, and that both of its WWPNs are listed under the host in QuantaStor.

'''Each QuantaStor volume appears more than once in Disk Management.''' MPIO has not claimed the devices. Check {{Code|1=Get-MSDSMSupportedHW}} for the OSNEXUS QUANTASTOR entry, add it if missing, and reboot.

'''Keep the appliances' Fibre Channel ports in the default target-only mode.''' Dual initiator and target mode ({{Code|1=qs-util enabledualmode}}) is a diagnostic setting. On an appliance that serves an HA pool, it can log out host sessions whenever the fabric changes, for example while the partner appliance reboots, which shows up on Windows as long I/O stalls.

== Related pages ==

* [[Clustered SCSI-3 Persistent Reservations]] -- the HA group setting Windows failover clusters need
* [[Multipath IO Configuration]] -- MPIO and Linux multipath basics for QuantaStor volumes
* [[ISCSI Initiator Setup]] -- connecting iSCSI initiators
* [[Fibre Channel Target Port Management]] -- presenting volumes over Fibre Channel
* [[Hosts and Host Groups]] -- host records, initiators, and Host Groups for clusters
* [[HA Cluster Setup (JBODs)]] -- HA groups and failover

----

Guides:Index

Qadmin — Sat, 03 Oct 2026 02:45:06 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 02:45, 3 October 2026
Line 4:		Line 4:

	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
			* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

Guides:Multi-Admin Approval for Destructive Storage Operations

Qadmin — Sat, 03 Oct 2026 02:45:02 GMT

osn-seo-utilities: multi-admin-approval @ d95f434838dc (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 3, 2026''

Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it. QuantaStor applies it grid-wide from the Security Manager, covering deletes of storage volumes, network shares, buckets, scale-up and scale-out pools, object storage classes and Ceph clusters. Approvers work from the Multi-admin Approval toolbar in the web interface, and held deletes issued from the CLI report how many approvals they have so far.

== Why it matters ==

Most storage-side ransomware and insider attacks end with the same step: someone with valid administrator credentials deletes the pools, volumes or buckets that hold the data, and with them the snapshots that would have allowed recovery. Password policy and [[Multi-Factor Authentication Manager|multi-factor authentication]] make it harder to steal those credentials, but once an attacker holds them, a single account can still destroy everything.

Multi-admin approval removes that single point of failure. A stolen or misused account can request a delete, but the delete does not start until other administrators agree. The same control catches honest mistakes, such as a pool deleted on the wrong system or a share removed during the wrong change window.

For regulated environments, it fits alongside the controls described in [[Security Configuration]]: password and lockout policy with a '''Suggested Defaults''' preset aimed at standards such as HIPAA, CJIS and NIST 800-53 / 800-171, role based access control over every management operation, LDAP single sign-on for administrators, and always-on [[Audit Logging|audit logging]] of every management operation, login attempt and authorization decision. Agencies and contractors that need separation of duties for destructive actions can enforce it on the storage itself instead of relying on a change-control process alone.

== How it works ==

The workflow has three parts: a policy, a pending request and the approval votes.

# '''Policy.''' In '''Security Manager''', the '''Multi-admin Approvals''' tab turns the feature on, sets the minimum number of approvals and the expiry time, and selects which delete operations require approval. The policy applies to the whole storage grid.
# '''Pending request.''' When an administrator deletes an object whose type is selected, QuantaStor records a pending multi-admin approval request instead of starting the delete task. The request carries the object name, type and ID, its status, the required and current approval counts, the request originator and an expiry time.
# '''Approval votes.''' Other administrators open '''Security → Multi-admin Approval → Approve''', which "casts an approval vote for the associated data-destructive operation", or '''Reject''' to refuse it. Once the current count reaches the required count, the delete proceeds. If the request is not approved before it expires, the delete does not run.

From the CLI, a held delete reports its approval status, for example <code><nowiki>1 of 3 approvals met</nowiki></code>, rather than returning an empty task, so a script or operator can see exactly why the delete has not started.

[[File:Guide-multi-admin-approval-multi-admin-approval-security-manager.png|frame|center|The Multi-admin Approvals tab of Security Manager: the Enable checkbox, Minimum Approvals and Hours Until Auto-expiration, and the list of delete operations that can require approval]]

=== Operations that can require approval ===

{| class="wikitable"
! Required approval type !! Reference
|-
| Storage Volume : Delete || [[Storage Volume Delete]]
|-
| Network Share : Delete || [[Network Share Delete]]
|-
| Bucket : Delete || [[Object Bucket Delete]]
|-
| Scale-up Storage Pool : Delete || [[Storage Pools]]
|-
| Scale-out Block Pool : Delete || [[Storage Pools]]
|-
| Scale-out File Pool : Delete || [[Storage Pools]]
|-
| Scale-out Object Pool : Delete || [[Storage Pools]]
|-
| Object Storage Class : Delete ||
|-
| Ceph Cluster : Delete ||
|}

Each type is a separate checkbox, and '''Select All''' and '''Clear All''' set them in one step.

== Design and sizing ==

Three settings define the policy. Choose them with your team size and on-call coverage in mind.

{| class="wikitable"
! Setting !! What it controls !! Guidance
|-
| Minimum Approvals || Approvals a pending delete needs before it runs (2 by default in the dialog) || 2 suits most teams. Use a higher number only if that many administrators can realistically be reached during a change window.
|-
| Hours Until Auto-expiration || How long a request waits for approval (48 hours by default in the dialog) || Long enough to span a weekend change window, short enough that stale requests don't linger. A value of 0 means requests never expire.
|-
| Required Approval Types || Which delete operations are held || Select all pool, cluster and bucket types at a minimum, because those deletes remove the most data in one step.
|}

Some practical points:

* '''Give every administrator a named account.''' Approval only means something if each vote comes from a different person. Shared logins such as a single <code><nowiki>admin</nowiki></code> account defeat the purpose. Map named accounts to roles, or use LDAP single sign-on so administrator groups are managed in your directory.
* '''Decide who may approve.''' Approving and rejecting are ordinary RBAC permissions on the <code><nowiki>PendingOperationRequest</nowiki></code> object type (<code><nowiki>view</nowiki></code>, <code><nowiki>create</nowiki></code>, <code><nowiki>approve</nowiki></code>, <code><nowiki>reject</nowiki></code>, <code><nowiki>clear</nowiki></code>). Grant <code><nowiki>approve</nowiki></code> and <code><nowiki>reject</nowiki></code> to the roles that should vote, and leave them off roles such as System Monitor.
* '''Protect the approvers.''' Turn on multi-factor authentication for every account that holds the approve permission, so a single stolen password cannot both request and approve.
* '''Keep snapshots in the plan.''' Approval stops a malicious delete, but it does nothing against data that is encrypted or overwritten in place. Pair it with [[Snapshot Schedules|snapshot schedules]] so there is a clean recovery point when data inside a volume or share is damaged.

[[File:Guide-multi-admin-approval-multi-admin-approval-role-permissions.png|frame|center|Update Role Permissions filtered to the PendingOperationRequest operations, where approve and reject rights are granted to a role. The Administrator role already has this permission but you can create new roles that are limited to a subset of operations including PendingOperationRequest approval and rejection.]]

== Setting it up ==

# Create a named account for each administrator and assign roles under '''Security → Management Users''' and '''Management Roles'''. In '''Add/Remove Permissions''' for each approving role, confirm <code><nowiki>PendingOperationRequest</nowiki></code> <code><nowiki>approve</nowiki></code> and <code><nowiki>reject</nowiki></code> are granted at the scope you need.
# Enable multi-factor authentication for those accounts from '''Multi-Factor Auth Manager'''.
# Open '''Security → Management Users → Security Manager''' (toolbar) and select the '''Multi-admin Approvals''' tab.
# Tick '''Enable Multi-admin Approvals'''.
# Set '''Minimum Approvals''' and '''Hours Until Auto-expiration'''.
# Under '''Required Approval Types''', tick the delete operations to protect, or use '''Select All'''.
# Click '''OK'''. Changes made only on this tab save without the logout and password-expiry confirmation that password-policy changes trigger, so you can enable the feature during working hours.

To roll it out without blocking routine work, start with the operations that destroy the most data at once (pools, Ceph clusters, object storage classes and buckets), run with them for a few weeks, then add volume and share deletes once the team is used to the approval step. Volume and share cleanup is more frequent, so plan who covers approvals before you protect those types.

== Operating and testing ==

Test the policy before relying on it. On a non-production object, for example a scratch volume <code><nowiki>vol-test1</nowiki></code> in <code><nowiki>pool1</nowiki></code>:

# As one administrator, delete <code><nowiki>vol-test1</nowiki></code>. The delete should not start. From the CLI, the response shows the approval status, such as <code><nowiki>1 of 2 approvals met</nowiki></code>.
# As a second administrator, open '''Security → Multi-admin Approval → Approve'''. The '''Pending Multi-admin Approval Requests''' list shows the object name, object type, required and current counts, the originator and the expiry time.
# Select the request and approve it. When the required count is reached, the delete task runs and appears in the Tasks pane.
# Repeat with a second scratch object and use '''Reject''' to confirm a refused delete never runs.

If nothing is waiting, the Approve dialog reports that there are no pending operation requests to approve.

Day to day, treat an unexpected pending request as a security event: check who originated it, and reject it if it isn't tied to an approved change. Every request, approval and delete is recorded by [[Audit Logging|audit logging]], which gives you the trail auditors ask for. For scripted cleanup jobs, check the CLI approval status so the job reports a held delete instead of treating it as a failure. Command syntax is in the [[QuantaStor CLI Command Reference]].

== FAQ ==

=== Which operations does multi-admin approval cover? ===

It covers deletes: storage volumes, network shares, buckets, scale-up storage pools, scale-out block, file and object pools, object storage classes and Ceph clusters. You choose which of these require approval on the Multi-admin Approvals tab of Security Manager. Other management operations are governed by role based access control as usual.

=== Does multi-admin approval protect against ransomware? ===

It protects against the destructive step attackers use once they hold administrator credentials: deleting pools, volumes or buckets so there is nothing left to recover from. It does not stop data from being encrypted inside a volume or share, so combine it with snapshot schedules, replication schedules, multi-factor authentication and audit logging.

=== What happens if nobody approves a request? ===

The delete never runs. The request expires after the configured '''Hours Until Auto-expiration''', and the object stays in place. Setting the expiry to 0 keeps requests open until they are approved or rejected.

=== Is multi-admin approval suitable for government and regulated deployments? ===

It enforces separation of duties for data destruction on the storage system itself, which supports audit and access-control requirements in frameworks such as NIST 800-53 and 800-171. It works alongside QuantaStor's password and lockout policy presets, RBAC, multi-factor authentication, LDAP single sign-on and always-on audit logging, all described in Security Configuration.

=== Can an administrator approve their own delete? ===

Treat approval as needing different people: give each administrator a named account, grant the approve permission only to roles that should vote, and protect those accounts with multi-factor authentication. Check the Request Originator column before approving a request.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-multi-admin-approval-multi-admin-approval-role-permissions.png

Qadmin — Sat, 03 Oct 2026 02:45:02 GMT

Qadmin uploaded File:Guide-multi-admin-approval-multi-admin-approval-role-permissions.png osn-seo-utilities: screenshot for Guides (multi-admin-approval)

New page

osn-seo-utilities: screenshot for Guides (multi-admin-approval)

File:Guide-multi-admin-approval-multi-admin-approval-security-manager.png

Qadmin — Sat, 03 Oct 2026 02:45:02 GMT

Qadmin uploaded File:Guide-multi-admin-approval-multi-admin-approval-security-manager.png osn-seo-utilities: screenshot for Guides (multi-admin-approval)

New page

osn-seo-utilities: screenshot for Guides (multi-admin-approval)

Guides:Index

Qadmin — Sat, 03 Oct 2026 02:30:07 GMT

osn-seo-utilities: regenerate index

← Older revision		Revision as of 02:30, 3 October 2026
Line 3:		Line 3:
	== Guides ==		== Guides ==

			* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

Guides:Asynchronous Remote Replication and Disaster Recovery Failover

Qadmin — Sat, 03 Oct 2026 02:30:03 GMT

osn-seo-utilities: remote-replication-dr @ f2ee090c4281 (approved in the portal)

New page

''By Steve Umbehocker, CTO, OSNexus · Updated October 3, 2026''

Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed. QuantaStor replicates Storage Volumes and Network Shares between Storage Pools on systems in the same grid. When the primary site is lost, you activate the replica checkpoints at the DR site, and later you can fail back with or without the changes made.

== Why it matters ==

Snapshots protect against mistakes, but they live in the same pool as the data, so they go down with the pool, the system or the site. Replication puts a full copy somewhere else. The three scheduled data-protection features answer different questions:

{| class="wikitable"
! Feature !! Where the copy lands !! Use it for
|-
| [[Snapshot Schedules]] || Destination is the same Storage Pool || Fast local recovery points, undoing a bad change
|-
| Remote replication || Destination is a Storage Pool on another system within the grid || Surviving the loss of a system, pool or site
|-
| [[Backup Policies]] || Destination is cloud object storage or an an external NFS/SMB source || Provides great flexibility and you can do inbound or outbound policies and autotiering but it doesn't have the block-level incremental efficiency of the other methods
|}

A replication schedule already snapshots its sources and keeps its own retention on both sides, so you rarely need a separate Snapshot Schedule on the same volume or share.

== How it works ==

Replication is snapshot-driven ZFS send/receive between two pools. The full reference is [[Remote-replication (DR)]]. The moving parts:

* '''Grid membership.''' Both systems must belong to the same storage grid. A grid can span sites, so replicating to a remote site means joining the remote appliances to this grid. See [[Grid Configuration]].
* '''Storage System Replication Link.''' A trust relationship and data path between two systems. Creating one generates a dedicated SSH key pair, registers each side's public key on the other, and pins replication traffic to a fixed IP address on each end. Links come in pairs: A to B creates B to A as well.
* '''Replication schedule.''' Names a link (which sets the direction), a destination pool, the volumes and shares, when to run, and how many snapshots to keep on each side. The system that owns the destination pool owns and runs the schedule.
* '''Replica checkpoint.''' For each source, the destination pool holds a real volume or share named with a <code><nowiki>_chkpnt</nowiki></code> suffix (<code><nowiki>backups</nowiki></code> becomes <code><nowiki>backups_chkpnt</nowiki></code>), plus timestamped snapshots that serve as retained recovery points.
* '''Replica association.''' The persistent source-to-checkpoint relationship, where live status and sync times are recorded.

Only the first run is a full copy. After that, each run snapshots the source, finds the newest GMT-stamped snapshot that both source and checkpoint have, and sends the difference. The pools don't need to match in size, layout, disk type or hardware, and the remote-replication feature must be included in the license.

On a Ceph scale-out cluster, Ceph RBD pools can be a destination only when the remote system belongs to the same Ceph cluster as the source. Network Shares on scale-out CephFS replicate to a second Ceph cluster through CephFS snapshot mirroring instead; see [[Scale-out File Replication]].

== Design and sizing ==

'''RPO.''' On the Schedule Interval tab, a timer interval is the rest time between the ''end'' of one run and the start of the next: 3 minutes minimum, 30 by default, up to 360. Your worst-case data loss is about one interval plus the duration of a run, so the real lever is how long a run takes. A run never starts while the previous one is still working. For a fixed timetable, the day/hour grid gives a calendar schedule, with an offset of 0–59 minutes to stagger several schedules. Frequent small runs usually transfer less each time and keep the DR copy closer than one big nightly run.

'''Bandwidth.''' Each link has a Bandwidth Limit in MB/sec (the dialog suggests 200; entering 0 means the service default of 100). The limit is shared evenly across the streams currently running on that link and rebalanced as streams finish, so four concurrent replications on a 200 MB/sec link get 50 MB/sec each. Size the limit against your daily change rate. This is plain arithmetic, not a measured benchmark:

{| class="wikitable"
! Changed data per run !! Time to send at 100 MB/sec !! Time to send at 200 MB/sec
|-
| 10 GB || about 1.7 minutes || about 50 seconds
|-
| 100 GB || about 17 minutes || about 8.5 minutes
|-
| 1 TB || about 2.8 hours || about 1.4 hours
|}

The first full copy is the long one. Seeding with a one-time replica (below) before you enable a tight schedule keeps the initial copy from colliding with production hours.

'''Transport.''' Encryption is on by default and tunnels the stream over SSH. Turning it off sends the stream over a plain mbuffer TCP session, which is faster on a trusted private link and offers no confidentiality. Compression adds lz4 only when the source dataset isn't already compressed.

'''Retention.''' The Snapshot Settings tab keeps 3 short-term snapshots by default (the recommended value) for computing deltas, plus long-term hourly, daily, weekly, monthly and quarterly counts set separately for source and checkpoint. Keep enough snapshots that both sides always share one; otherwise the next run falls back to a full copy.

'''HA pools.''' Each run re-selects its link based on which systems currently own the source and destination pools. With an HA pool on systems A and B replicating to one on C and D, create all four links (A–C, A–D, B–C, B–D) so replication keeps running after a failover on either end.

== Setting it up ==

# Join both systems to one grid ([[Grid Configuration]]) and create a destination pool ([[Storage Pools]]).
# Create the link: '''Remote Replication → Storage System Replication Links → Replication Link → Create'''. Choose the storage system and a fixed IP address for each side. Floating cluster VIF addresses are filtered out. For a cloud system with a public address, the far side can use '''External IP Address''', which is pre-filled from the system's external hostname (set with <code><nowiki>qs system-modify --ext-hostname</nowiki></code>, see the [[QuantaStor CLI Command Reference]]).

[[File:Guide-remote-replication-dr-remote-replication-dr-replication-link-create.png|frame|center|The Create Storage System Replication Link dialog, with an existing link pair (primary site to DR site) behind it: pick a system and fixed IP address for each end, then check the Bandwidth Limit and the Encryption box, which is on by default]]

# Create the schedule: '''Remote Replication → Volume & Share Replication Schedules → Replication Schedule → Create'''. On the General tab, pick the link (this sets the direction) and the Remote Pool. Then set the interval, select volumes and shares (selecting a parent share and one nested inside it is rejected), set retention, and review Advanced Settings. Leave '''Enable resumable replication''' and '''Enable target checkpoint recovery''' on, as they are by default.
# To seed a destination or copy one volume on demand without a schedule, use [[Create Volume Replica]]. The diff-copy option sends changes against an existing checkpoint; full copy creates a new one.
# Interval schedules start shortly after you create them. To run any schedule immediately:

<pre>qs replication-schedule-trigger --schedule=nightly-dr
qs replica-assoc-list
qs replica-report-summary-list</pre>

== Operating and testing ==

Watch the '''Volume & Share Replica Associations''' section. Each source shows its status (Synchronizing, Synchronized, Sync Failed (Resumable), Skipped and so on), elapsed time, and when the last sync started and completed. Each run also writes a summary report with bytes transferred and average speed, which is the number to check against your RPO math.

[[File:Guide-remote-replication-dr-remote-replication-dr-replica-associations.png|frame|center|The Remote Replica Associations grid after a scheduled run: a volume and a share, both Synchronized, with sync start and completion times and their _chkpnt targets on the DR system]]

[[File:Guide-remote-replication-dr-remote-replication-dr-replication-report.png|frame|center|The Replication Report for the schedule: a full first sync of 1.29 GB, then incremental runs that send only the changed blocks, plus one run skipped because the previous one was still active]]

If a run is interrupted (the WAN drops, a node reboots), resumable replication keeps a ZFS resume token and the next run continues from where it stopped. A failed run raises an alert, at most one per hour per schedule and source; see [[Call-home / Alerting]] for sending alerts off the appliance. A run is skipped, with its reason shown on the schedule, if a link is missing or not Normal, if a pool is unhealthy, or if a destination HA failover is in progress.

'''Failover.''' '''Activate Checkpoints''' on the schedule toolbar is the promotion step. It disables the schedule, marks the checkpoints as Active Replica Checkpoints, brings checkpoint shares online and makes checkpoint volumes available to map to hosts. It can also create share aliases named after the sources, so clients reach <code><nowiki>backups</nowiki></code> rather than <code><nowiki>backups_chkpnt</nowiki></code>. Optional <code><nowiki>dr-prefailover</nowiki></code> and <code><nowiki>dr-postfailover</nowiki></code> scripts in <code><nowiki>/var/opt/osnexus/custom/</nowiki></code> handle site-specific steps such as repointing DNS. For hands-off failover, the Automatic Activation tab activates checkpoints when a Site Cluster VIF ([[Site Cluster Create]]) moves to the destination node.

[[File:Guide-remote-replication-dr-remote-replication-dr-replication-schedule-toolbar.png|frame|center|A replication schedule that runs every 15 minutes and keeps 10 checkpoints. Its toolbar has Activate Checkpoints, Deactivate Checkpoints and Rollback for failover and failback]]

'''Failback.''' After a test, disconnect DR-site clients, then use '''Deactivate Checkpoints''' with '''Re-enable Replication Schedule''' ticked; the next run overwrites the checkpoints from the source. If the DR site took writes you need to keep, use '''Rollback''' first (<code><nowiki>qs replication-schedule-trigger-rollback --schedule=<name></nowiki></code>). It sends the checkpoint's changes back and overwrites the original source. Then deactivate and re-enable.

'''Test it regularly.''' A DR test is activate, mount and verify from a DR-site client, then deactivate. Any client session on a checkpoint marks it active again on its own, and while any checkpoint is active the schedule stays blocked. A test that ends with clients still attached quietly stops replication, so check the schedule's state afterwards.

== FAQ ==

=== Does QuantaStor support asynchronous replication with DR failover between sites? ===

Yes. Replication schedules copy Storage Volumes and Network Shares asynchronously and incrementally to a Storage Pool on another grid member, which can be at another site. Failover is the Activate Checkpoints step, either manual or automatic on a Site Cluster VIF move. Failback either discards the DR-site changes or rolls them back to the source first.

=== What RPO can I get? ===

The shortest timer interval is 3 minutes, measured from the end of one run to the start of the next. Your effective RPO is that interval plus however long a run takes, which depends on change rate and the link's bandwidth limit. Check the average speed in the replication reports to see what you're actually achieving.

=== Do the source and destination systems need identical hardware? ===

No. The pools don't need to match in size, layout, disk type or hardware. Both systems do need to be in the same grid and licensed for remote replication. Ceph RBD destinations must be in the same Ceph cluster as the source.

=== Can one source replicate to more than one site? ===

Yes. Create one schedule per destination (N-way), or replicate from the DR site's checkpoint to a third site (cascading). On the second and later schedules, enable source snapshot reuse so all destinations share a common snapshot.

=== Will replication overwrite changes made at the DR site? ===

Not while a checkpoint is active. An active checkpoint blocks the schedule, and any iSCSI, FC, NFS or SMB client session marks a checkpoint active automatically. Overwriting happens only after you deactivate, so roll back first if you want to keep DR-site writes.

----
''Part of the [[Guides:Index|QuantaStor Guides]] series. For reference documentation, see the [[Main Page|QuantaStor documentation]].''

[[Category:Guides]]

File:Guide-remote-replication-dr-remote-replication-dr-replication-schedule-toolbar.png

Qadmin — Sat, 03 Oct 2026 02:30:03 GMT

Qadmin uploaded File:Guide-remote-replication-dr-remote-replication-dr-replication-schedule-toolbar.png osn-seo-utilities: screenshot for Guides (remote-replication-dr)

New page

osn-seo-utilities: screenshot for Guides (remote-replication-dr)

File:Guide-remote-replication-dr-remote-replication-dr-replication-report.png

Qadmin — Sat, 03 Oct 2026 02:30:03 GMT

Qadmin uploaded File:Guide-remote-replication-dr-remote-replication-dr-replication-report.png osn-seo-utilities: screenshot for Guides (remote-replication-dr)

New page

osn-seo-utilities: screenshot for Guides (remote-replication-dr)

File:Guide-remote-replication-dr-remote-replication-dr-replica-associations.png

Qadmin — Sat, 03 Oct 2026 02:30:03 GMT

Qadmin uploaded File:Guide-remote-replication-dr-remote-replication-dr-replica-associations.png osn-seo-utilities: screenshot for Guides (remote-replication-dr)

New page

osn-seo-utilities: screenshot for Guides (remote-replication-dr)

← Older revision		Revision as of 21:00, 8 October 2026
Line 6:		Line 6:
	* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.		* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[Guides:Enterprise Shared Storage for Proxmox VE Clusters\|Enterprise Shared Storage for Proxmox VE Clusters]]''' — Shared storage for a Proxmox VE cluster is a block or file store that every node can reach, so virtual machines and containers can migrate between nodes and restart elsewhere when a node fails.
	* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).		* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
	* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''		* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''

← Older revision		Revision as of 00:00, 8 October 2026
Line 4:		Line 4:

	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
			* '''[[Ceph over ZFS\|Ceph over ZFS]]''' — [[Category:admin_guide]] Ceph over ZFS runs Ceph OSDs on ZFS Storage Volumes (zvols) instead of directly on Physical Disks.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
	* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).		* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
Line 13:		Line 14:
	* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]		* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
	* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.		* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.
			* '''[[Snapshot Schedules\|Snapshot Schedules]]''' — [[Category:admin_guide]] A '''Snapshot Schedule''' takes point-in-time snapshots of a chosen set of [[Storage Volumes\|Storage Volumes]] and [[Network Shares\|Network Shares]] on a repeating schedule, and rotates them so each volume and share keeps a moving window of recovery points.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->

← Older revision		Revision as of 20:00, 7 October 2026
Line 5:		Line 5:
	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
	* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.		* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[Guides:Fibre Channel and iSCSI Block Storage with ALUA Multipathing\|Fibre Channel and iSCSI Block Storage with ALUA Multipathing]]''' — QuantaStor presents storage volumes as block LUNs over Fibre Channel, through QLogic host bus adapters running in target mode, and over iSCSI, and it reports each path's state to the host with ALUA (Asymmetric Logical Unit Access).
			* '''[[HA Cluster Setup (JBODs)\|HA Cluster Setup (JBODs)]]''' — [[Category:admin_guide]] ''_TOC_''
			* '''[[High-availability VIF Management\|High-availability VIF Management]]''' — Cluster virtual interfaces (VIFs) are floating IP addresses that move between the appliances of a site cluster so that a service address stays reachable when the appliance hosting it fails.
	* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.		* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.
	* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.		* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.

← Older revision		Revision as of 05:45, 7 October 2026
Line 4:		Line 4:

	* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.		* '''[[Guides:Asynchronous Remote Replication and Disaster Recovery Failover\|Asynchronous Remote Replication and Disaster Recovery Failover]]''' — Asynchronous remote replication keeps a second, mountable copy of your volumes and shares at another site, updated on a schedule by sending only the blocks that changed.
			* '''[[Guides:Encryption at Rest and FIPS Mode for On-Premises Storage\|Encryption at Rest and FIPS Mode for On-Premises Storage]]''' — Encryption at rest protects the data on a storage pool's media, so drives that leave the data center, whether failed, stolen or decommissioned, can't be read without the keys.
			* '''[[ISCSI Target Configuration\|ISCSI Target Configuration]]''' — QuantaStor presents each Storage Volume to iSCSI clients as its own target, with its own IQN, on the network ports you allow for iSCSI.
	* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.		* '''[[Guides:Multi-Admin Approval for Destructive Storage Operations\|Multi-Admin Approval for Destructive Storage Operations]]''' — Multi-admin approval is a two-person rule for data-destructive operations: when it is enabled, a delete of a protected object type is held as a pending request until a set number of administrators approve it.
	* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]		* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
			* '''[[Network Ports\|Network Ports]]''' — [[Category:admin_guide]]
			* '''[[Guides:Nightly Database Analysis from iSCSI Snapshots\|Nightly Database Analysis from iSCSI Snapshots]]''' — Offline database analysis means running reports, audits and heavy queries against a point-in-time copy of production data on a separate server, so that the production database never sees the load.
	* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.		* '''[[Guides:Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies\|Tiering On-Premises Object Storage to AWS S3 with Lifecycle Policies]]''' — Cloud tiering moves older objects from a local S3 bucket to a bucket at AWS while the bucket keeps serving the same namespace.

	[[Category:Guides]]		[[Category:Guides]]
	<!-- osn-seo-utilities: generated index -->		<!-- osn-seo-utilities: generated index -->