Storage Pools: Difference between revisions

From OSNEXUS Online Documentation Site
Jump to navigation Jump to search
m Link multi-admin approval to Security Configuration#Multi-admin Approvals
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Category:admin_guide]]
[[Category:admin_guide]]


== Storage Pool Management ==
A Storage Pool aggregates one or more physical disks into a fault-tolerant pool of storage from which you provision [[Storage Volumes|Storage Volumes]] (block/SAN) and [[Network Shares|Network Shares]] (file/NAS). This page covers the scale-up (OpenZFS) Storage Pool: how to choose a layout, how to create and modify a pool, how to add flash offload tiers, and how to import, export, scrub, grow and troubleshoot one.


Storage pools combine or aggregate one or more physical disks (SATA, NL-SAS, NVMe, or SAS SSD) into a pool of fault tolerant storage.  From the storage pool users can provision both SAN/storage volumes (iSCSI/FC targets) and NAS/network shares (CIFS/NFS).  Storage pools can be provisioned using using all major RAID types (RAID0/1/10, single-parity RAIDZ1/5/50, double-parity RAIDZ2/6/60, triple-parity RAIDZ3, and DRAID) and in general OSNEXUS recommends RAIDZ2 4d+2p for the best combination of fault-tolerance and performance for most workloads.  For assistance in selecting the optimal layout we recommend engaging the OSNEXUS SE team to get advice on configuring your system to get the most out of it.
A single pool serves block and file at the same time, and each resource can be reached over several protocols at once -- a Storage Volume over iSCSI and Fibre Channel together, a Network Share over NFS and SMB together. Scale-out (Ceph) pools are a different object type and are configured under the Scale-out Storage Configuration tab; everything below applies to scale-up pools, which the interface labels '''Scale-up (ZFS)'''.


==== Pool RAID Layout Selection / General Guidelines ====
[[File:pool_section.png|thumb|right|800px|The Storage Pools section, with the pool grid, the four detail tabs and the Storage Pool toolbar group.]]
We strongly recommend using RAIDZ2 4d+2p or RAID10/mirroring for all virtualization workloads and databases.   RAIDZ3 or RAIDZ2 are ideal for archive applications that require higher-capacity for applications that produce mostly sequential IO patterns.  RAID10 will maximize random IOPS but is more expensive since usable capacity before compression is 50% of the raw capacity or 33% with triple mirroring.  Compression is the enabled by default, and can deliver performance gains for some workloads in addition to increasing usable capacity.  We recommend leaving it enabled unless you're certain the data will be uncompressible such as with pre-encrypted data and pre-compressed video/image formats.
For archival storage or other similar workloads RAIDZ2 or RAIDZ3 is best and provides higher usable capacity.  Single-parity (RAIDZ1) is not sufficiently fault tolerant and is not supported in production deployments.  RAID0 provides no fault-tolerance or bit-rot detection and is also not supported for use in production deployments.


[[File:Storage Pools Web 6.jpg|1024px]]
{| class="wikitable"
! Section !! Purpose
|-
| [[#Choosing a RAID layout|Choosing a RAID layout]] || Which layout to use, and the disk counts each one needs.
|-
| [[#Creating a Storage Pool|Creating a Storage Pool]] || Every field on all three tabs of the Create dialog.
|-
| [[#Modifying a Storage Pool|Modifying a Storage Pool]] || Compression, sync policy, cache policy, hot-spare policy, I/O profile.
|-
| [[#Device groups, offload tiers and hot-spares|Device groups, offload tiers and hot-spares]] || VDEVs, read cache, write log, metadata and dedup offload, spares.
|-
| [[#Growing and expanding a pool|Growing and expanding a pool]] || The difference between Grow and Expand, and the disk-count rule.
|-
| [[#Starting, stopping and rescanning|Starting, stopping and rescanning]] || Taking a pool offline and bringing it back.
|-
| [[#Importing and exporting pools|Importing and exporting pools]] || Moving a pool between appliances, and what export removes.
|-
| [[#Scrubbing, trim and health|Scrubbing, trim and health]] || Bit-rot verification, scheduled maintenance, capacity alerting.
|-
| [[#Pool checkpoints|Pool checkpoints]] || A whole-pool rollback point for upgrade safety.
|-
| [[#Encryption|Encryption]] || Software and hardware encryption, passphrases, key export and re-key.
|-
| [[#Deleting a Storage Pool|Deleting a Storage Pool]] || Deletion and the secure data shredding modes.
|-
| [[#Replacing and repairing devices|Replacing and repairing devices]] || Attach, detach, replace, online and offline.
|-
| [[#Troubleshooting|Troubleshooting]] || Degraded and failed pools, slow rebuilds, inconsistent performance.
|-
| [[#CLI command reference|CLI command reference]] || Every <code>qs</code> pool command, with the operations that are CLI only.
|}


===Creating Storage Pools===
== Choosing a RAID layout ==


'''Navigation:''' Storage Management --> Storage Pools --> Storage Pool --> Create (toolbar)
A pool is built from one or more device groups -- VDEVs in OpenZFS terms, and '''Storage Pool Device Groups''' in QuantaStor. Each group carries its own redundancy, and the pool stripes across the groups. Fault tolerance is per group: a double-parity group survives two failed members, and a third failure in that same group loses the pool regardless of how healthy the other groups are.


[[File:Create Storage Pool - Gerneral 5.4.jpg|412px|thumb|Create a Storage Pool.]]
We recommend '''Double-parity (RAIDZ2+0)''' with groups of six disks (4 data + 2 parity) as the best balance of fault tolerance, performance and usable capacity for most workloads, and it is what the Create dialog defaults to when enough disks are available. Use '''Mirrored (RAID10)''' for virtualization and databases where random IOPS matter most. Use '''Triple-parity (RAIDZ3+0)''' or double-parity for archive and backup, where capacity and sequential throughput matter more than IOPS.


One of the first steps in configuring an system is to create a Storage Pool.  The Storage Pool is an aggregation of one or more devices into a fault-tolerant "pool" of storage that will continue operating without interruption in the event of disk failures.  The amount of fault-tolerance is dependent on multiple factors including hardware type, RAID layout selection and pool configuration.  QuantaStor's block storage (Storage Volumes) as well as file storage (Network Shares) are both provisioned from ''Storage Pools''.  A given pool of storage can be used for block (SAN) and file (NAS) storage use cases at the same time.  Additionally, clients can access the storage using multiple protocols at the same time.  For example Storage Volumes may be accessed via iSCSI & FC at the same time and Network Shares may be accessed via NFS & SMB at the same time.  Creating a storage pool is very straight forward, simply name the pool (eg: pool-1), select disks and a layout (Double-Parity recommended for most use cases) for the pool, and then press OK.  If you're not familiar with RAID levels then choose the Double-Parity layout and select devices in groups of 6x (4d+2p). There are a number of advanced options available during pool creation but the most important one to consider is encryption as this cannot be turned on after the pool is created.
Single-parity (RAIDZ1) is not sufficiently fault tolerant for production, and RAID0 has no redundancy and no bit-rot repair at all -- the Create dialog raises an explicit confirmation before it will build a RAID0 pool. Neither is supported for production deployments.


====Enclosure Aware Intelligent Disk Selection====
{| class="wikitable"
For systems employing multiple JBOD disk enclosures QuantaStor automatically detects which enclosure each disk is sourced from and selects disks in a spanning "round-robin" fashion during the pool provisioning process to ensure fault-tolerance at the JBOD level. This means that should a JBOD be powered off the pool will continue to operate in a degraded state.   If the detected number of enclosures is insufficient for JBOD fault-tolerance the disk selection algorithm switches to a sequential selection mode which groups vdevs by enclosure. For example a pool with RAID10 layout provisioned from disks from two or more JBOD units will use the "round-robin" technique so that mirror pairs span the JBODs.  In contrast a storage pool with the RAID50 layout (4d+1p) with two JBOD/disk enclosures will use the sequential selection mode.
! Layout (as labelled in the dialog) !! Minimum disks offered from !! Notes
|-
| Striped (RAID0) || 1 || No redundancy, no self-healing. A single device failure loses the pool.
|-
| Mirrored (RAID10) || 2 || Two-way or three-way mirrors. Highest random IOPS; usable capacity is 50% of raw (33% with three-way).
|-
| Single-parity (RAIDZ1+0) || 3 || One parity block per stripe. Not supported for production.
|-
| Double-parity (RAIDZ2+0) || 4 || Two parity blocks per stripe. Recommended default.
|-
| Triple-parity (RAIDZ3+0) || 7 || Three parity blocks per stripe. Best for long-term archive.
|-
| Distributed Single-parity (DRAID1 1VS/2VS/3VS) || 10 || Distributed RAID with 1, 2 or 3 distributed spares.
|-
| Distributed Double-parity (DRAID2 1VS/2VS/3VS) || 16 || As above with two parity blocks per stripe.
|-
| Distributed Triple-parity (DRAID3 1VS/2VS/3VS) || 20 || As above with three parity blocks per stripe.
|}


====Enabling Encryption (One-click Encryption)====
The '''RAID Type''' list is built from the number of disks currently ''available'' on the selected Storage System, not the number you have selected. If a layout you expect is missing from the list, the appliance does not have enough unused disks to offer it -- the distributed RAID layouts in particular need at least 10, 16 or 20 available disks depending on parity level. Distributed RAID rebuilds far faster than parity RAID because the spare capacity is spread across every member device, which is why we recommend it for the large 60-, 84-, 90- and 106-bay JBOD backup and archive configurations where a conventional rebuild would run for days.
[[File:Create Storage Pool - Encryption 5.5.jpg|412px|thumb|Enable encryption.]]


Encryption must be turned on at the time the pool is created.  If you need encryption and you currently have a storage pool that is un-encrypted you'll need to copy/migrate the storage to an encrypted storage pool as the data cannot be encrypted in-place after the fact.  Other options like compression, cache sync policy, and the pool IO tuning policy can be changed at any time via the ''Modify Storage Pool'' dialog. To enable encryption select the Advanced Settings tab within the Create Storage Pool dialog and check the ''[x] Enable Encryption'' checkbox.
Build more than one pool where the workloads differ: low-cost fault-tolerant capacity in RAIDZ2 for backup and archive, and a mirrored pool for virtual machines, containers and databases. See [[Provisioning Tiers]] for grouping several pools behind one provisioning target.


=== Enclosure-aware disk selection ===


=====Passphrase protecting Pool Encryption Keys=====
On a system with more than one JBOD, QuantaStor detects which enclosure each disk came from and selects disks round-robin across the enclosures so that the members of each device group land in different chassis. A pool built that way keeps running in a degraded state if a whole JBOD is powered off.


For QuantaStor used in portable systems or in high-security environments we recommend assigning a passphrase to the encrypted pool which will be used to encrypt the pool's keys. The drawback of assigning a passphrase is that it will prevent the system from automatically starting the storage pool after a reboot.   Passphrase protected pools require that an administrator login to the system  and choose 'Start Storage Pool..' where the passphrase may be manually entered and the pool subsequently started.  If you need the encrypted pool to start automatically or to be used in an HA configuration, do not assign a passphrase.  If you have set a passphrase by mistake or would like to change it you can change it at any time using the ''Change/Clear Pool Passphrase...'' dialog at any time.  Note that passphrases must be at least 10 characters in length and no more than 80 and may comprise alpha-numeric characters as well as these basic symbols: "'''.-:_'''"  (period, hyphen, colon, underscore).
Round-robin selection is only used when there are enough enclosures for the layout to actually benefit -- the requirement scales with the group size, so a mirrored pool needs as many enclosures as the mirror is wide, while a RAIDZ1 pool with five-disk groups needs five. When there are too few enclosures the selection falls back to a sequential mode that groups each device group within one enclosure. Within an enclosure the selection also prefers disks of the same size, so a 2&nbsp;TB disk is never mirrored against a 6&nbsp;TB one.


====Enabling Compression====
== Creating a Storage Pool ==
[[File:Create Storage Pool - Adv Settings 5.4.jpg|412px|thumb|Control use of I/O tuning and or compression.]]


Pool compression is enabled by default as it typically increases usable capacity and boosts read/write performance at the same time.  This boost in performance is due to the fact that modern CPUs can compress data much faster than the media can read/write data.  The reduction in the amount of data to be read/written due to compression subsequently boosts performance.  For workloads which are working with compressed data (common in M&E) we recommend turning compression off.  The default compression mode is LZ4 but this can be changed at any time via the ''Modify Storage Pool'' dialog.
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Create ''(toolbar)''}}


=== Boosting Performance with SSD Caching ===
Creating the first pool is normally the first configuration step on a new appliance. Name the pool, pick the disks and a layout, and click '''OK'''. The one setting you cannot change afterwards is encryption -- decide that before you create the pool, because there is no way to encrypt a pool in place later.
ZFS based storage pools support the addition of SSD devices for use as read cache or write log. SSD cache devices must be dedicated to a specific storage pool and cannot be shared across multiple storage pools.  Some hardware RAID controllers support SSD caching but in our testing we've found that ZFS is more effective at managing the layers of cache than the RAID controllers so we do not recommend using SSD caching at the hardware RAID controller.


[[File:Add Cache Device Web 6.jpg|1024px]]
The dialog is a three-tab wizard; '''Previous''' and '''Next''' step through the tabs and '''OK''' submits from any of them.


==== Accelerating All Performance with SSD Meta-data & Small File Offload (ZFS special VDEVs) ====
=== General tab ===
Scale-up Storage Pools using HDDs should always include 3x SSDs for metadata offload and small file offload.  This feature of the filesystem is so powerful and impactful to improving performance for scrubs, replication, HA failover, small-file performance and more that we do not design any HDD based pools without it.  All-flash storage pools do not need it and should not use it unless you're looking to optimize the use of QLC media.  QLC media is not ideal for small block IO so using TLC media to offload the filesystem metadata and small files is a good idea.  That would be the only use case for using metadata-offload SSDs with an all-flash storage pool.


==== Accelerating Write Performance with SSD Write Log (ZFS SLOG/ZIL) ====
[[File:pool_create_general.png|thumb|right|800px|The General tab of Create Storage Pool: name, layout, set size and disk selection.]]
Scale-up Storage Pools use the OpenZFS filesystem which has the option of adding an external log device (SLOG/ZIL) rather than using the primary data VDEVs for logging.  Modern deployments use the SLOG/ZIL write-log offload less and less so we don't recommend it for most deployments.  This is because the feature is generally used to boost the performance of HDD based pools that are being used for database and VM workloads.  Modern deployments should not use HDD based pools for database and VM use cases.  Now that NVMe SSDs and SAS SSDs are much more affordable than they were from 2010-2020 it is best to make an all-flash pool for all workloads that are IOPS intensive. Interestingly with new types of NVMe entering the market there may be a renewed use of SLOG/ZIL in the future to optimize the use of QLC NVMe devices with TLC NVMe media.  Note that the default 'sync' mode for Storage Pools is the 'standard' mode which bypasses the SLOG/ZIL unless the write is using the O_SYNC flag which is used by hypervisors and databases.  If you're using a SLOG/ZIL for other workloads you'll notice that it is getting bypassed all the time.  In such cases you can change the 'sync' mode to 'always' as a way of forcing the write-log to be used for all IO but note that if your log device is not fast enough you may actually see a performance decrease.  That said, with dual-ported NVMe based HA configurations with NVMe media that can do 4GB/sec and faster, the 'sync=always' pool sync policy may come in vogue again.  Note also that multiple pairs of SLOG/ZIL devices can be added to boost log performance.  For example, if you add 4x 3.84TB NVMe devices you'll have two mirror pairs and if they can each do 4GB/sec then in aggregate you'll have a 8GB/sec log.  If you have a HDD based pool then such a log would be able to ingest and coalesce 80x HDDs worth of write IO and make the pool usable for a broader set of use cases.


==== Accelerating Read Performance with SSD Read Cache (L2ARC) ====
{| class="wikitable"
You can add up to 4x devices for SSD read-cache (L2ARC) to any ZFS based storage pool and these devices do not need to be fault tolerant. You can add up to 4x devices directly to the storage pool by selecting 'Add Cache Devices..' after right-clicking on any storage pool. You can also opt to create a RAID0 logical device using the RAID controller out of multiple SSD devices and then add this device to the storage pool as SSD cache. The size of the SSD Cache should be roughly the size of the working set for your application, database, or VMs. For most applications a pair of 400GB SSD drives will be sufficient but for larger configurations you may want to use upwards of 2TB or more of SSD read cache. Note that the SSD read-cache doesn't provide an immediate performance boost because it takes time for it to learn which blocks of data should to be cached to provide better read performance.
! Field !! Default !! Notes
|-
| '''Name''' || <code>pool-1</code> (next unused <code>pool-''n''</code>) || 2 to 40 characters, letters, digits and <code>. - _</code> only. No spaces. Invalid characters are dropped as you type. The name is a label only -- see [[#What a pool looks like on disk|What a pool looks like on disk]].
|-
| '''Description''' || empty || Free text.
|-
| '''RAID Type''' || Double-parity (RAIDZ2+0) with 6 or more available disks, otherwise Mirrored (RAID10), otherwise Striped (RAID0) || See [[#Choosing a RAID layout|Choosing a RAID layout]]. The list is filtered by available disk count.
|-
| '''Set Size''' || Auto || Number of disks in each device group. Auto picks a sensible group width for the layout. Disabled for RAID0.
|-
| '''+''' and '''-''' || || Select or deselect exactly one more device group's worth of disks, spread across enclosures. Disabled for the distributed RAID layouts, which build a single group.
|-
| '''Auto Config''' || || Select as many whole device groups as the available disks allow. Also disabled for distributed RAID.
|-
| '''Storage System''' || the system you launched the dialog from || Which grid member's disks to build from. A pool is owned by one system.
|-
| Disk grid || none selected || Only unused, usable disks appear. Boot and system disks, disks already in a pool, journal members, Ceph OSD disks, multipath child paths and anything under 50&nbsp;MB are filtered out. '''Search''' filters across every column; commas separate multiple terms.
|-
| '''Force''' || unchecked || Scrubs existing partitions off the selected disks. Required when the disks carry an old ZFS label or LUKS header from a previously exported pool, and when the selected disks do not all report the same physical block size.
|}


==== RAM Read Cache Configuration (ARC) ====
Set Size is what turns a parity layout into a striped set of parity groups. A RAIDZ1+0 pool built from eight disks with a set size of four is two four-disk RAIDZ1 groups striped together. Choosing a set size that the selected disk count is not an exact multiple of is rejected -- for mirroring the dialog itself tells you how many more devices to select, and for the parity layouts the appliance returns the error.
ZFS based storage pools use what are called "ARC" as a in-memory read cache rather than the Linux filesystem buffer cache to boost disk read performance.  Having a good amount of RAM in your system is critical to deliver solid performance. It is very common with disk systems for blocks to be read multiple times.  The frequently accessed "hot data" is cached into RAM where it can serve read requests orders of magnitude faster than reading it from spinning disk. Since the cache takes on some of the load it also reduces the load on the disks and this too leads to additional boosts in read and write performance.  It is recommended to have a minimum of 32GB to 64GB of RAM in small systems, 96GB to 128GB of RAM for medium sized systems and 256GB or more in large systems. Our [https://www.osnexus.com/zfs-designer design tools] can help with solution design recommendations here.


=== Storage Pool Layout Types (ZFS based) ===
=== Encryption tab ===


'''Navigation:''' Storage Management --> Storage Pools --> Storage Pool --> Create (toolbar)
[[File:pool_create_encryption.png|thumb|right|800px|The Encryption tab with software encryption and a user-defined passphrase selected. Encrypt and Store on Key Server is unavailable until a Key Server Profile exists.]]


[[File:Create Storage Pool RAID options.jpg|412px|thumb|Raid options displayed are dependent on the number of Pools available.]]
All Storage Pool encryption is AES-256. Pick one of three encryption types:


QuantaStor supports all industry standard RAID levels (RAID1/10/5(Z1)/50/6(Z2)/60), some additional advanced RAID levels (RAID1 triple copy, RAIDZ3 triple-parity), and simple striping with RAID0.  Over time all disk media degrades and as such we recommend marking at least one device as a hot-spare disk so that the system can automatically heal itself when a bad device needs replacing. One can assign hot-spare disks as universal spares for use with any storage pools as well as pinning of hot-spares to specific storage pools. Finally, RAID0 is not fault-tolerant at all but it is your only choice if you have only one disk and it can be useful in some scenarios where fault-tolerance is not required.  Here's a breakdown of the various RAID types and their pros & cons.
* '''No Encryption''' -- the default.
* '''Software Encryption''' -- LUKS/dm-crypt full-disk encryption applied to every device in the pool. AES-NI hardware acceleration on modern server processors carries the cost, so the overhead is small.
* '''Hardware Encryption''' -- self-encrypting drive (SED) encryption, handled by the drives themselves. Every disk in the pool has to be SED capable; the '''SED Capable''' and '''SED Status''' columns in the disk grid tell you which are.


* Striping '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion.  This has the effect of greatly boosting performance.  The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost.  As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.
'''If the array presenting the devices already encrypts at rest, QuantaStor should not encrypt them again.''' Systems such as the Seagate Corvault, Seagate Exos E/EP and Dell PowerVault families encrypt in hardware and present logical devices; layering software encryption over them costs throughput and protects data that is already protected. QuantaStor recognises these devices by vendor and model and '''skips its own encryption automatically''' -- see [[Encryption Bypass]], which governs scale-up Storage Pool devices as well as scale-out OSDs. The choice is fixed once a device is provisioned, so check that your media is listed there before you create the pool.
* Mirroring '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data.  If one drive fails, the other has a complete copy and the storage pool continues to run.  RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.
* Striped-Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors.  This gives you the fault tolerance of RAID1 combined with the performance of RAID10.  The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing.  If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.
* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via a single parity block per data stripe. It's fast but not fault-tolerant enough for production workloads and should be avoided.
* Double-parity '''RAID60/RAIDZ2''' provides two parity blocks per data stripe and provides robust throughput and IOPS performance. It is our recommended layout for most workloads and is the default in QuantaStor.
* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive and backup applications as it can sustain three simultaneous disk failures per device group (VDEV).
* Distributed-RAID '''DRAID''' This layout is a variation of the Double-parity and Triple-parity layouts that provides much faster rebuilds by distributing the data and parity information over a much larger group of devices.  We recommend this on systems with multiple large 60, 84, and 90, and 106 bay JBODs used for backup and archive use cases.


It can be useful to create more than one storage pool so that you have low cost fault-tolerant storage available in RAIDZ2/60 for backup and archive and higher IOPS storage in RAID10/mirrored layout for virtual machines, containers and databases.
With either encryption type enabled, the '''Key Protection''' group chooses where the key that protects the pool key lives:


===Identifying Storage Pools===
* '''Encrypt with No Passphrase''' (default) -- the pool unlocks and starts automatically at boot with no operator action. Use this for anything in an HA group.
* '''Encrypt with User Defined Passphrase''' -- a passphrase of 10 to 80 characters, from letters, digits and <code>. - : _</code>. Enter it twice; the eye button reveals what you typed.
* '''Encrypt and Store on Key Server''' -- store the key on an external KMIP or IBM Key Protect key server. This option stays greyed out until at least one Key Server Profile has been created, and then '''Key Server Profile''' selects which one to use.


'''Navigation:''' Storage Management --> Controllers & Enclosures --> Identify ''(rightclick)''
A passphrase stops the pool from starting on its own. After a reboot a passphrase-protected pool stays offline and locked until an administrator runs '''Start Storage Pool''' and supplies it. That is exactly what you want on a portable or high-security system, and exactly what you do not want on an HA pool, where the automatic failover has no way to prompt for it. See [[#Encryption|Encryption]] below for changing or clearing a passphrase after the fact.
<br>or<br>
'''Navigation:''' Storage Management --> Storage Pools --> Identify ''(toolbar)''


[[File:Identify Hardware Controller Device Web 6.jpg|1024px]]
=== Advanced Settings tab ===


The group of disk devices which comprise a given storage pool can be easily identified within a rack by using the [[Storage_Pool_Identify | Storage Pool Identify]] dialog which will blink the LEDs in a pattern.
[[File:pool_create_advanced.png|thumb|right|800px|The Advanced Settings tab: I/O profile, compression, auto-trim, ashift and use case.]]


===Deleting Storage Pools===
{| class="wikitable"
! Field !! Default !! Notes
|-
| '''I/O Profile''' || Default || Tunes the member disks for a workload -- read-ahead, queue depth and I/O scheduler. See [[#I/O profiles|I/O profiles]].
|-
| '''I/O Profile Description''' || || Read-only. Shows the selected profile's description.
|-
| '''Enable Compression''' || '''checked''' || Turns on LZ4 compression. Leave it on unless you are certain the data is incompressible.
|-
| '''Enable SSD Auto Trim''' || unchecked || Sends TRIM to flash media continuously so it can reclaim freed blocks. Only useful on a pool built from SSD or NVMe media.
|-
| '''Block Size Offset (ashift)''' || Auto || <code>Auto</code>, <code>9 (512 bytes)</code> or <code>12 (4K bytes)</code>. Leave it on Auto so the appliance derives it from the reported physical block size of the disks. ashift is fixed for the life of the pool, and setting it too small costs you performance while setting it too large wastes capacity on small blocks.
|-
| '''Use Case''' || Default || <code>Default</code> provisions Storage Volumes and Network Shares as usual. <code>Ceph OSD</code> reserves the pool for zvols that back Ceph OSDs, and volumes created in such a pool cannot be presented to hosts over iSCSI, FC or NVMe-oF. This is set at create time only and cannot be changed later.
|}


'''Navigation:''' Storage Management --> Storage Pools --> Storage Pool --> Delete ''(toolbar)''
Compression is on by default because it usually increases both usable capacity and throughput at once -- a modern CPU compresses faster than the media can read or write, so moving less data is a net win. Turn it off for workloads that are already compressed, which is common in media and entertainment. The algorithm can be changed at any time from '''Modify Storage Pool'''; only the on/off choice is available at create time.


Storage pool deletion is final so be careful and double check to make the correct pool is selected.  For secure deletion of storage pools please select one of the data scrub options such as the default 4-pass DoD (Department of Defense) procedure.
=== What a pool looks like on disk ===


[[File:Delete a Storage Pool.jpg|512px]]
The pool's name in the interface is a label. On the appliance the pool is always named <code>qs-''&lt;pool-uuid&gt;''</code>, mounted at <code>/mnt/storage-pools/qs-''&lt;pool-uuid&gt;''</code>, and the friendly name is carried in a <code>quantastor:name</code> property on the pool. That is why renaming a pool is instant and why the name survives a move to another appliance.


====Data Shredding Options====
Two directories are maintained inside every pool:
[[File:Delete Storage Pool - Shred.jpg|512px|thumb|Choose a shred option.]]


QuantaStor uses the Linux ''scrub'' utility which is compliant to various government standards to securely shred data on devices. QuantaStor provides three data scrub procedure options including DoD, NNSA, and USARMY data scrub modes:
* {{Code|1=.poolmeta/}} -- a metadata archive describing the Storage Volumes, Network Shares, hosts, portal groups and schedule associations in the pool. It is refreshed periodically and always rewritten immediately before an export, and it is what an import replays to bring those objects back.
* {{Code|1=.qsbackups/}} -- rolling hourly and daily backups of the appliance configuration database.


* '''DoD mode (default)'''
== Modifying a Storage Pool ==
** 4-pass DoD 5220.22-M section 8-306 procedure (d) for sanitizing removable and non-removable rigid disks which requires overwriting all addressable locations with a character, its complement, a random character, then verify. **
* '''NNSA mode'''
** 4-pass NNSA Policy Letter NAP-14.1-C (XVI-8) for sanitizing removable and non-removable hard disks, which requires overwriting all locations with a pseudorandom pattern twice and then with a known pattern: random(x2), 0x00, verify. **
* '''US ARMY mode'''
** US Army AR380-19 method: 0x00, 0xff, random. **
* '''Fill-zeros'''
** Fills with zeros with a single-pass procedure
* '''Random-data'''
** Fills with random-data using a single-pass procedure


''** Note: short descriptions of the QuantaStor supported scrub procedures are borrowed from the [http://manpages.ubuntu.com/manpages/zesty/man1/scrub.1.html scrub utility manual pages which can be found here].''
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Modify ''(toolbar)''}}


===Importing Storage Pools===
[[File:pool_modify.png|thumb|right|558px|Modify Storage Pool. Everything except encryption and the on-disk layout can be changed here at any time.]]


Storage pools can be physically moved from system to another system by moving all of the disks associated with that given pool from the old system to a new system . After the devices have been moved over one should use the  ''Scan for Disks...'' option in the web UI so that the devices are immediately discovered by the system .  After that the storage pool may be imported using the ''Import Pool...'' dialog in the web UI or via QS CLI/REST API commands.
Everything on this dialog can be changed on a running pool with volumes and shares in it. Encryption is the one pool property that is not here, because it cannot be changed after creation.


====Importing Encrypted Storage Pools====
{| class="wikitable"
! Field !! Default !! Notes
|-
| '''Storage Pool''' || the selected pool || Switch to a different pool without reopening the dialog.
|-
| '''Name''', '''Description''' || current values || Renaming is metadata only and does not disturb I/O.
|-
| '''Hot-Spare Policy''' || Autoselect best match from assigned or universal spares || See [[#Hot-spare policies|Hot-spare policies]].
|-
| '''I/O Profile''' || the pool's current profile || See [[#I/O profiles|I/O profiles]].
|-
| '''Sync Policy''' || <code>standard</code> || <code>standard</code> honours the application's <code>O_SYNC</code> flag, so hypervisors and databases get write-through and everything else gets write-back. <code>always</code> forces every write through the write log. <code>disabled</code> makes all writes asynchronous.
|-
| '''Cache Policy Primary''' || <code>all</code> || What the RAM read cache (ARC) holds: <code>all</code>, <code>metadata</code> only, or <code>none</code>.
|-
| '''Cache Policy Secondary''' || <code>all</code> || The same choice for the SSD read cache (L2ARC).
|-
| '''Small Block Offload''' || Disabled || Maximum write size routed to the metadata offload device group. Only meaningful once a metadata offload group exists. We recommend 8K to 32K depending on how much SSD capacity you gave it.
|-
| '''Compression''' || <code>on</code> || <code>on</code> (LZ4), <code>off</code>, or a specific algorithm: <code>lz4</code>, <code>lzjb</code>, <code>zle</code>, <code>gzip</code> and <code>gzip-1</code> through <code>gzip-9</code>, <code>zstd</code> and <code>zstd-1</code> through <code>zstd-19</code>, and <code>zstd-fast</code> and <code>zstd-fast-1</code> through <code>zstd-fast-10</code>.
|-
| '''Enable SSD Auto Trim''' || the pool's current setting || As on the Create dialog.
|-
| '''Encryption Key Settings''' || Encrypt and Store Locally || Enabled only on an encrypted pool. Moves the pool key between local storage and a key server. The key-server option needs a Key Server Profile to exist.
|}


The keys for the pool (qs-xxxx.key) must be available in the /etc/cryptconf/keys directory in order for the pool to be imported.  Additionally, there is an XML file for each pool which contains information about which devices comprise the pool and their serial numbers which is located in the /etc/cryptconf/metadata directory which is called (qs-xxxx.metadata) where xxxx is the UUID of the pool.  We highly recommend making a backup of the /etc/cryptconf folder so that one has a backup copy of the encryption keys for all encrypted storage pools on a given system.  With these requirements met, the encrypted pool may be imported via the Web Management interface using the Import Storage Pools dialog the same one used do for non-encrypted storage pools.
Changing compression affects new writes only; existing blocks keep whatever they were written with. Changing '''Sync Policy''' to <code>disabled</code> raises a confirmation, and rightly so -- it acknowledges writes before they are durable and can cause data loss on an unexpected power failure. It is not supported for production use.


====Importing 3rd-party OpenZFS Storage Pools====
The '''Cache Policy''' settings are the lever for a workload that pollutes the cache rather than benefiting from it -- a large sequential backup ingest, for example, can evict the working set of everything else on the appliance. Setting the primary cache policy to <code>metadata</code> keeps the pool's directory structure cached while leaving the data blocks out of RAM.


Importing OpenZFS based storage pools from other servers and platforms (Illumos/FreeBSD/ZoL) is made easy with the Import Storage Pool dialog.  QuantaStor uses globally unique identifiers (UUIDs) to identify storage pools (zpools) and storage volumes (zvols) so after the import is complete one would notice that the system will have made some adjustments as part of the import process.  Use the '''Scan''' button to scan the available disks to search for pools that are available to be imported.  Select the pools to be imported and press OK to complete the process.
=== I/O profiles ===


[[File:Import Storage Pools- Web.jpg|924px]]
An I/O profile is a bundle of block-device tunings applied to the pool's member disks -- read-ahead size, queue depth, I/O scheduler, and the SCST target driver thread and tasklet counts -- with separate values for HDD, SSD and NVMe media. It is not a filesystem setting, and it is re-applied every time the pool starts, on HA failover, and after a grow, attach or replace.


For more complex import scenarios there is a CLI level utility that can be used to do the pool import which is documented here [[Console Level OpenZFS Storage Pool Importing]].
Seven profiles ship with the appliance:


===Hot-Spare Management Policies===
{| class="wikitable"
Select ''Recovery'' from the '''Storage Pool''' toolbar or right click on a Storage Pool from the the tree view and click on ''Recover Storage Pool / Add Spares...''.
! Profile !! Optimizes for
|-
[[file:Add Hot Spare - Web.jpg|800px]]
| Default || General purpose server application workloads.
|-
| Virtualization || Virtual machine workloads such as database and web servers under ESXi, XenServer, Hyper-V or Virtuozzo.
|-
| Disk Archive || Disk-to-disk backup and archive applications.
|-
| Media Post-Production (Ingest Optimized) || Media post-production editing.
|-
| Media Post-Production (Playback Optimized) || Media post-production playback.
|-
| Edgeware IP TV Optimized || The Edgeware IP TV application workflow.
|-
| Edgeware Web TV Optimized || The Edgeware Web TV application workflow.
|}


Hot Spares can also be selected from the Modify Storage Pool dialog by clicking on ''Modify'' in the '''Storage Pool''' toolbar.
The profiles are defined in {{Code|1=/opt/osnexus/quantastor/conf/qs_io_profiles.conf}} on the appliance; use <code>[[QuantaStor CLI Command Reference#pool-profile-list|qs pool-profile-list]]</code> and <code>[[QuantaStor CLI Command Reference#pool-profile-get|qs pool-profile-get]]</code> to read the values a profile actually applies. Profiles are per pool; the system-wide OpenZFS module parameters are a separate mechanism, covered in [[Storage System Optimization]]. For choosing between them, and for RAM sizing, see [[Performance Tuning]].


[[file:Mdfy Stor Pool Hot Spare - Web.jpg|800px]]
== Device groups, offload tiers and hot-spares ==


[[file:Mdfy Stor Pool - Hot Spare Plcy.jpg|512px|thumb|Choose a management policy for the Hot Spares.]]
[[File:pool_devices.png|thumb|right|800px|The Storage Pool Devices tab. The RAID Segment column names the device group each disk belongs to -- a data group, cache, or logs.]]


Modern versions of QuantaStor include additional options for how Hot Spares are automatically chosen at the time a rebuild needs to occur to replace a faulted disk. Policies can be chosen on a per Storage Pool basis and include:
The four detail tabs under the pool grid show the pool from different angles. '''Storage Pool Device Groups''' lists the VDEVs with their RAID level and size, and '''Storage Pool Devices''' lists the individual member disks with the group each one belongs to in the '''RAID Segment''' column. A data group appears as <code>mirror-0</code>, <code>raidz2-0</code> and so on; the offload tiers appear as <code>cache</code> and <code>logs</code>.


* Auto-select best match from assigned or universal spares (default)
The number of data device groups matters as much as the layout. A pool with a single group has no way to spread load while that group is rebuilding, so build every pool with at least two, and prefer four or more for anything IOPS sensitive.
* Auto-select best match from pool assigned spares only
* Auto-select exact match from assigned or universal spares
* Auto-select exact match from pool assigned spares only
* Manual Hot-spare Management Only


=== Adding offload devices ===


If the policy is set to one that includes 'exact match', the Storage Pool will first attempt to replace the failed data drive with a disk that is of the same model and capacity before trying other options.  The ''Manual Hot-spare Management Only'' mode will disable QuantaStor's hot-spare management system for the pool.  This is useful if there are manual/custom reconfiguration steps being run by an administrator via a root SSH session.
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool ''(select + right-click)'' &rarr; Add Log/Cache/Metadata Devices}}


===Troubleshooting===
[[File:pool_addcache_metadata.png|thumb|right|800px|The Metadata Offload tab of Add Log/Cache/Metadata Device(s). The four tabs cover write log, read cache, metadata offload and dedup offload.]]


==== Degraded Pool ====
This dialog has one tab per offload class. Move disks from '''Available Disks''' into the selected list with the arrow buttons, or use '''Auto Config''' to pick a matched pair. Offload devices are dedicated to one pool and cannot be shared between pools.
When a pool becomes '''degraded''' this state indicates more media devices has failed or become disconnected from the system.  If the pool has an assigned hot-spare after a minute it will try to repair the pool using the spare.  If the system has no hot-spares add a device that is the same size (or larger) and type as the failed device and mark it as a hot-spare in the Physical Disks section. QuantaStor will automatically inject the spare device into the pool to start the resilvering/rebuild process to bring the pool back to full health.  


==== Rebuild Taking Long Time ====
{| class="wikitable"
The amount of time it takes to heal a pool varies based on the speed and type of the media (HDD, SSD, NVMe) how much load is on the system and how many vdevs you have in the pool. All storage pools should be comprised of at least two VDEVs so that the pool can better manage workloads during drive rebuilds. If the pool is HDD based we highly recommend adding 3x SSDs/NVMe for meta-data offload. Rebuilds are very metadata intensive so having the metadata in an all-flash vdev ("special" type vdev) greatly speeds up pool rebuilds and pool scrubs.
! Tab !! What it adds !! Redundancy !! Removable
|-
| '''Write Log''' || SLOG/ZIL -- an external intent log so synchronous writes do not have to land in the data groups first. || Automatically mirrored. Add either a single device or one or more pairs; any other odd count is rejected. || Yes, one group per operation
|-
| '''Read Cache''' || L2ARC -- a second-level read cache below RAM. || None, and none is needed; a failed cache device costs you cache, not data. || Yes
|-
| '''Metadata Offload''' || A ZFS special VDEV holding filesystem metadata and, if you set '''Small Block Offload''', small blocks as well. || Always mirrored. A single device is refused. || '''No -- permanent'''
|-
| '''Deduplication Offload''' || A dedicated VDEV for the deduplication table. Adding one turns deduplication on for the pool. || Always mirrored. || '''No -- permanent'''
|}


==== Pool Performance Inconsistent ====
'''Metadata and deduplication offload device groups cannot be removed once added.''' The dialog says so, and there is no CLI command to remove them either. Size them properly the first time, and mirror them properly -- losing that group loses the pool.
Inconsistent performance can be an indicator of media that is failing. QuantaStor looks for predictive drive failure indicators and reports on that and also marks bad and failing media with a 'Warning' state. To check the media first navigate to the Physical Disks section and check to see that all devices are in the Normal state.  If any are in the Warning or Error state click on them, then review the State Detail in the Properties section to get information about what is wrong with the device. If predictive drive failures are reporting then the media should be replaced.  When predictive drive failures are detected QuantaStor generates alerts so it is also important to make sure you have email or other alert notification options configured in the Alert Manager section.


If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read.  If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices.  We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.
On an encrypted pool every device added as cache, log, metadata or dedup offload is encrypted automatically as part of the operation. Never add devices to a QuantaStor pool with <code>zpool add</code> directly: an encrypted pool grown that way ends up with plaintext members, and the appliance raises an alert when it detects them.


==== Small File Performance Issues ====
==== Metadata and small-file offload ====
If you're having performance issues with small files in a HDD based pool we recommend adding 3x SSDs to the pool as meta-data offload with a 32K small-file offload size.  Best performance for IOPS, small files, databases, VMs, and other workloads with effectively random and small block IO access patterns is to use small parity length layout (RAIDZ2 / 4d+2p) or mirroring (RAID10).  The other important factor is having many VDEVs, at least 3 and ideally 4 or more.  If you don't have enough VDEVs expand you system to increase the number of VDEVs which are referred to as Storage Pool Device Groups in QuantaStor.


==== Failed Pool ====
A scale-up pool built from HDDs should always include SSDs for metadata and small-file offload. The effect on scrub time, replication, HA failover and small-file performance is large enough that we do not design HDD-based pools without it -- three SSDs with a 32K small-block offload size is a good starting point.


==== due to media connectivity ====
An all-flash pool does not need it and should not use it, with one exception: QLC media handles small-block I/O poorly, so offloading metadata and small blocks onto TLC media in front of a QLC pool is worthwhile.
If a pool is in the Failed state that indicates that more devices in the pool have failed within a given VDEV/stripe than can be repaired and the pool has stopped.  Often this problem is related to device connectivity and not failed media.  For example, if an extern JBOD is powered off this will remove access to many devices and if the pool is not striped across many JBODs the pool will stop in a Failed state.  To address this check any external JBODs and make sure they're cabled up properly and powered on.  


==== due to HA failover ====
==== Write log (SLOG/ZIL) ====
Another reason one could see a Failed state is in a HA configuration where the other node has taken over the pool.  The System seeing the pool in the Failed state is reporting that because it no longer has write access to the pool.  In an HA cluster configuration the node that had the pool at the time it transitioned to the Failed state will need to be rebooted.  This is because any write I/O in-flight is effectively stuck in the disk IO stack and this cannot be cleared without a reboot.  QuantaStor has many guards in place to prevent a Failed pool from being re-imported on a system until it is rebooted.


==== due to too many failed devices ====
Modern deployments use the write log less than they used to. Its value was in making an HDD pool usable for database and virtual-machine workloads, and those workloads should be on flash now that NVMe and SAS SSDs are affordable.
If you see multiple devices in a single VDEV that are failed beyond what the pool can repair (for example 3x devices in a VDEV of 4d+2p) then the pool may not be recoverable without assistance from a drive repair service.  In such conditions we recommend powering everything off and trying to restart the system (cold boot including cold boot of any JBODs) a few times to see if one or more of the failed devices can recover.  


==== due to 'foreign media' ====
Note that the default <code>standard</code> sync policy bypasses the write log unless the write carries <code>O_SYNC</code>, which hypervisors and databases set and most other applications do not. If you add a write log for some other workload you will see it sitting idle. Setting '''Sync Policy''' to <code>always</code> forces every write through it -- but if the log device is not fast enough that makes things slower, not faster.
If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode.  These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu.  This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.


==== Multiple Pool Failovers ====
Multiple pairs can be added to scale log throughput. Four 3.84&nbsp;TB NVMe devices give you two mirrored pairs, and if each pair sustains 4&nbsp;GB/s you have an 8&nbsp;GB/s log -- enough to absorb and coalesce the write I/O of roughly eighty HDDs.
Storage Pools configured for HA have an associated High-availability Group with a set of HA failover policies. The default policy is to do a failover if all data ports go offline.  Systems should have ethernet ports connected to two separate switches that a switch outage does not bring all ports offline. Similarly with systems with FC ports a FC switch should always be used in-between the QuantaStor system and the client machines else the HA Group policy should be reconfigured to not failover when all FC ports are down as a direct connection to a FC initiator that is restarting will confuse the system into thinking all ports are down and that a failover is needed. QuantaStor will use the heart-beat network ports to communicate with the secondary node to ensure that connectivity is available before doing a failover so if both nodes lose access there is not an unnecessary pool failover.
 
==== Read cache (L2ARC) ====
 
Read cache devices do not need to be fault tolerant. Size the cache to roughly the working set of the application, database or VM estate using the pool. A read cache gives no immediate benefit: it has to learn which blocks are worth holding, so judge it after a few days of representative load, not in the first hour.
 
==== RAM read cache (ARC) ====
 
Scale-up pools use the OpenZFS ARC in RAM as the primary read cache rather than the Linux page cache. Blocks are frequently read more than once, and serving the hot set from RAM is orders of magnitude faster than reading it from disk -- which also takes load off the disks and indirectly improves write performance. Plan for a minimum of 32&nbsp;GB to 64&nbsp;GB of RAM on a small system, 96&nbsp;GB to 128&nbsp;GB on a medium one, and 256&nbsp;GB or more on a large one. The [https://www.osnexus.com/zfs-designer OSNEXUS design tools] will size this for a given configuration.
 
=== Hot-spares ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Add Hot-spare(s) ''(toolbar)''}}
 
[[File:pool_addspare.png|thumb|right|780px|Add Hot-spare(s) / Recover Storage Pool assigns spare disks to one pool.]]
 
All media degrades, so assign at least one hot-spare to every pool and let the appliance heal itself. A spare assigned here is dedicated to this pool. A ''universal'' spare, marked from the '''Physical Disks''' section instead, is available to any pool in the grid -- see [[Physical Disks/Devices]].
 
If the pool is already degraded when you add a spare, the repair starts automatically. On a pool that belongs to an HA group the disk list is restricted to devices visible from both nodes, since a spare only one node can see would break the failover.
 
Remove a spare with '''Remove Hot-spares''' from the pool's right-click menu, which returns the disk to the available pool for other uses.
 
==== Hot-spare policies ====
 
The '''Hot-Spare Policy''' on '''Modify Storage Pool''' controls how a spare is chosen when a rebuild has to start:
 
{| class="wikitable"
! Policy !! Behaviour
|-
| Autoselect best match from assigned or universal spares || Default. Any assigned or universal spare of a suitable size may be used.
|-
| Autoselect best match from pool assigned spares only || Ignores universal spares.
|-
| Autoselect exact match from assigned or universal spares || Prefers a spare of the same model and capacity as the failed disk before considering anything else.
|-
| Autoselect exact match from pool assigned spares only || As above, restricted to assigned spares.
|-
| Manual Hot-Spare Management Only || Disables automatic repair for this pool entirely.
|}
 
Set '''Manual Hot-Spare Management Only''' when an administrator is driving the reconfiguration by hand and does not want the appliance injecting a spare underneath them.
 
=== Removing cache devices ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool ''(select + right-click)'' &rarr; Remove Cache Device}}
 
[[File:pool_removecache.png|thumb|right|750px|Remove Pool Cache Devices, with separate lists for read cache and write log devices.]]
 
This dialog removes read cache and write log devices only; metadata and dedup offload groups do not appear because they cannot be removed. Removing a read cache device raises a confirmation, because a warm cache that is carrying a lot of frequently-read data is doing real work and dropping it will show up as a performance regression until the remaining cache warms up. Write log devices are removed one group per operation.
 
Removing the last write log device also resets the pool's sync policy to <code>standard</code>, so a pool that was running <code>sync=always</code> against a dedicated log quietly returns to hybrid behaviour rather than forcing every write into the data groups.
 
== Growing and expanding a pool ==
 
'''Grow''' and '''Expand''' solve different problems and are not interchangeable:
 
* '''Grow''' adds new disks, forming additional device groups. Use it when you have physically added drives.
* '''Expand''' grows the pool into capacity that already exists on its current devices. Use it when the LUNs underneath the pool have been enlarged -- an iSCSI or FC LUN resized on a back-end array, as in an [[HA Cluster Setup (external SAN)|external SAN]] configuration. It does not apply to a pool built on directly attached disks, whose capacity only changes by adding drives.
 
=== Grow ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Grow ''(toolbar)''}}
 
[[File:pool_grow.png|thumb|right|780px|Grow Storage Pool adds whole device groups. The number of disks must be an exact multiple of the pool's current group width.]]
 
A grow always replicates the pool's existing layout, so the disk count has to be an exact multiple of the current device group width. Selecting a single disk for a mirrored pool is rejected:
 
<pre style="font-size: smaller">
Specified group of devices (1) is not a multiple of the Storage Pool's RAID set size.
Must specify groups of '2' devices to grow Storage Pool 'pool-1'.
</pre>
 
Adding a device group to a pool that currently has one turns a RAID1 pool into RAID10, a RAIDZ1 pool into RAIDZ1+0, and so on. The new disks must be at least as large as the smallest disk already in the pool.
 
A grow is refused while the pool is degraded, resilvering or reshaping. If a scrub is running, stop it first. Recover a degraded pool before growing it -- add hot-spares and let the repair finish.
 
Growing is additive only. Device groups cannot be removed from a pool afterwards, so a pool cannot be shrunk.
 
=== Expand ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool ''(select + right-click)'' &rarr; Expand Storage Pool}}
 
Expand takes no disk selection -- pick the pool and click '''OK'''. It rescans the underlying devices, extends the partition on each one into the new space, and brings the enlarged devices online.
 
'''Expand is not supported on encrypted pools.''' The menu item is still offered, but the operation is refused:
 
<pre style="font-size: smaller">
Expansion of underlying media is only supported on non-encrypted Storage Pools.
</pre>
 
Plan capacity growth on an encrypted pool as a Grow with additional devices instead.
 
== Starting, stopping and rescanning ==
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Stop ''(toolbar)''}}
 
[[File:pool_start.png|thumb|right|640px|Activate/Start Storage Pool. The Current Passphrase field is enabled only for a passphrase-protected encrypted pool.]]
 
'''Stop''' takes a pool offline for maintenance. Any active client session with a volume in that pool is dropped, so schedule it. '''Start''' brings it back. The '''Current Passphrase''' field is only enabled for a passphrase-protected pool, and '''Force''' is required to start a pool that is degraded.
 
'''Rescan''' scans for hot-plugged disks, then looks for storage pools, and then attempts to activate any offline or newly discovered pool it finds. Run it on the Storage System after physically moving disks in. It is the WUI equivalent of <code>[[QuantaStor CLI Command Reference#pool-scan|qs pool-scan]]</code>.
 
== Importing and exporting pools ==
 
A pool can be moved between appliances by moving all of its disks. Export it on the old system, move the disks, run '''Rescan''' on the new system so the disks are discovered, then import.
 
=== Export ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Export ''(toolbar)''}}
 
[[File:pool_export.png|thumb|right|640px|Export Storage Pool. The configuration of the volumes and shares in the pool is saved into the pool before it is detached.]]
 
Export detaches the pool from the appliance and '''removes the pool and its associated objects from the appliance configuration database'''. Before it does, it writes a fresh metadata archive into the pool's {{Code|1=.poolmeta/}} directory describing the Storage Volumes, Network Shares and Backup Policies it contained, and an import replays that archive to recreate them. The confirmation prompt warns that the export clears the Replica Association and Schedule Association information held against those volumes and shares, so plan to re-establish replication and snapshot schedules after the import.
 
Two options are available, both off by default:
 
* '''Release I/O Fence''' -- drops the SCSI-3 persistent reservations the appliance holds on the pool's disks. Only use this when the disks are being repurposed for a different pool; releasing the fence on disks you intend to re-import removes a protection you want.
* '''Clear Encryption Table Entries''' -- removes the pool's devices from the appliance's <code>crypttab</code>. Relevant only for an encrypted pool.
 
=== Import ===
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Import ''(toolbar)''}}
 
[[File:pool_import.png|thumb|right|645px|Import Storage Pools after clicking Scan. An exported pool is listed under its internal qs-&lt;uuid&gt; name.]]
 
Select the Storage System, click '''Scan''', tick the pools to import and click '''OK'''. The '''Passphrase''' field becomes available when you select a pool that is passphrase protected.
 
An exported pool appears in the scan list under its internal <code>qs-''&lt;uuid&gt;''</code> name rather than its friendly name, because the friendly name is a property inside the pool and cannot be read until it is imported. Its own name comes back with it. The scan reports the pool's status, its raw capacity and the number of devices it is built from, which is the quickest way to confirm every disk made the move.
 
A pool listed as <code>UNAVAIL</code> usually indicates an unclean export; the appliance may need a reboot to clear the stale pool state before the import will succeed. An import can also be refused before it starts if a foreign SCSI-3 reservation is found on the disks -- another appliance still holds them. Powering off the systems and the JBODs is generally the cleanest way to clear stale reservations.
 
Two options are available from the CLI for a pool whose metadata archive is stale or wrong: <code>qs pool-import --skip-metadata=true</code> imports the volumes and shares without replaying the archive, and <code>qs pool-import --metadata-only=true</code> replays the archive for a pool that is already imported.
 
=== Importing an encrypted pool ===
 
A software-encrypted pool keeps its key material on the appliance's own boot disk, not on the pool's disks, so moving the disks alone is not enough. Three things live under {{Code|1=/etc/cryptconf}} and all of them have to be present on the target appliance before the pool can be imported:
 
{| class="wikitable"
! Directory !! Holds
|-
| {{Code|1=/etc/cryptconf/keys}} || The pool's wrapped encryption key and the salt used to wrap it, one pair per pool, named for the pool's UUID. A {{Code|1=backup}} subdirectory keeps copies from before each passphrase change or re-key.
|-
| {{Code|1=/etc/cryptconf/metadata}} || One file per pool recording which devices make up the pool and their serial numbers. '''Without this file the pool does not appear in the pre-import scan at all.'''
|-
| {{Code|1=/etc/cryptconf/headers}} || A backup of the LUKS header of each encrypted device.
|}
 
'''Back up {{Code|1=/etc/cryptconf}}''', and keep the backup somewhere other than the appliance. Losing it with no key export, key server copy or HA peer holding the key means the pool is unrecoverable -- the LUKS header backups do not help, because they hold the drive's master key wrapped by the pool key you just lost.
 
The supported way to move that material between appliances is '''Export Keys''' and '''Import Keys''' -- see [[#Encryption|Encryption]] below. Once the keys are in place, import the pool through the same '''Import Storage Pools''' dialog used for unencrypted pools.
 
=== Importing a third-party OpenZFS pool ===
 
Pools created by other OpenZFS platforms (illumos, FreeBSD, ZFS on Linux) import through the same dialog. QuantaStor identifies pools and volumes by UUID, so the import renames the pool to the <code>qs-''&lt;uuid&gt;''</code> form and preserves the original name as the pool's friendly name. For more complex cases there is a console-level utility, documented in [[Console Level OpenZFS Storage Pool Importing]].
 
== Scrubbing, trim and health ==
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool ''(select + right-click)'' &rarr; Start Pool Scrub}}
 
A scrub reads every block in the pool, verifies it against its checksum, and repairs anything that has silently rotted using the pool's redundancy. It is the only mechanism that finds latent corruption in data nobody has read for a year, and it should be run about once a month.
 
A scrub competes with client I/O. If it is hurting production performance, stop it with '''Stop Pool Scrub''' and restart it during a quieter window -- OpenZFS resumes rather than starting over. Progress and the result of the last scrub appear in the pool's properties as the '''Scan Status'''.
 
Do not run a scrub and a grow at the same time; a grow is refused while a scrub is active.
 
=== Scheduling scrubs ===
 
Nothing schedules a scrub automatically -- creating a pool does not create a schedule for it. Set one up under '''Schedules''' as a Maintenance Schedule, or with <code>[[QuantaStor CLI Command Reference#maintenance-schedule-create|qs maintenance-schedule-create]]</code>. A schedule takes a maintenance type of either <code>pool-scrub</code> (the default) or <code>pool-trim</code>, a list of pools, a minimum number of days between runs from 2 to 365 (90 by default), and the day of the week and hour it is allowed to start. One schedule can cover several pools; add and remove them with <code>[[QuantaStor CLI Command Reference#maintenance-schedule-add|qs maintenance-schedule-add]]</code> and <code>[[QuantaStor CLI Command Reference#maintenance-schedule-remove|qs maintenance-schedule-remove]]</code>.
 
=== Trim ===
 
<code>[[QuantaStor CLI Command Reference#pool-trim|qs pool-trim]]</code> starts a one-off TRIM pass over the pool's flash media, freeing blocks the filesystem has released so the media can garbage-collect them. '''Enable SSD Auto Trim''' does the same thing continuously. Use one or the other on any pool built from SSD or NVMe media; a scheduled <code>pool-trim</code> maintenance schedule is the middle ground when continuous auto-trim is too intrusive.
 
=== Health and capacity ===
 
<code>[[QuantaStor CLI Command Reference#pool-health-check|qs pool-health-check]]</code> checks the pool and recursively checks every disk underneath it, which is faster than working down the stack by hand.
 
Capacity alerting is driven by three thresholds, expressed as the percentage of '''free space remaining''', and they are appliance-wide rather than per pool:
 
{| class="wikitable"
! Threshold !! Default !! Fires when
|-
| Pool Free Space Warning Threshold || 30 || 30% or less of the pool remains free.
|-
| Pool Free Space Alert Threshold || 10 || 10% or less remains free.
|-
| Pool Free Space Critical Alert Threshold || 5 || 5% or less remains free.
|}
 
Read them with <code>qs alert-config-get</code> and change them with <code>[[QuantaStor CLI Command Reference#alert-config-set|qs alert-config-set]]</code>. Alerting only reaches you if a delivery method is configured -- see [[Alert Manager]] and [[Call-home / Alerting]].
 
A thin-provisioned pool can be over-committed, so watch '''% Provisioned''' alongside '''% Utilized''' in the pool grid. The first tells you what you have promised, the second what has actually been written.
 
== Pool checkpoints ==
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool ''(select + right-click)'' &rarr; Create Pool Checkpoint}}
 
A checkpoint saves the whole pool's current state so the pool can be rolled back to that point. It exists for upgrade safety: take a checkpoint immediately before a pool version upgrade, confirm the upgrade is good, then discard it.
 
Only one checkpoint may exist per pool. A second attempt is refused:
 
<pre style="font-size: smaller">
A checkpoint already exists for storage pool '&lt;pool-id&gt;'. Only one checkpoint is allowed per pool;
use pool-checkpoint-modify to change its expiration, or pool-checkpoint-delete to discard it first.
</pre>
 
'''Auto-Expiration''' sets the number of days after which the appliance discards the checkpoint on its own; the dialog defaults to 7 and 0 disables automatic expiry. Set an expiry rather than relying on remembering, because a checkpoint holds on to every block the pool has freed since it was taken, and that space is only released when the checkpoint goes. While a checkpoint exists, changes to the pool's VDEV layout are blocked, so a grow or a device removal has to wait for it. Change the expiry with '''Modify Pool Checkpoint''' and discard it with '''Delete Pool Checkpoint'''.
 
== Encryption ==
 
Encryption is set at pool creation and cannot be added to, or removed from, an existing pool. To encrypt data that is already on an unencrypted pool, create an encrypted pool and migrate the data across. The rest of the encryption controls manage the keys of a pool that is already encrypted. Software encryption is covered in more depth in [[Software Encryption]].
 
{| class="wikitable"
! Operation !! Where !! What it does
|-
| '''Change/Clear Pool Passphrase''' || pool right-click menu, encrypted pools only || Sets, changes or clears the passphrase that protects the pool's keys. Supply the current passphrase to change it; tick '''Clear Passphrase''' to remove it and restore automatic startup.
|-
| '''Export Keys''' || Storage Pool Encryption toolbar group || Downloads the pool's key material as a <code>.metakey</code> file. Store it securely and separately -- for a pool with no passphrase this file is enough to unlock the pool.
|-
| '''Import Keys''' || Storage Pool Encryption toolbar group || Loads a previously exported key block onto a Storage System. Do this before importing the pool itself.
|-
| '''Re-Key Pool''' || Storage Pool Encryption toolbar group || Generates a new encryption key and registers it on the pool's devices, keeping a backup of the previous key material. Software encryption only, and only for pools with no passphrase -- passphrase-protected pools are excluded from the dialog's pool list. Each device needs a free LUKS key slot, and there are eight.
|}
 
Clearing a passphrase is what restores unattended startup: supply the current passphrase, leave the new one blank, and the pool goes back to unlocking itself at boot. Re-keying is gated on that having been done -- attempting it on a passphrase-protected pool is refused:
 
<pre style="font-size: smaller">
Encrypted storage pool 'pool-1' requires a passphrase.
Passphrase must be cleared/removed before the pool may be re-keyed.
</pre>
 
Passphrases are 10 to 80 characters long and may use letters, digits and the symbols <code>. - : _</code> (period, hyphen, colon, underscore).
 
Two further constraints are worth planning around. A passphrase-protected pool cannot fail over in an HA group unless the passphrase is already cached, because the failover has nobody to prompt. And an idle universal hot-spare is not encrypted while it is idle -- it is encrypted at the moment it is pulled into an encrypted pool.
 
== Deleting a Storage Pool ==
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool &rarr; Delete ''(toolbar)''}}
 
[[File:pool_delete.png|thumb|right|640px|Delete Storage Pool. Every volume and share in the pool goes with it.]]
 
Deletion destroys the pool and every Storage Volume and Network Share in it, and releases the disks for reuse. It is final. The dialog is also covered in [[Storage Pool Delete]]. Check the pool name in the dialog before you confirm -- the confirmation prompt states the pool name, the appliance, how utilized it is, how much data it holds, and how many volumes and shares are about to go, precisely so a wrong selection is obvious before you commit.
 
'''Force''' is required when volumes or shares in the pool have active client sessions.
 
Where [[Security Configuration#Multi-admin Approvals|multi-admin approval]] covers Storage Pool deletes, the delete is held until the required number of administrators approve it, and then runs with the security options chosen here.
 
=== Security options ===
 
'''Shred Encryption Keys''' is enabled and ticked automatically for an encrypted pool. It shreds the pool's key material, which makes the data unrecoverable immediately without having to overwrite it.
 
'''Shred Data''' overwrites the disks as part of the delete, using the mode selected beside it. It is off by default; the mode combo is disabled until you tick it and defaults to '''Auto'''. Shredding a large pool can take many hours, and the dialog warns you of that when you confirm.
 
{| class="wikitable"
! Mode !! Procedure
|-
| Auto (default) || Lets the appliance select the shred method for each device.
|-
| 4-pass DoD 5220.22-M section 8-306 procedure || Overwrites all addressable locations with a character, its complement, a random character, then verifies.
|-
| 4-pass NNSA Policy Letter NAP-14.1-C procedure || Overwrites all locations with a pseudorandom pattern twice, then with a known pattern: random, random, 0x00, verify.
|-
| US Army AR380-19 method || 0x00, 0xff, random.
|-
| Fill-zeros single-pass procedure || A single pass of zeros.
|-
| Random-data single-pass procedure || A single pass of random data.
|-
| SED Instant Erase || Cryptographic erase on a self-encrypting drive. Effectively instant.
|-
| NVME Secure || The NVMe device's own secure-erase command.
|-
| SSD Secure || The SSD's own secure-erase command.
|}
 
The overwrite-based modes are implemented with the Linux <code>scrub</code> utility, which implements these government standards; the SED, NVMe and SSD modes hand the job to the drive's own firmware and are far faster on media that supports them.
 
== Replacing and repairing devices ==
 
{{Navigation|Storage Management &rarr; Storage Pools &rarr; Storage Pool Device &rarr; Replace ''(toolbar)''}}
 
The '''Storage Pool Device''' toolbar group operates on one member disk at a time, and its buttons are the manual counterpart to automatic hot-spare repair. Select the device on the '''Storage Pool Devices''' tab first.
 
{| class="wikitable"
! Button !! Use it to
|-
| '''Replace''' || Swap a failed or failing device for a new one. The new disk must be on the same Storage System, unused, and -- on a pool using SED encryption -- SED capable.
|-
| '''Attach''' || Add a device to an existing device, joining its group, or creating a mirrored group from a single device.
|-
| '''Detach''' || Permanently remove a device from its mirror. Only valid for a member of a mirror group; the appliance refuses it on a parity group.
|-
| '''Offline''' || Take a working device out of service so a disk that is failing but still online can be swapped. This usually puts the pool into a degraded state. If the pool has no hot-spare the confirmation says so, because nothing will step in automatically.
|-
| '''Online''' || Bring an offline device back into service. The pool resilvers it as needed.
|}
 
'''Force''' on Attach and Replace forces the use of a new device that appears to be in use. Only reach for it after an unforced attempt has failed.
 
Each of these dialogs asks for confirmation naming the specific device, so read the serial number before agreeing. Physical disk-level operations -- identifying a drive by its LED, formatting, marking a universal spare -- live under [[Physical Disks/Devices]].
 
Note that the pool-level '''Identify''' toolbar button and the '''Identify Storage Pool''' menu item open the '''Identify Hardware Controller Disk Device''' dialog, which lists disks behind a hardware RAID controller. Scale-up pools are built on HBAs rather than RAID cards, so that dialog has nothing to show and neither entry point does anything useful. To blink the LEDs of every disk in a pool, use <code>[[QuantaStor CLI Command Reference#pool-identify|qs pool-identify]] --pool=&lt;pool&gt; --blink-type=on --duration=&lt;seconds&gt;</code>.
 
== The pool context menu ==
 
[[File:pool_context_menu.png|thumb|right|257px|Right-clicking a Storage Pool. Several operations are only available here.]]
 
Several pool operations have no toolbar button and are reachable only by selecting a pool and right-clicking it: '''Expand Storage Pool''', '''Remove Hot-spares''', '''Add Log/Cache/Metadata Devices''', '''Remove Cache Device''', '''Start Pool Scrub''', '''Stop Pool Scrub''', '''Create Pool Checkpoint''' (and Modify and Delete once one exists), '''Change/Clear Pool Passphrase''' on an encrypted pool, and '''Create Storage Tier'''.
 
The menu adapts to the pool. Grow, Expand, hot-spare and cache management, scrub and checkpoints are offered on scale-up (ZFS) pools only; a Ceph-backed pool gets a different subset. The boot pool gets '''Properties''' and nothing else.
 
== Troubleshooting ==
 
=== Degraded pool ===
 
'''Degraded''' means one or more member devices has failed or been disconnected. If the pool has an assigned hot-spare, the repair starts on its own within about a minute. If there are no spares, add a disk of the same type that is the same size or larger, mark it as a hot-spare in the '''Physical Disks''' section, and the appliance injects it and starts the resilver.
 
=== Rebuild is taking a long time ===
 
Rebuild time depends on the media type and speed, the load on the system, and the number of device groups in the pool. Every pool should have at least two groups so it can serve I/O from one while another rebuilds. On an HDD pool, add three SSDs for metadata offload -- rebuilds are metadata intensive, and putting the metadata on flash speeds up both rebuilds and scrubs substantially.
 
=== Pool performance is inconsistent ===
 
Inconsistent performance often means failing media. The appliance watches predictive failure indicators and marks suspect disks with a '''Warning''' state. Check the '''Physical Disks''' section first: every device should be '''Normal'''. Click any device in '''Warning''' or '''Error''' and read '''State Detail''' in the '''Properties''' panel for the specific fault. Replace anything reporting predictive failure. Predictive failures generate alerts, so make sure a delivery method is configured in [[Alert Manager]] -- otherwise the first you hear of it is a performance complaint.
 
If every disk reports '''Normal''', run a read performance test from the '''Physical Disks''' section. It reads from each selected device with a basic sequential read. A large gap between one or two devices and the rest points at those devices. Open a support ticket and send logs for help interpreting the results.
 
=== Small-file performance issues ===
 
On an HDD pool, add three SSDs as metadata offload with a 32K small-block offload size. Beyond that, small-file, database, VM and other random small-block workloads want a short parity stripe (RAIDZ2 with 4 data + 2 parity) or mirroring, and they want many device groups -- at least three and ideally four or more. If the pool does not have enough groups, expand the system so it can.
 
=== Failed pool ===
 
==== Because of media connectivity ====
 
'''Failed''' means more devices have failed within a single device group than that group's redundancy can repair, and the pool has stopped. This is frequently connectivity rather than dead media. If an external JBOD is powered off, every disk in it disappears at once, and a pool that is not striped across several JBODs stops. Check the cabling and power on every external JBOD first.
 
==== Because of an HA failover ====
 
In an HA configuration, a pool showing '''Failed''' on one node can simply mean the other node has taken it over -- this node is reporting that it no longer has write access. The node that held the pool when it transitioned to '''Failed''' has to be rebooted, because in-flight write I/O is stuck in the disk I/O stack and nothing else clears it. The appliance has several guards that prevent a failed pool from being re-imported on a system until it has been rebooted.
 
==== Because of too many failed devices ====
 
If more devices in a single group have failed than the layout can repair -- three devices in a 4d+2p group, for instance -- the pool may not be recoverable without a drive repair service. Before concluding that, power everything off and cold boot the whole stack including the JBODs a few times; occasionally one of the failed devices comes back.
 
=== Repeated pool failovers ===
 
A pool configured for HA has a High-availability Group with failover policies attached, and the default policy fails over when all data ports go offline. Connect the appliance's Ethernet ports to two separate switches so that one switch outage cannot take every port down. Do the same for Fibre Channel: always put an FC switch between the appliance and the clients, because a directly attached FC initiator rebooting looks to the appliance like all FC ports going down and triggers a failover. The heartbeat network is checked before any failover to confirm the peer is reachable, so a total connectivity loss on both nodes does not cause a needless failover.
 
=== Volumes or shares showing as Missing ===
 
<code>[[QuantaStor CLI Command Reference#pool-clear-missing|qs pool-clear-missing]]</code> removes the Storage Volume and Network Share records that the appliance has marked '''Missing''' for a pool. It clears database records only; it does not touch the pool. Use it to tidy up after a pool has been re-imported and some objects genuinely no longer exist, and be aware it is not reversible -- the records are gone, and only re-discovery or a metadata re-import brings them back.
 
== CLI command reference ==
 
Every pool operation is available from the <code>qs</code> CLI and the REST API. <code>--disk-list</code> accepts disk names, IDs and filter expressions such as <code>[unused]</code>, <code>[gt:4TB]</code> or <code>[tag:''value'']</code>, which makes scripted pool builds practical without hardcoding serial numbers.
 
<pre style="font-size: smaller">
# create a six-disk double-parity pool with compression on
qs pool-create --name=pool-1 --disk-list=sdd,sde,sdf,sdg,sdh,sdi \
    --raid-type=RAID60 --raid-set-size=6 --compress=true
 
# add a mirrored pair of write log devices and a read cache device
qs pool-add-write-log --pool=pool-1 --disk-list=sdx,sdy
qs pool-add-read-cache --pool=pool-1 --disk-list=sdz
 
# start a scrub, then check its progress in the Scan Status field
qs pool-scrub-start --pool=pool-1
qs pool-get --pool=pool-1
</pre>
 
Reference a disk by its device name, its full SCSI ID or its object ID. Prefer the SCSI ID in scripts: device names are not stable across reboots and are not unique across the grid, so <code>sdd</code> is ambiguous on a multi-node grid while a SCSI ID is not.
 
{| class="wikitable"
! Command !! Purpose
|-
| <code>[[QuantaStor CLI Command Reference#pool-list|qs pool-list]]</code>, <code>[[QuantaStor CLI Command Reference#pool-get|qs pool-get]]</code>, <code>[[QuantaStor CLI Command Reference#pool-search|qs pool-search]]</code> || List, inspect and search pools.
|-
| <code>[[QuantaStor CLI Command Reference#pool-create|qs pool-create]]</code> || Create a pool. Also carries <code>--ashift</code>, <code>--profile</code>, <code>--encrypt</code>, <code>--encryption-type</code>, <code>--passphrase</code>, <code>--key-server-type</code> and <code>--key-server-profile</code>.
|-
| <code>[[QuantaStor CLI Command Reference#pool-modify|qs pool-modify]]</code> || Change name, description, compression, auto-trim, I/O profile, sync policy, cache policies, hot-spare repair policy and small block threshold.
|-
| <code>[[QuantaStor CLI Command Reference#pool-grow|qs pool-grow]]</code> || Add device groups. <code>--dry-run-only=true</code> reports what would happen without changing anything.
|-
| <code>[[QuantaStor CLI Command Reference#pool-expand|qs pool-expand]]</code> || Grow into enlarged underlying devices.
|-
| <code>[[QuantaStor CLI Command Reference#pool-import|qs pool-import]]</code>, <code>[[QuantaStor CLI Command Reference#pool-import-encrypted|qs pool-import-encrypted]]</code>, <code>[[QuantaStor CLI Command Reference#pool-preimport-scan|qs pool-preimport-scan]]</code> || Scan for and import pools.
|-
| <code>[[QuantaStor CLI Command Reference#pool-export|qs pool-export]]</code> || Export a pool. <code>--metadata-only=true</code> writes a fresh metadata archive without exporting.
|-
| <code>[[QuantaStor CLI Command Reference#pool-start|qs pool-start]]</code>, <code>[[QuantaStor CLI Command Reference#pool-stop|qs pool-stop]]</code>, <code>[[QuantaStor CLI Command Reference#pool-scan|qs pool-scan]]</code> || Start, stop and rescan.
|-
| <code>[[QuantaStor CLI Command Reference#pool-destroy|qs pool-destroy]]</code> || Delete a pool. <code>--shred-keys</code> and <code>--shred-mode</code> cover the security options.
|-
| <code>[[QuantaStor CLI Command Reference#pool-add-read-cache|qs pool-add-read-cache]]</code>, <code>[[QuantaStor CLI Command Reference#pool-add-write-log|qs pool-add-write-log]]</code>, <code>[[QuantaStor CLI Command Reference#pool-add-metadata-offload|qs pool-add-metadata-offload]]</code>, <code>[[QuantaStor CLI Command Reference#pool-add-dedup-offload|qs pool-add-dedup-offload]]</code> || Add the offload tiers.
|-
| <code>[[QuantaStor CLI Command Reference#pool-remove-read-cache|qs pool-remove-read-cache]]</code>, <code>[[QuantaStor CLI Command Reference#pool-remove-write-log|qs pool-remove-write-log]]</code> || Remove read cache and write log devices. There is deliberately no equivalent for metadata or dedup offload.
|-
| <code>[[QuantaStor CLI Command Reference#pool-add-spare|qs pool-add-spare]]</code>, <code>[[QuantaStor CLI Command Reference#pool-remove-spare|qs pool-remove-spare]]</code> || Manage pool-assigned hot-spares.
|-
| <code>[[QuantaStor CLI Command Reference#pool-scrub-start|qs pool-scrub-start]]</code>, <code>[[QuantaStor CLI Command Reference#pool-scrub-stop|qs pool-scrub-stop]]</code>, <code>[[QuantaStor CLI Command Reference#pool-trim|qs pool-trim]]</code> || Scrub and trim.
|-
| <code>[[QuantaStor CLI Command Reference#pool-checkpoint-create|qs pool-checkpoint-create]]</code>, <code>[[QuantaStor CLI Command Reference#pool-checkpoint-modify|qs pool-checkpoint-modify]]</code>, <code>[[QuantaStor CLI Command Reference#pool-checkpoint-delete|qs pool-checkpoint-delete]]</code> || Manage the pool checkpoint.
|-
| <code>[[QuantaStor CLI Command Reference#pool-set-passphrase|qs pool-set-passphrase]]</code>, <code>[[QuantaStor CLI Command Reference#pool-rekey|qs pool-rekey]]</code>, <code>[[QuantaStor CLI Command Reference#pool-key-export|qs pool-key-export]]</code>, <code>[[QuantaStor CLI Command Reference#pool-key-import|qs pool-key-import]]</code> || Encryption key management.
|-
| <code>[[QuantaStor CLI Command Reference#pool-device-list|qs pool-device-list]]</code>, <code>[[QuantaStor CLI Command Reference#pool-device-get|qs pool-device-get]]</code>, <code>[[QuantaStor CLI Command Reference#pool-device-group-list|qs pool-device-group-list]]</code>, <code>[[QuantaStor CLI Command Reference#pool-device-group-get|qs pool-device-group-get]]</code> || Inspect device groups and their members.
|-
| <code>qs pool-device-attach</code>, <code>qs pool-device-detach</code>, <code>qs pool-device-replace</code>, <code>qs pool-device-online</code>, <code>qs pool-device-offline</code> || Per-device operations. These are newer than the generated CLI reference and do not have entries there yet; run <code>qs &lt;command&gt; --verbose</code> for the argument list.
|-
| <code>[[QuantaStor CLI Command Reference#pool-health-check|qs pool-health-check]]</code>, <code>[[QuantaStor CLI Command Reference#pool-clear-missing|qs pool-clear-missing]]</code>, <code>[[QuantaStor CLI Command Reference#pool-identify|qs pool-identify]]</code>, <code>[[QuantaStor CLI Command Reference#pool-portals-list|qs pool-portals-list]]</code> || Health check, clear missing records, blink the pool's disk LEDs, list the pool's portals.
|}
 
Three operations have no dialog in the web interface and are CLI or API only: <code>qs pool-trim</code>, <code>qs pool-health-check</code> and <code>qs pool-clear-missing</code>.
 
== Related pages ==
 
* [[Physical Disks/Devices]] -- disk discovery, formatting, universal hot-spares and per-disk operations
* [[Storage Volumes]] -- provisioning block storage from a pool
* [[Network Shares]] -- provisioning file storage from a pool
* [[Provisioning Tiers]] -- grouping pools behind a single provisioning target
* [[Performance Tuning]] -- workload-level tuning, including where I/O profiles fit
* [[Storage System Optimization]] -- appliance-wide OpenZFS and kernel tunables
* [[Software Encryption]] -- how software encryption and its key material work
* [[Storage Pool Cache]] -- the Add Log/Cache/Metadata Devices dialog in detail
* [[Storage Pool Scrub]] -- the scrub dialogs in detail
* [[Console Level OpenZFS Storage Pool Importing]] -- console-level import for complex cases
* [[Create Storage Pool High-Availability Group]] -- making a pool highly available across two appliances
* [[Alert Manager]] -- configuring where capacity and disk failure alerts are delivered
* [[Performance Testing]] -- measuring a pool before it goes into production
* [[QuantaStor CLI Command Reference]] -- full argument lists for every command above
 
----
<small>''Verified against QuantaStor 6.9.0.''</small>

Latest revision as of 16:40, 28 September 2026


A Storage Pool aggregates one or more physical disks into a fault-tolerant pool of storage from which you provision Storage Volumes (block/SAN) and Network Shares (file/NAS). This page covers the scale-up (OpenZFS) Storage Pool: how to choose a layout, how to create and modify a pool, how to add flash offload tiers, and how to import, export, scrub, grow and troubleshoot one.

A single pool serves block and file at the same time, and each resource can be reached over several protocols at once -- a Storage Volume over iSCSI and Fibre Channel together, a Network Share over NFS and SMB together. Scale-out (Ceph) pools are a different object type and are configured under the Scale-out Storage Configuration tab; everything below applies to scale-up pools, which the interface labels Scale-up (ZFS).

The Storage Pools section, with the pool grid, the four detail tabs and the Storage Pool toolbar group.
Section Purpose
Choosing a RAID layout Which layout to use, and the disk counts each one needs.
Creating a Storage Pool Every field on all three tabs of the Create dialog.
Modifying a Storage Pool Compression, sync policy, cache policy, hot-spare policy, I/O profile.
Device groups, offload tiers and hot-spares VDEVs, read cache, write log, metadata and dedup offload, spares.
Growing and expanding a pool The difference between Grow and Expand, and the disk-count rule.
Starting, stopping and rescanning Taking a pool offline and bringing it back.
Importing and exporting pools Moving a pool between appliances, and what export removes.
Scrubbing, trim and health Bit-rot verification, scheduled maintenance, capacity alerting.
Pool checkpoints A whole-pool rollback point for upgrade safety.
Encryption Software and hardware encryption, passphrases, key export and re-key.
Deleting a Storage Pool Deletion and the secure data shredding modes.
Replacing and repairing devices Attach, detach, replace, online and offline.
Troubleshooting Degraded and failed pools, slow rebuilds, inconsistent performance.
CLI command reference Every qs pool command, with the operations that are CLI only.

Choosing a RAID layout

A pool is built from one or more device groups -- VDEVs in OpenZFS terms, and Storage Pool Device Groups in QuantaStor. Each group carries its own redundancy, and the pool stripes across the groups. Fault tolerance is per group: a double-parity group survives two failed members, and a third failure in that same group loses the pool regardless of how healthy the other groups are.

We recommend Double-parity (RAIDZ2+0) with groups of six disks (4 data + 2 parity) as the best balance of fault tolerance, performance and usable capacity for most workloads, and it is what the Create dialog defaults to when enough disks are available. Use Mirrored (RAID10) for virtualization and databases where random IOPS matter most. Use Triple-parity (RAIDZ3+0) or double-parity for archive and backup, where capacity and sequential throughput matter more than IOPS.

Single-parity (RAIDZ1) is not sufficiently fault tolerant for production, and RAID0 has no redundancy and no bit-rot repair at all -- the Create dialog raises an explicit confirmation before it will build a RAID0 pool. Neither is supported for production deployments.

Layout (as labelled in the dialog) Minimum disks offered from Notes
Striped (RAID0) 1 No redundancy, no self-healing. A single device failure loses the pool.
Mirrored (RAID10) 2 Two-way or three-way mirrors. Highest random IOPS; usable capacity is 50% of raw (33% with three-way).
Single-parity (RAIDZ1+0) 3 One parity block per stripe. Not supported for production.
Double-parity (RAIDZ2+0) 4 Two parity blocks per stripe. Recommended default.
Triple-parity (RAIDZ3+0) 7 Three parity blocks per stripe. Best for long-term archive.
Distributed Single-parity (DRAID1 1VS/2VS/3VS) 10 Distributed RAID with 1, 2 or 3 distributed spares.
Distributed Double-parity (DRAID2 1VS/2VS/3VS) 16 As above with two parity blocks per stripe.
Distributed Triple-parity (DRAID3 1VS/2VS/3VS) 20 As above with three parity blocks per stripe.

The RAID Type list is built from the number of disks currently available on the selected Storage System, not the number you have selected. If a layout you expect is missing from the list, the appliance does not have enough unused disks to offer it -- the distributed RAID layouts in particular need at least 10, 16 or 20 available disks depending on parity level. Distributed RAID rebuilds far faster than parity RAID because the spare capacity is spread across every member device, which is why we recommend it for the large 60-, 84-, 90- and 106-bay JBOD backup and archive configurations where a conventional rebuild would run for days.

Build more than one pool where the workloads differ: low-cost fault-tolerant capacity in RAIDZ2 for backup and archive, and a mirrored pool for virtual machines, containers and databases. See Provisioning Tiers for grouping several pools behind one provisioning target.

Enclosure-aware disk selection

On a system with more than one JBOD, QuantaStor detects which enclosure each disk came from and selects disks round-robin across the enclosures so that the members of each device group land in different chassis. A pool built that way keeps running in a degraded state if a whole JBOD is powered off.

Round-robin selection is only used when there are enough enclosures for the layout to actually benefit -- the requirement scales with the group size, so a mirrored pool needs as many enclosures as the mirror is wide, while a RAIDZ1 pool with five-disk groups needs five. When there are too few enclosures the selection falls back to a sequential mode that groups each device group within one enclosure. Within an enclosure the selection also prefers disks of the same size, so a 2 TB disk is never mirrored against a 6 TB one.

Creating a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Create (toolbar)

Creating the first pool is normally the first configuration step on a new appliance. Name the pool, pick the disks and a layout, and click OK. The one setting you cannot change afterwards is encryption -- decide that before you create the pool, because there is no way to encrypt a pool in place later.

The dialog is a three-tab wizard; Previous and Next step through the tabs and OK submits from any of them.

General tab

The General tab of Create Storage Pool: name, layout, set size and disk selection.
Field Default Notes
Name pool-1 (next unused pool-n) 2 to 40 characters, letters, digits and . - _ only. No spaces. Invalid characters are dropped as you type. The name is a label only -- see What a pool looks like on disk.
Description empty Free text.
RAID Type Double-parity (RAIDZ2+0) with 6 or more available disks, otherwise Mirrored (RAID10), otherwise Striped (RAID0) See Choosing a RAID layout. The list is filtered by available disk count.
Set Size Auto Number of disks in each device group. Auto picks a sensible group width for the layout. Disabled for RAID0.
+ and - Select or deselect exactly one more device group's worth of disks, spread across enclosures. Disabled for the distributed RAID layouts, which build a single group.
Auto Config Select as many whole device groups as the available disks allow. Also disabled for distributed RAID.
Storage System the system you launched the dialog from Which grid member's disks to build from. A pool is owned by one system.
Disk grid none selected Only unused, usable disks appear. Boot and system disks, disks already in a pool, journal members, Ceph OSD disks, multipath child paths and anything under 50 MB are filtered out. Search filters across every column; commas separate multiple terms.
Force unchecked Scrubs existing partitions off the selected disks. Required when the disks carry an old ZFS label or LUKS header from a previously exported pool, and when the selected disks do not all report the same physical block size.

Set Size is what turns a parity layout into a striped set of parity groups. A RAIDZ1+0 pool built from eight disks with a set size of four is two four-disk RAIDZ1 groups striped together. Choosing a set size that the selected disk count is not an exact multiple of is rejected -- for mirroring the dialog itself tells you how many more devices to select, and for the parity layouts the appliance returns the error.

Encryption tab

The Encryption tab with software encryption and a user-defined passphrase selected. Encrypt and Store on Key Server is unavailable until a Key Server Profile exists.

All Storage Pool encryption is AES-256. Pick one of three encryption types:

  • No Encryption -- the default.
  • Software Encryption -- LUKS/dm-crypt full-disk encryption applied to every device in the pool. AES-NI hardware acceleration on modern server processors carries the cost, so the overhead is small.
  • Hardware Encryption -- self-encrypting drive (SED) encryption, handled by the drives themselves. Every disk in the pool has to be SED capable; the SED Capable and SED Status columns in the disk grid tell you which are.

If the array presenting the devices already encrypts at rest, QuantaStor should not encrypt them again. Systems such as the Seagate Corvault, Seagate Exos E/EP and Dell PowerVault families encrypt in hardware and present logical devices; layering software encryption over them costs throughput and protects data that is already protected. QuantaStor recognises these devices by vendor and model and skips its own encryption automatically -- see Encryption Bypass, which governs scale-up Storage Pool devices as well as scale-out OSDs. The choice is fixed once a device is provisioned, so check that your media is listed there before you create the pool.

With either encryption type enabled, the Key Protection group chooses where the key that protects the pool key lives:

  • Encrypt with No Passphrase (default) -- the pool unlocks and starts automatically at boot with no operator action. Use this for anything in an HA group.
  • Encrypt with User Defined Passphrase -- a passphrase of 10 to 80 characters, from letters, digits and . - : _. Enter it twice; the eye button reveals what you typed.
  • Encrypt and Store on Key Server -- store the key on an external KMIP or IBM Key Protect key server. This option stays greyed out until at least one Key Server Profile has been created, and then Key Server Profile selects which one to use.

A passphrase stops the pool from starting on its own. After a reboot a passphrase-protected pool stays offline and locked until an administrator runs Start Storage Pool and supplies it. That is exactly what you want on a portable or high-security system, and exactly what you do not want on an HA pool, where the automatic failover has no way to prompt for it. See Encryption below for changing or clearing a passphrase after the fact.

Advanced Settings tab

The Advanced Settings tab: I/O profile, compression, auto-trim, ashift and use case.
Field Default Notes
I/O Profile Default Tunes the member disks for a workload -- read-ahead, queue depth and I/O scheduler. See I/O profiles.
I/O Profile Description Read-only. Shows the selected profile's description.
Enable Compression checked Turns on LZ4 compression. Leave it on unless you are certain the data is incompressible.
Enable SSD Auto Trim unchecked Sends TRIM to flash media continuously so it can reclaim freed blocks. Only useful on a pool built from SSD or NVMe media.
Block Size Offset (ashift) Auto Auto, 9 (512 bytes) or 12 (4K bytes). Leave it on Auto so the appliance derives it from the reported physical block size of the disks. ashift is fixed for the life of the pool, and setting it too small costs you performance while setting it too large wastes capacity on small blocks.
Use Case Default Default provisions Storage Volumes and Network Shares as usual. Ceph OSD reserves the pool for zvols that back Ceph OSDs, and volumes created in such a pool cannot be presented to hosts over iSCSI, FC or NVMe-oF. This is set at create time only and cannot be changed later.

Compression is on by default because it usually increases both usable capacity and throughput at once -- a modern CPU compresses faster than the media can read or write, so moving less data is a net win. Turn it off for workloads that are already compressed, which is common in media and entertainment. The algorithm can be changed at any time from Modify Storage Pool; only the on/off choice is available at create time.

What a pool looks like on disk

The pool's name in the interface is a label. On the appliance the pool is always named qs-<pool-uuid>, mounted at /mnt/storage-pools/qs-<pool-uuid>, and the friendly name is carried in a quantastor:name property on the pool. That is why renaming a pool is instant and why the name survives a move to another appliance.

Two directories are maintained inside every pool:

  • .poolmeta/ -- a metadata archive describing the Storage Volumes, Network Shares, hosts, portal groups and schedule associations in the pool. It is refreshed periodically and always rewritten immediately before an export, and it is what an import replays to bring those objects back.
  • .qsbackups/ -- rolling hourly and daily backups of the appliance configuration database.

Modifying a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Modify (toolbar)
Modify Storage Pool. Everything except encryption and the on-disk layout can be changed here at any time.

Everything on this dialog can be changed on a running pool with volumes and shares in it. Encryption is the one pool property that is not here, because it cannot be changed after creation.

Field Default Notes
Storage Pool the selected pool Switch to a different pool without reopening the dialog.
Name, Description current values Renaming is metadata only and does not disturb I/O.
Hot-Spare Policy Autoselect best match from assigned or universal spares See Hot-spare policies.
I/O Profile the pool's current profile See I/O profiles.
Sync Policy standard standard honours the application's O_SYNC flag, so hypervisors and databases get write-through and everything else gets write-back. always forces every write through the write log. disabled makes all writes asynchronous.
Cache Policy Primary all What the RAM read cache (ARC) holds: all, metadata only, or none.
Cache Policy Secondary all The same choice for the SSD read cache (L2ARC).
Small Block Offload Disabled Maximum write size routed to the metadata offload device group. Only meaningful once a metadata offload group exists. We recommend 8K to 32K depending on how much SSD capacity you gave it.
Compression on on (LZ4), off, or a specific algorithm: lz4, lzjb, zle, gzip and gzip-1 through gzip-9, zstd and zstd-1 through zstd-19, and zstd-fast and zstd-fast-1 through zstd-fast-10.
Enable SSD Auto Trim the pool's current setting As on the Create dialog.
Encryption Key Settings Encrypt and Store Locally Enabled only on an encrypted pool. Moves the pool key between local storage and a key server. The key-server option needs a Key Server Profile to exist.

Changing compression affects new writes only; existing blocks keep whatever they were written with. Changing Sync Policy to disabled raises a confirmation, and rightly so -- it acknowledges writes before they are durable and can cause data loss on an unexpected power failure. It is not supported for production use.

The Cache Policy settings are the lever for a workload that pollutes the cache rather than benefiting from it -- a large sequential backup ingest, for example, can evict the working set of everything else on the appliance. Setting the primary cache policy to metadata keeps the pool's directory structure cached while leaving the data blocks out of RAM.

I/O profiles

An I/O profile is a bundle of block-device tunings applied to the pool's member disks -- read-ahead size, queue depth, I/O scheduler, and the SCST target driver thread and tasklet counts -- with separate values for HDD, SSD and NVMe media. It is not a filesystem setting, and it is re-applied every time the pool starts, on HA failover, and after a grow, attach or replace.

Seven profiles ship with the appliance:

Profile Optimizes for
Default General purpose server application workloads.
Virtualization Virtual machine workloads such as database and web servers under ESXi, XenServer, Hyper-V or Virtuozzo.
Disk Archive Disk-to-disk backup and archive applications.
Media Post-Production (Ingest Optimized) Media post-production editing.
Media Post-Production (Playback Optimized) Media post-production playback.
Edgeware IP TV Optimized The Edgeware IP TV application workflow.
Edgeware Web TV Optimized The Edgeware Web TV application workflow.

The profiles are defined in /opt/osnexus/quantastor/conf/qs_io_profiles.conf on the appliance; use qs pool-profile-list and qs pool-profile-get to read the values a profile actually applies. Profiles are per pool; the system-wide OpenZFS module parameters are a separate mechanism, covered in Storage System Optimization. For choosing between them, and for RAM sizing, see Performance Tuning.

Device groups, offload tiers and hot-spares

The Storage Pool Devices tab. The RAID Segment column names the device group each disk belongs to -- a data group, cache, or logs.

The four detail tabs under the pool grid show the pool from different angles. Storage Pool Device Groups lists the VDEVs with their RAID level and size, and Storage Pool Devices lists the individual member disks with the group each one belongs to in the RAID Segment column. A data group appears as mirror-0, raidz2-0 and so on; the offload tiers appear as cache and logs.

The number of data device groups matters as much as the layout. A pool with a single group has no way to spread load while that group is rebuilding, so build every pool with at least two, and prefer four or more for anything IOPS sensitive.

Adding offload devices

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Add Log/Cache/Metadata Devices
The Metadata Offload tab of Add Log/Cache/Metadata Device(s). The four tabs cover write log, read cache, metadata offload and dedup offload.

This dialog has one tab per offload class. Move disks from Available Disks into the selected list with the arrow buttons, or use Auto Config to pick a matched pair. Offload devices are dedicated to one pool and cannot be shared between pools.

Tab What it adds Redundancy Removable
Write Log SLOG/ZIL -- an external intent log so synchronous writes do not have to land in the data groups first. Automatically mirrored. Add either a single device or one or more pairs; any other odd count is rejected. Yes, one group per operation
Read Cache L2ARC -- a second-level read cache below RAM. None, and none is needed; a failed cache device costs you cache, not data. Yes
Metadata Offload A ZFS special VDEV holding filesystem metadata and, if you set Small Block Offload, small blocks as well. Always mirrored. A single device is refused. No -- permanent
Deduplication Offload A dedicated VDEV for the deduplication table. Adding one turns deduplication on for the pool. Always mirrored. No -- permanent

Metadata and deduplication offload device groups cannot be removed once added. The dialog says so, and there is no CLI command to remove them either. Size them properly the first time, and mirror them properly -- losing that group loses the pool.

On an encrypted pool every device added as cache, log, metadata or dedup offload is encrypted automatically as part of the operation. Never add devices to a QuantaStor pool with zpool add directly: an encrypted pool grown that way ends up with plaintext members, and the appliance raises an alert when it detects them.

Metadata and small-file offload

A scale-up pool built from HDDs should always include SSDs for metadata and small-file offload. The effect on scrub time, replication, HA failover and small-file performance is large enough that we do not design HDD-based pools without it -- three SSDs with a 32K small-block offload size is a good starting point.

An all-flash pool does not need it and should not use it, with one exception: QLC media handles small-block I/O poorly, so offloading metadata and small blocks onto TLC media in front of a QLC pool is worthwhile.

Write log (SLOG/ZIL)

Modern deployments use the write log less than they used to. Its value was in making an HDD pool usable for database and virtual-machine workloads, and those workloads should be on flash now that NVMe and SAS SSDs are affordable.

Note that the default standard sync policy bypasses the write log unless the write carries O_SYNC, which hypervisors and databases set and most other applications do not. If you add a write log for some other workload you will see it sitting idle. Setting Sync Policy to always forces every write through it -- but if the log device is not fast enough that makes things slower, not faster.

Multiple pairs can be added to scale log throughput. Four 3.84 TB NVMe devices give you two mirrored pairs, and if each pair sustains 4 GB/s you have an 8 GB/s log -- enough to absorb and coalesce the write I/O of roughly eighty HDDs.

Read cache (L2ARC)

Read cache devices do not need to be fault tolerant. Size the cache to roughly the working set of the application, database or VM estate using the pool. A read cache gives no immediate benefit: it has to learn which blocks are worth holding, so judge it after a few days of representative load, not in the first hour.

RAM read cache (ARC)

Scale-up pools use the OpenZFS ARC in RAM as the primary read cache rather than the Linux page cache. Blocks are frequently read more than once, and serving the hot set from RAM is orders of magnitude faster than reading it from disk -- which also takes load off the disks and indirectly improves write performance. Plan for a minimum of 32 GB to 64 GB of RAM on a small system, 96 GB to 128 GB on a medium one, and 256 GB or more on a large one. The OSNEXUS design tools will size this for a given configuration.

Hot-spares

Navigation: Storage Management → Storage Pools → Storage Pool → Add Hot-spare(s) (toolbar)
Add Hot-spare(s) / Recover Storage Pool assigns spare disks to one pool.

All media degrades, so assign at least one hot-spare to every pool and let the appliance heal itself. A spare assigned here is dedicated to this pool. A universal spare, marked from the Physical Disks section instead, is available to any pool in the grid -- see Physical Disks/Devices.

If the pool is already degraded when you add a spare, the repair starts automatically. On a pool that belongs to an HA group the disk list is restricted to devices visible from both nodes, since a spare only one node can see would break the failover.

Remove a spare with Remove Hot-spares from the pool's right-click menu, which returns the disk to the available pool for other uses.

Hot-spare policies

The Hot-Spare Policy on Modify Storage Pool controls how a spare is chosen when a rebuild has to start:

Policy Behaviour
Autoselect best match from assigned or universal spares Default. Any assigned or universal spare of a suitable size may be used.
Autoselect best match from pool assigned spares only Ignores universal spares.
Autoselect exact match from assigned or universal spares Prefers a spare of the same model and capacity as the failed disk before considering anything else.
Autoselect exact match from pool assigned spares only As above, restricted to assigned spares.
Manual Hot-Spare Management Only Disables automatic repair for this pool entirely.

Set Manual Hot-Spare Management Only when an administrator is driving the reconfiguration by hand and does not want the appliance injecting a spare underneath them.

Removing cache devices

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Remove Cache Device
Remove Pool Cache Devices, with separate lists for read cache and write log devices.

This dialog removes read cache and write log devices only; metadata and dedup offload groups do not appear because they cannot be removed. Removing a read cache device raises a confirmation, because a warm cache that is carrying a lot of frequently-read data is doing real work and dropping it will show up as a performance regression until the remaining cache warms up. Write log devices are removed one group per operation.

Removing the last write log device also resets the pool's sync policy to standard, so a pool that was running sync=always against a dedicated log quietly returns to hybrid behaviour rather than forcing every write into the data groups.

Growing and expanding a pool

Grow and Expand solve different problems and are not interchangeable:

  • Grow adds new disks, forming additional device groups. Use it when you have physically added drives.
  • Expand grows the pool into capacity that already exists on its current devices. Use it when the LUNs underneath the pool have been enlarged -- an iSCSI or FC LUN resized on a back-end array, as in an external SAN configuration. It does not apply to a pool built on directly attached disks, whose capacity only changes by adding drives.

Grow

Navigation: Storage Management → Storage Pools → Storage Pool → Grow (toolbar)
Grow Storage Pool adds whole device groups. The number of disks must be an exact multiple of the pool's current group width.

A grow always replicates the pool's existing layout, so the disk count has to be an exact multiple of the current device group width. Selecting a single disk for a mirrored pool is rejected:

Specified group of devices (1) is not a multiple of the Storage Pool's RAID set size.
Must specify groups of '2' devices to grow Storage Pool 'pool-1'.

Adding a device group to a pool that currently has one turns a RAID1 pool into RAID10, a RAIDZ1 pool into RAIDZ1+0, and so on. The new disks must be at least as large as the smallest disk already in the pool.

A grow is refused while the pool is degraded, resilvering or reshaping. If a scrub is running, stop it first. Recover a degraded pool before growing it -- add hot-spares and let the repair finish.

Growing is additive only. Device groups cannot be removed from a pool afterwards, so a pool cannot be shrunk.

Expand

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Expand Storage Pool

Expand takes no disk selection -- pick the pool and click OK. It rescans the underlying devices, extends the partition on each one into the new space, and brings the enlarged devices online.

Expand is not supported on encrypted pools. The menu item is still offered, but the operation is refused:

Expansion of underlying media is only supported on non-encrypted Storage Pools.

Plan capacity growth on an encrypted pool as a Grow with additional devices instead.

Starting, stopping and rescanning

Navigation: Storage Management → Storage Pools → Storage Pool → Stop (toolbar)
Activate/Start Storage Pool. The Current Passphrase field is enabled only for a passphrase-protected encrypted pool.

Stop takes a pool offline for maintenance. Any active client session with a volume in that pool is dropped, so schedule it. Start brings it back. The Current Passphrase field is only enabled for a passphrase-protected pool, and Force is required to start a pool that is degraded.

Rescan scans for hot-plugged disks, then looks for storage pools, and then attempts to activate any offline or newly discovered pool it finds. Run it on the Storage System after physically moving disks in. It is the WUI equivalent of qs pool-scan.

Importing and exporting pools

A pool can be moved between appliances by moving all of its disks. Export it on the old system, move the disks, run Rescan on the new system so the disks are discovered, then import.

Export

Navigation: Storage Management → Storage Pools → Storage Pool → Export (toolbar)
Export Storage Pool. The configuration of the volumes and shares in the pool is saved into the pool before it is detached.

Export detaches the pool from the appliance and removes the pool and its associated objects from the appliance configuration database. Before it does, it writes a fresh metadata archive into the pool's .poolmeta/ directory describing the Storage Volumes, Network Shares and Backup Policies it contained, and an import replays that archive to recreate them. The confirmation prompt warns that the export clears the Replica Association and Schedule Association information held against those volumes and shares, so plan to re-establish replication and snapshot schedules after the import.

Two options are available, both off by default:

  • Release I/O Fence -- drops the SCSI-3 persistent reservations the appliance holds on the pool's disks. Only use this when the disks are being repurposed for a different pool; releasing the fence on disks you intend to re-import removes a protection you want.
  • Clear Encryption Table Entries -- removes the pool's devices from the appliance's crypttab. Relevant only for an encrypted pool.

Import

Navigation: Storage Management → Storage Pools → Storage Pool → Import (toolbar)
Import Storage Pools after clicking Scan. An exported pool is listed under its internal qs-<uuid> name.

Select the Storage System, click Scan, tick the pools to import and click OK. The Passphrase field becomes available when you select a pool that is passphrase protected.

An exported pool appears in the scan list under its internal qs-<uuid> name rather than its friendly name, because the friendly name is a property inside the pool and cannot be read until it is imported. Its own name comes back with it. The scan reports the pool's status, its raw capacity and the number of devices it is built from, which is the quickest way to confirm every disk made the move.

A pool listed as UNAVAIL usually indicates an unclean export; the appliance may need a reboot to clear the stale pool state before the import will succeed. An import can also be refused before it starts if a foreign SCSI-3 reservation is found on the disks -- another appliance still holds them. Powering off the systems and the JBODs is generally the cleanest way to clear stale reservations.

Two options are available from the CLI for a pool whose metadata archive is stale or wrong: qs pool-import --skip-metadata=true imports the volumes and shares without replaying the archive, and qs pool-import --metadata-only=true replays the archive for a pool that is already imported.

Importing an encrypted pool

A software-encrypted pool keeps its key material on the appliance's own boot disk, not on the pool's disks, so moving the disks alone is not enough. Three things live under /etc/cryptconf and all of them have to be present on the target appliance before the pool can be imported:

Directory Holds
/etc/cryptconf/keys The pool's wrapped encryption key and the salt used to wrap it, one pair per pool, named for the pool's UUID. A backup subdirectory keeps copies from before each passphrase change or re-key.
/etc/cryptconf/metadata One file per pool recording which devices make up the pool and their serial numbers. Without this file the pool does not appear in the pre-import scan at all.
/etc/cryptconf/headers A backup of the LUKS header of each encrypted device.

Back up /etc/cryptconf, and keep the backup somewhere other than the appliance. Losing it with no key export, key server copy or HA peer holding the key means the pool is unrecoverable -- the LUKS header backups do not help, because they hold the drive's master key wrapped by the pool key you just lost.

The supported way to move that material between appliances is Export Keys and Import Keys -- see Encryption below. Once the keys are in place, import the pool through the same Import Storage Pools dialog used for unencrypted pools.

Importing a third-party OpenZFS pool

Pools created by other OpenZFS platforms (illumos, FreeBSD, ZFS on Linux) import through the same dialog. QuantaStor identifies pools and volumes by UUID, so the import renames the pool to the qs-<uuid> form and preserves the original name as the pool's friendly name. For more complex cases there is a console-level utility, documented in Console Level OpenZFS Storage Pool Importing.

Scrubbing, trim and health

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Start Pool Scrub

A scrub reads every block in the pool, verifies it against its checksum, and repairs anything that has silently rotted using the pool's redundancy. It is the only mechanism that finds latent corruption in data nobody has read for a year, and it should be run about once a month.

A scrub competes with client I/O. If it is hurting production performance, stop it with Stop Pool Scrub and restart it during a quieter window -- OpenZFS resumes rather than starting over. Progress and the result of the last scrub appear in the pool's properties as the Scan Status.

Do not run a scrub and a grow at the same time; a grow is refused while a scrub is active.

Scheduling scrubs

Nothing schedules a scrub automatically -- creating a pool does not create a schedule for it. Set one up under Schedules as a Maintenance Schedule, or with qs maintenance-schedule-create. A schedule takes a maintenance type of either pool-scrub (the default) or pool-trim, a list of pools, a minimum number of days between runs from 2 to 365 (90 by default), and the day of the week and hour it is allowed to start. One schedule can cover several pools; add and remove them with qs maintenance-schedule-add and qs maintenance-schedule-remove.

Trim

qs pool-trim starts a one-off TRIM pass over the pool's flash media, freeing blocks the filesystem has released so the media can garbage-collect them. Enable SSD Auto Trim does the same thing continuously. Use one or the other on any pool built from SSD or NVMe media; a scheduled pool-trim maintenance schedule is the middle ground when continuous auto-trim is too intrusive.

Health and capacity

qs pool-health-check checks the pool and recursively checks every disk underneath it, which is faster than working down the stack by hand.

Capacity alerting is driven by three thresholds, expressed as the percentage of free space remaining, and they are appliance-wide rather than per pool:

Threshold Default Fires when
Pool Free Space Warning Threshold 30 30% or less of the pool remains free.
Pool Free Space Alert Threshold 10 10% or less remains free.
Pool Free Space Critical Alert Threshold 5 5% or less remains free.

Read them with qs alert-config-get and change them with qs alert-config-set. Alerting only reaches you if a delivery method is configured -- see Alert Manager and Call-home / Alerting.

A thin-provisioned pool can be over-committed, so watch % Provisioned alongside % Utilized in the pool grid. The first tells you what you have promised, the second what has actually been written.

Pool checkpoints

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Create Pool Checkpoint

A checkpoint saves the whole pool's current state so the pool can be rolled back to that point. It exists for upgrade safety: take a checkpoint immediately before a pool version upgrade, confirm the upgrade is good, then discard it.

Only one checkpoint may exist per pool. A second attempt is refused:

A checkpoint already exists for storage pool '<pool-id>'. Only one checkpoint is allowed per pool;
use pool-checkpoint-modify to change its expiration, or pool-checkpoint-delete to discard it first.

Auto-Expiration sets the number of days after which the appliance discards the checkpoint on its own; the dialog defaults to 7 and 0 disables automatic expiry. Set an expiry rather than relying on remembering, because a checkpoint holds on to every block the pool has freed since it was taken, and that space is only released when the checkpoint goes. While a checkpoint exists, changes to the pool's VDEV layout are blocked, so a grow or a device removal has to wait for it. Change the expiry with Modify Pool Checkpoint and discard it with Delete Pool Checkpoint.

Encryption

Encryption is set at pool creation and cannot be added to, or removed from, an existing pool. To encrypt data that is already on an unencrypted pool, create an encrypted pool and migrate the data across. The rest of the encryption controls manage the keys of a pool that is already encrypted. Software encryption is covered in more depth in Software Encryption.

Operation Where What it does
Change/Clear Pool Passphrase pool right-click menu, encrypted pools only Sets, changes or clears the passphrase that protects the pool's keys. Supply the current passphrase to change it; tick Clear Passphrase to remove it and restore automatic startup.
Export Keys Storage Pool Encryption toolbar group Downloads the pool's key material as a .metakey file. Store it securely and separately -- for a pool with no passphrase this file is enough to unlock the pool.
Import Keys Storage Pool Encryption toolbar group Loads a previously exported key block onto a Storage System. Do this before importing the pool itself.
Re-Key Pool Storage Pool Encryption toolbar group Generates a new encryption key and registers it on the pool's devices, keeping a backup of the previous key material. Software encryption only, and only for pools with no passphrase -- passphrase-protected pools are excluded from the dialog's pool list. Each device needs a free LUKS key slot, and there are eight.

Clearing a passphrase is what restores unattended startup: supply the current passphrase, leave the new one blank, and the pool goes back to unlocking itself at boot. Re-keying is gated on that having been done -- attempting it on a passphrase-protected pool is refused:

Encrypted storage pool 'pool-1' requires a passphrase.
Passphrase must be cleared/removed before the pool may be re-keyed.

Passphrases are 10 to 80 characters long and may use letters, digits and the symbols . - : _ (period, hyphen, colon, underscore).

Two further constraints are worth planning around. A passphrase-protected pool cannot fail over in an HA group unless the passphrase is already cached, because the failover has nobody to prompt. And an idle universal hot-spare is not encrypted while it is idle -- it is encrypted at the moment it is pulled into an encrypted pool.

Deleting a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Delete (toolbar)
Delete Storage Pool. Every volume and share in the pool goes with it.

Deletion destroys the pool and every Storage Volume and Network Share in it, and releases the disks for reuse. It is final. The dialog is also covered in Storage Pool Delete. Check the pool name in the dialog before you confirm -- the confirmation prompt states the pool name, the appliance, how utilized it is, how much data it holds, and how many volumes and shares are about to go, precisely so a wrong selection is obvious before you commit.

Force is required when volumes or shares in the pool have active client sessions.

Where multi-admin approval covers Storage Pool deletes, the delete is held until the required number of administrators approve it, and then runs with the security options chosen here.

Security options

Shred Encryption Keys is enabled and ticked automatically for an encrypted pool. It shreds the pool's key material, which makes the data unrecoverable immediately without having to overwrite it.

Shred Data overwrites the disks as part of the delete, using the mode selected beside it. It is off by default; the mode combo is disabled until you tick it and defaults to Auto. Shredding a large pool can take many hours, and the dialog warns you of that when you confirm.

Mode Procedure
Auto (default) Lets the appliance select the shred method for each device.
4-pass DoD 5220.22-M section 8-306 procedure Overwrites all addressable locations with a character, its complement, a random character, then verifies.
4-pass NNSA Policy Letter NAP-14.1-C procedure Overwrites all locations with a pseudorandom pattern twice, then with a known pattern: random, random, 0x00, verify.
US Army AR380-19 method 0x00, 0xff, random.
Fill-zeros single-pass procedure A single pass of zeros.
Random-data single-pass procedure A single pass of random data.
SED Instant Erase Cryptographic erase on a self-encrypting drive. Effectively instant.
NVME Secure The NVMe device's own secure-erase command.
SSD Secure The SSD's own secure-erase command.

The overwrite-based modes are implemented with the Linux scrub utility, which implements these government standards; the SED, NVMe and SSD modes hand the job to the drive's own firmware and are far faster on media that supports them.

Replacing and repairing devices

Navigation: Storage Management → Storage Pools → Storage Pool Device → Replace (toolbar)

The Storage Pool Device toolbar group operates on one member disk at a time, and its buttons are the manual counterpart to automatic hot-spare repair. Select the device on the Storage Pool Devices tab first.

Button Use it to
Replace Swap a failed or failing device for a new one. The new disk must be on the same Storage System, unused, and -- on a pool using SED encryption -- SED capable.
Attach Add a device to an existing device, joining its group, or creating a mirrored group from a single device.
Detach Permanently remove a device from its mirror. Only valid for a member of a mirror group; the appliance refuses it on a parity group.
Offline Take a working device out of service so a disk that is failing but still online can be swapped. This usually puts the pool into a degraded state. If the pool has no hot-spare the confirmation says so, because nothing will step in automatically.
Online Bring an offline device back into service. The pool resilvers it as needed.

Force on Attach and Replace forces the use of a new device that appears to be in use. Only reach for it after an unforced attempt has failed.

Each of these dialogs asks for confirmation naming the specific device, so read the serial number before agreeing. Physical disk-level operations -- identifying a drive by its LED, formatting, marking a universal spare -- live under Physical Disks/Devices.

Note that the pool-level Identify toolbar button and the Identify Storage Pool menu item open the Identify Hardware Controller Disk Device dialog, which lists disks behind a hardware RAID controller. Scale-up pools are built on HBAs rather than RAID cards, so that dialog has nothing to show and neither entry point does anything useful. To blink the LEDs of every disk in a pool, use qs pool-identify --pool=<pool> --blink-type=on --duration=<seconds>.

The pool context menu

Right-clicking a Storage Pool. Several operations are only available here.

Several pool operations have no toolbar button and are reachable only by selecting a pool and right-clicking it: Expand Storage Pool, Remove Hot-spares, Add Log/Cache/Metadata Devices, Remove Cache Device, Start Pool Scrub, Stop Pool Scrub, Create Pool Checkpoint (and Modify and Delete once one exists), Change/Clear Pool Passphrase on an encrypted pool, and Create Storage Tier.

The menu adapts to the pool. Grow, Expand, hot-spare and cache management, scrub and checkpoints are offered on scale-up (ZFS) pools only; a Ceph-backed pool gets a different subset. The boot pool gets Properties and nothing else.

Troubleshooting

Degraded pool

Degraded means one or more member devices has failed or been disconnected. If the pool has an assigned hot-spare, the repair starts on its own within about a minute. If there are no spares, add a disk of the same type that is the same size or larger, mark it as a hot-spare in the Physical Disks section, and the appliance injects it and starts the resilver.

Rebuild is taking a long time

Rebuild time depends on the media type and speed, the load on the system, and the number of device groups in the pool. Every pool should have at least two groups so it can serve I/O from one while another rebuilds. On an HDD pool, add three SSDs for metadata offload -- rebuilds are metadata intensive, and putting the metadata on flash speeds up both rebuilds and scrubs substantially.

Pool performance is inconsistent

Inconsistent performance often means failing media. The appliance watches predictive failure indicators and marks suspect disks with a Warning state. Check the Physical Disks section first: every device should be Normal. Click any device in Warning or Error and read State Detail in the Properties panel for the specific fault. Replace anything reporting predictive failure. Predictive failures generate alerts, so make sure a delivery method is configured in Alert Manager -- otherwise the first you hear of it is a performance complaint.

If every disk reports Normal, run a read performance test from the Physical Disks section. It reads from each selected device with a basic sequential read. A large gap between one or two devices and the rest points at those devices. Open a support ticket and send logs for help interpreting the results.

Small-file performance issues

On an HDD pool, add three SSDs as metadata offload with a 32K small-block offload size. Beyond that, small-file, database, VM and other random small-block workloads want a short parity stripe (RAIDZ2 with 4 data + 2 parity) or mirroring, and they want many device groups -- at least three and ideally four or more. If the pool does not have enough groups, expand the system so it can.

Failed pool

Because of media connectivity

Failed means more devices have failed within a single device group than that group's redundancy can repair, and the pool has stopped. This is frequently connectivity rather than dead media. If an external JBOD is powered off, every disk in it disappears at once, and a pool that is not striped across several JBODs stops. Check the cabling and power on every external JBOD first.

Because of an HA failover

In an HA configuration, a pool showing Failed on one node can simply mean the other node has taken it over -- this node is reporting that it no longer has write access. The node that held the pool when it transitioned to Failed has to be rebooted, because in-flight write I/O is stuck in the disk I/O stack and nothing else clears it. The appliance has several guards that prevent a failed pool from being re-imported on a system until it has been rebooted.

Because of too many failed devices

If more devices in a single group have failed than the layout can repair -- three devices in a 4d+2p group, for instance -- the pool may not be recoverable without a drive repair service. Before concluding that, power everything off and cold boot the whole stack including the JBODs a few times; occasionally one of the failed devices comes back.

Repeated pool failovers

A pool configured for HA has a High-availability Group with failover policies attached, and the default policy fails over when all data ports go offline. Connect the appliance's Ethernet ports to two separate switches so that one switch outage cannot take every port down. Do the same for Fibre Channel: always put an FC switch between the appliance and the clients, because a directly attached FC initiator rebooting looks to the appliance like all FC ports going down and triggers a failover. The heartbeat network is checked before any failover to confirm the peer is reachable, so a total connectivity loss on both nodes does not cause a needless failover.

Volumes or shares showing as Missing

qs pool-clear-missing removes the Storage Volume and Network Share records that the appliance has marked Missing for a pool. It clears database records only; it does not touch the pool. Use it to tidy up after a pool has been re-imported and some objects genuinely no longer exist, and be aware it is not reversible -- the records are gone, and only re-discovery or a metadata re-import brings them back.

CLI command reference

Every pool operation is available from the qs CLI and the REST API. --disk-list accepts disk names, IDs and filter expressions such as [unused], [gt:4TB] or [tag:value], which makes scripted pool builds practical without hardcoding serial numbers.

# create a six-disk double-parity pool with compression on
qs pool-create --name=pool-1 --disk-list=sdd,sde,sdf,sdg,sdh,sdi \
     --raid-type=RAID60 --raid-set-size=6 --compress=true

# add a mirrored pair of write log devices and a read cache device
qs pool-add-write-log --pool=pool-1 --disk-list=sdx,sdy
qs pool-add-read-cache --pool=pool-1 --disk-list=sdz

# start a scrub, then check its progress in the Scan Status field
qs pool-scrub-start --pool=pool-1
qs pool-get --pool=pool-1

Reference a disk by its device name, its full SCSI ID or its object ID. Prefer the SCSI ID in scripts: device names are not stable across reboots and are not unique across the grid, so sdd is ambiguous on a multi-node grid while a SCSI ID is not.

Command Purpose
qs pool-list, qs pool-get, qs pool-search List, inspect and search pools.
qs pool-create Create a pool. Also carries --ashift, --profile, --encrypt, --encryption-type, --passphrase, --key-server-type and --key-server-profile.
qs pool-modify Change name, description, compression, auto-trim, I/O profile, sync policy, cache policies, hot-spare repair policy and small block threshold.
qs pool-grow Add device groups. --dry-run-only=true reports what would happen without changing anything.
qs pool-expand Grow into enlarged underlying devices.
qs pool-import, qs pool-import-encrypted, qs pool-preimport-scan Scan for and import pools.
qs pool-export Export a pool. --metadata-only=true writes a fresh metadata archive without exporting.
qs pool-start, qs pool-stop, qs pool-scan Start, stop and rescan.
qs pool-destroy Delete a pool. --shred-keys and --shred-mode cover the security options.
qs pool-add-read-cache, qs pool-add-write-log, qs pool-add-metadata-offload, qs pool-add-dedup-offload Add the offload tiers.
qs pool-remove-read-cache, qs pool-remove-write-log Remove read cache and write log devices. There is deliberately no equivalent for metadata or dedup offload.
qs pool-add-spare, qs pool-remove-spare Manage pool-assigned hot-spares.
qs pool-scrub-start, qs pool-scrub-stop, qs pool-trim Scrub and trim.
qs pool-checkpoint-create, qs pool-checkpoint-modify, qs pool-checkpoint-delete Manage the pool checkpoint.
qs pool-set-passphrase, qs pool-rekey, qs pool-key-export, qs pool-key-import Encryption key management.
qs pool-device-list, qs pool-device-get, qs pool-device-group-list, qs pool-device-group-get Inspect device groups and their members.
qs pool-device-attach, qs pool-device-detach, qs pool-device-replace, qs pool-device-online, qs pool-device-offline Per-device operations. These are newer than the generated CLI reference and do not have entries there yet; run qs <command> --verbose for the argument list.
qs pool-health-check, qs pool-clear-missing, qs pool-identify, qs pool-portals-list Health check, clear missing records, blink the pool's disk LEDs, list the pool's portals.

Three operations have no dialog in the web interface and are CLI or API only: qs pool-trim, qs pool-health-check and qs pool-clear-missing.

Related pages


Verified against QuantaStor 6.9.0.