Storage Pools

From OSNEXUS Online Documentation Site
(Redirected from Storage Pool Grow)
Jump to navigation Jump to search


A Storage Pool aggregates one or more physical disks into a fault-tolerant pool of storage from which you provision Storage Volumes (block/SAN) and Network Shares (file/NAS). This page covers the scale-up (OpenZFS) Storage Pool: how to choose a layout, how to create and modify a pool, how to add flash offload tiers, and how to import, export, scrub, grow and troubleshoot one.

A single pool serves block and file at the same time, and each resource can be reached over several protocols at once -- a Storage Volume over iSCSI and Fibre Channel together, a Network Share over NFS and SMB together. Scale-out (Ceph) pools are a different object type and are configured under the Scale-out Storage Configuration tab; everything below applies to scale-up pools, which the interface labels Scale-up (ZFS).

The Storage Pools section, with the pool grid, the four detail tabs and the Storage Pool toolbar group.
Section Purpose
Choosing a RAID layout Which layout to use, and the disk counts each one needs.
Creating a Storage Pool Every field on all three tabs of the Create dialog.
Modifying a Storage Pool Compression, sync policy, cache policy, hot-spare policy, I/O profile.
Device groups, offload tiers and hot-spares VDEVs, read cache, write log, metadata and dedup offload, spares.
Growing and expanding a pool The difference between Grow and Expand, and the disk-count rule.
Starting, stopping and rescanning Taking a pool offline and bringing it back.
Importing and exporting pools Moving a pool between appliances, and what export removes.
Scrubbing, trim and health Bit-rot verification, scheduled maintenance, capacity alerting.
Pool checkpoints A whole-pool rollback point for upgrade safety.
Encryption Software and hardware encryption, passphrases, key export and re-key.
Deleting a Storage Pool Deletion and the secure data shredding modes.
Replacing and repairing devices Attach, detach, replace, online and offline.
Troubleshooting Degraded and failed pools, slow rebuilds, inconsistent performance.
CLI command reference Every qs pool command, with the operations that are CLI only.

Choosing a RAID layout

A pool is built from one or more device groups -- VDEVs in OpenZFS terms, and Storage Pool Device Groups in QuantaStor. Each group carries its own redundancy, and the pool stripes across the groups. Fault tolerance is per group: a double-parity group survives two failed members, and a third failure in that same group loses the pool regardless of how healthy the other groups are.

We recommend Double-parity (RAIDZ2+0) with groups of six disks (4 data + 2 parity) as the best balance of fault tolerance, performance and usable capacity for most workloads, and it is what the Create dialog defaults to when enough disks are available. Use Mirrored (RAID10) for virtualization and databases where random IOPS matter most. Use Triple-parity (RAIDZ3+0) or double-parity for archive and backup, where capacity and sequential throughput matter more than IOPS.

Single-parity (RAIDZ1) is not sufficiently fault tolerant for production, and RAID0 has no redundancy and no bit-rot repair at all -- the Create dialog raises an explicit confirmation before it will build a RAID0 pool. Neither is supported for production deployments.

Layout (as labelled in the dialog) Minimum disks offered from Notes
Striped (RAID0) 1 No redundancy, no self-healing. A single device failure loses the pool.
Mirrored (RAID10) 2 Two-way or three-way mirrors. Highest random IOPS; usable capacity is 50% of raw (33% with three-way).
Single-parity (RAIDZ1+0) 3 One parity block per stripe. Not supported for production.
Double-parity (RAIDZ2+0) 4 Two parity blocks per stripe. Recommended default.
Triple-parity (RAIDZ3+0) 7 Three parity blocks per stripe. Best for long-term archive.
Distributed Single-parity (DRAID1 1VS/2VS/3VS) 10 Distributed RAID with 1, 2 or 3 distributed spares.
Distributed Double-parity (DRAID2 1VS/2VS/3VS) 16 As above with two parity blocks per stripe.
Distributed Triple-parity (DRAID3 1VS/2VS/3VS) 20 As above with three parity blocks per stripe.

The RAID Type list is built from the number of disks currently available on the selected Storage System, not the number you have selected. If a layout you expect is missing from the list, the appliance does not have enough unused disks to offer it -- the distributed RAID layouts in particular need at least 10, 16 or 20 available disks depending on parity level. Distributed RAID rebuilds far faster than parity RAID because the spare capacity is spread across every member device, which is why we recommend it for the large 60-, 84-, 90- and 106-bay JBOD backup and archive configurations where a conventional rebuild would run for days.

Build more than one pool where the workloads differ: low-cost fault-tolerant capacity in RAIDZ2 for backup and archive, and a mirrored pool for virtual machines, containers and databases. See Provisioning Tiers for grouping several pools behind one provisioning target.

Enclosure-aware disk selection

On a system with more than one JBOD, QuantaStor detects which enclosure each disk came from and selects disks round-robin across the enclosures so that the members of each device group land in different chassis. A pool built that way keeps running in a degraded state if a whole JBOD is powered off.

Round-robin selection is only used when there are enough enclosures for the layout to actually benefit -- the requirement scales with the group size, so a mirrored pool needs as many enclosures as the mirror is wide, while a RAIDZ1 pool with five-disk groups needs five. When there are too few enclosures the selection falls back to a sequential mode that groups each device group within one enclosure. Within an enclosure the selection also prefers disks of the same size, so a 2 TB disk is never mirrored against a 6 TB one.

Creating a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Create (toolbar)

Creating the first pool is normally the first configuration step on a new appliance. Name the pool, pick the disks and a layout, and click OK. The one setting you cannot change afterwards is encryption -- decide that before you create the pool, because there is no way to encrypt a pool in place later.

The dialog is a three-tab wizard; Previous and Next step through the tabs and OK submits from any of them.

General tab

The General tab of Create Storage Pool: name, layout, set size and disk selection.
Field Default Notes
Name pool-1 (next unused pool-n) 2 to 40 characters, letters, digits and . - _ only. No spaces. Invalid characters are dropped as you type. The name is a label only -- see What a pool looks like on disk.
Description empty Free text.
RAID Type Double-parity (RAIDZ2+0) with 6 or more available disks, otherwise Mirrored (RAID10), otherwise Striped (RAID0) See Choosing a RAID layout. The list is filtered by available disk count.
Set Size Auto Number of disks in each device group. Auto picks a sensible group width for the layout. Disabled for RAID0.
+ and - Select or deselect exactly one more device group's worth of disks, spread across enclosures. Disabled for the distributed RAID layouts, which build a single group.
Auto Config Select as many whole device groups as the available disks allow. Also disabled for distributed RAID.
Storage System the system you launched the dialog from Which grid member's disks to build from. A pool is owned by one system.
Disk grid none selected Only unused, usable disks appear. Boot and system disks, disks already in a pool, journal members, Ceph OSD disks, multipath child paths and anything under 50 MB are filtered out. Search filters across every column; commas separate multiple terms.
Force unchecked Scrubs existing partitions off the selected disks. Required when the disks carry an old ZFS label or LUKS header from a previously exported pool, and when the selected disks do not all report the same physical block size.

Set Size is what turns a parity layout into a striped set of parity groups. A RAIDZ1+0 pool built from eight disks with a set size of four is two four-disk RAIDZ1 groups striped together. Choosing a set size that the selected disk count is not an exact multiple of is rejected -- for mirroring the dialog itself tells you how many more devices to select, and for the parity layouts the appliance returns the error.

Encryption tab

The Encryption tab with software encryption and a user-defined passphrase selected. Encrypt and Store on Key Server is unavailable until a Key Server Profile exists.

All Storage Pool encryption is AES-256. Pick one of three encryption types:

  • No Encryption -- the default.
  • Software Encryption -- LUKS/dm-crypt full-disk encryption applied to every device in the pool. AES-NI hardware acceleration on modern server processors carries the cost, so the overhead is small.
  • Hardware Encryption -- self-encrypting drive (SED) encryption, handled by the drives themselves. Every disk in the pool has to be SED capable; the SED Capable and SED Status columns in the disk grid tell you which are.

If the array presenting the devices already encrypts at rest, QuantaStor should not encrypt them again. Systems such as the Seagate Corvault, Seagate Exos E/EP and Dell PowerVault families encrypt in hardware and present logical devices; layering software encryption over them costs throughput and protects data that is already protected. QuantaStor recognises these devices by vendor and model and skips its own encryption automatically -- see Encryption Bypass, which governs scale-up Storage Pool devices as well as scale-out OSDs. The choice is fixed once a device is provisioned, so check that your media is listed there before you create the pool.

With either encryption type enabled, the Key Protection group chooses where the key that protects the pool key lives:

  • Encrypt with No Passphrase (default) -- the pool unlocks and starts automatically at boot with no operator action. Use this for anything in an HA group.
  • Encrypt with User Defined Passphrase -- a passphrase of 10 to 80 characters, from letters, digits and . - : _. Enter it twice; the eye button reveals what you typed.
  • Encrypt and Store on Key Server -- store the key on an external KMIP or IBM Key Protect key server. This option stays greyed out until at least one Key Server Profile has been created, and then Key Server Profile selects which one to use.

A passphrase stops the pool from starting on its own. After a reboot a passphrase-protected pool stays offline and locked until an administrator runs Start Storage Pool and supplies it. That is exactly what you want on a portable or high-security system, and exactly what you do not want on an HA pool, where the automatic failover has no way to prompt for it. See Encryption below for changing or clearing a passphrase after the fact.

Advanced Settings tab

The Advanced Settings tab: I/O profile, compression, auto-trim, ashift and use case.
Field Default Notes
I/O Profile Default Tunes the member disks for a workload -- read-ahead, queue depth and I/O scheduler. See I/O profiles.
I/O Profile Description Read-only. Shows the selected profile's description.
Enable Compression checked Turns on LZ4 compression. Leave it on unless you are certain the data is incompressible.
Enable SSD Auto Trim unchecked Sends TRIM to flash media continuously so it can reclaim freed blocks. Only useful on a pool built from SSD or NVMe media.
Block Size Offset (ashift) Auto Auto, 9 (512 bytes) or 12 (4K bytes). Leave it on Auto so the appliance derives it from the reported physical block size of the disks. ashift is fixed for the life of the pool, and setting it too small costs you performance while setting it too large wastes capacity on small blocks.
Use Case Default Default provisions Storage Volumes and Network Shares as usual. Ceph OSD reserves the pool for zvols that back Ceph OSDs, and volumes created in such a pool cannot be presented to hosts over iSCSI, FC or NVMe-oF. This is set at create time only and cannot be changed later.

Compression is on by default because it usually increases both usable capacity and throughput at once -- a modern CPU compresses faster than the media can read or write, so moving less data is a net win. Turn it off for workloads that are already compressed, which is common in media and entertainment. The algorithm can be changed at any time from Modify Storage Pool; only the on/off choice is available at create time.

What a pool looks like on disk

The pool's name in the interface is a label. On the appliance the pool is always named qs-<pool-uuid>, mounted at /mnt/storage-pools/qs-<pool-uuid>, and the friendly name is carried in a quantastor:name property on the pool. That is why renaming a pool is instant and why the name survives a move to another appliance.

Two directories are maintained inside every pool:

  • .poolmeta/ -- a metadata archive describing the Storage Volumes, Network Shares, hosts, portal groups and schedule associations in the pool. It is refreshed periodically and always rewritten immediately before an export, and it is what an import replays to bring those objects back.
  • .qsbackups/ -- rolling hourly and daily backups of the appliance configuration database.

Modifying a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Modify (toolbar)
Modify Storage Pool. Everything except encryption and the on-disk layout can be changed here at any time.

Everything on this dialog can be changed on a running pool with volumes and shares in it. Encryption is the one pool property that is not here, because it cannot be changed after creation.

Field Default Notes
Storage Pool the selected pool Switch to a different pool without reopening the dialog.
Name, Description current values Renaming is metadata only and does not disturb I/O.
Hot-Spare Policy Autoselect best match from assigned or universal spares See Hot-spare policies.
I/O Profile the pool's current profile See I/O profiles.
Sync Policy standard standard honours the application's O_SYNC flag, so hypervisors and databases get write-through and everything else gets write-back. always forces every write through the write log. disabled makes all writes asynchronous.
Cache Policy Primary all What the RAM read cache (ARC) holds: all, metadata only, or none.
Cache Policy Secondary all The same choice for the SSD read cache (L2ARC).
Small Block Offload Disabled Maximum write size routed to the metadata offload device group. Only meaningful once a metadata offload group exists. We recommend 8K to 32K depending on how much SSD capacity you gave it.
Compression on on (LZ4), off, or a specific algorithm: lz4, lzjb, zle, gzip and gzip-1 through gzip-9, zstd and zstd-1 through zstd-19, and zstd-fast and zstd-fast-1 through zstd-fast-10.
Enable SSD Auto Trim the pool's current setting As on the Create dialog.
Encryption Key Settings Encrypt and Store Locally Enabled only on an encrypted pool. Moves the pool key between local storage and a key server. The key-server option needs a Key Server Profile to exist.

Changing compression affects new writes only; existing blocks keep whatever they were written with. Changing Sync Policy to disabled raises a confirmation, and rightly so -- it acknowledges writes before they are durable and can cause data loss on an unexpected power failure. It is not supported for production use.

The Cache Policy settings are the lever for a workload that pollutes the cache rather than benefiting from it -- a large sequential backup ingest, for example, can evict the working set of everything else on the appliance. Setting the primary cache policy to metadata keeps the pool's directory structure cached while leaving the data blocks out of RAM.

I/O profiles

An I/O profile is a bundle of block-device tunings applied to the pool's member disks -- read-ahead size, queue depth, I/O scheduler, and the SCST target driver thread and tasklet counts -- with separate values for HDD, SSD and NVMe media. It is not a filesystem setting, and it is re-applied every time the pool starts, on HA failover, and after a grow, attach or replace.

Seven profiles ship with the appliance:

Profile Optimizes for
Default General purpose server application workloads.
Virtualization Virtual machine workloads such as database and web servers under ESXi, XenServer, Hyper-V or Virtuozzo.
Disk Archive Disk-to-disk backup and archive applications.
Media Post-Production (Ingest Optimized) Media post-production editing.
Media Post-Production (Playback Optimized) Media post-production playback.
Edgeware IP TV Optimized The Edgeware IP TV application workflow.
Edgeware Web TV Optimized The Edgeware Web TV application workflow.

The profiles are defined in /opt/osnexus/quantastor/conf/qs_io_profiles.conf on the appliance; use qs pool-profile-list and qs pool-profile-get to read the values a profile actually applies. Profiles are per pool; the system-wide OpenZFS module parameters are a separate mechanism, covered in Storage System Optimization. For choosing between them, and for RAM sizing, see Performance Tuning.

Device groups, offload tiers and hot-spares

The Storage Pool Devices tab. The RAID Segment column names the device group each disk belongs to -- a data group, cache, or logs.

The four detail tabs under the pool grid show the pool from different angles. Storage Pool Device Groups lists the VDEVs with their RAID level and size, and Storage Pool Devices lists the individual member disks with the group each one belongs to in the RAID Segment column. A data group appears as mirror-0, raidz2-0 and so on; the offload tiers appear as cache and logs.

The number of data device groups matters as much as the layout. A pool with a single group has no way to spread load while that group is rebuilding, so build every pool with at least two, and prefer four or more for anything IOPS sensitive.

Adding offload devices

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Add Log/Cache/Metadata Devices
The Metadata Offload tab of Add Log/Cache/Metadata Device(s). The four tabs cover write log, read cache, metadata offload and dedup offload.

This dialog has one tab per offload class. Move disks from Available Disks into the selected list with the arrow buttons, or use Auto Config to pick a matched pair. Offload devices are dedicated to one pool and cannot be shared between pools.

Tab What it adds Redundancy Removable
Write Log SLOG/ZIL -- an external intent log so synchronous writes do not have to land in the data groups first. Automatically mirrored. Add either a single device or one or more pairs; any other odd count is rejected. Yes, one group per operation
Read Cache L2ARC -- a second-level read cache below RAM. None, and none is needed; a failed cache device costs you cache, not data. Yes
Metadata Offload A ZFS special VDEV holding filesystem metadata and, if you set Small Block Offload, small blocks as well. Always mirrored. A single device is refused. No -- permanent
Deduplication Offload A dedicated VDEV for the deduplication table. Adding one turns deduplication on for the pool. Always mirrored. No -- permanent

Metadata and deduplication offload device groups cannot be removed once added. The dialog says so, and there is no CLI command to remove them either. Size them properly the first time, and mirror them properly -- losing that group loses the pool.

On an encrypted pool every device added as cache, log, metadata or dedup offload is encrypted automatically as part of the operation. Never add devices to a QuantaStor pool with zpool add directly: an encrypted pool grown that way ends up with plaintext members, and the appliance raises an alert when it detects them.

Metadata and small-file offload

A scale-up pool built from HDDs should always include SSDs for metadata and small-file offload. The effect on scrub time, replication, HA failover and small-file performance is large enough that we do not design HDD-based pools without it -- three SSDs with a 32K small-block offload size is a good starting point.

An all-flash pool does not need it and should not use it, with one exception: QLC media handles small-block I/O poorly, so offloading metadata and small blocks onto TLC media in front of a QLC pool is worthwhile.

Write log (SLOG/ZIL)

Modern deployments use the write log less than they used to. Its value was in making an HDD pool usable for database and virtual-machine workloads, and those workloads should be on flash now that NVMe and SAS SSDs are affordable.

Note that the default standard sync policy bypasses the write log unless the write carries O_SYNC, which hypervisors and databases set and most other applications do not. If you add a write log for some other workload you will see it sitting idle. Setting Sync Policy to always forces every write through it -- but if the log device is not fast enough that makes things slower, not faster.

Multiple pairs can be added to scale log throughput. Four 3.84 TB NVMe devices give you two mirrored pairs, and if each pair sustains 4 GB/s you have an 8 GB/s log -- enough to absorb and coalesce the write I/O of roughly eighty HDDs.

Read cache (L2ARC)

Read cache devices do not need to be fault tolerant. Size the cache to roughly the working set of the application, database or VM estate using the pool. A read cache gives no immediate benefit: it has to learn which blocks are worth holding, so judge it after a few days of representative load, not in the first hour.

RAM read cache (ARC)

Scale-up pools use the OpenZFS ARC in RAM as the primary read cache rather than the Linux page cache. Blocks are frequently read more than once, and serving the hot set from RAM is orders of magnitude faster than reading it from disk -- which also takes load off the disks and indirectly improves write performance. Plan for a minimum of 32 GB to 64 GB of RAM on a small system, 96 GB to 128 GB on a medium one, and 256 GB or more on a large one. The OSNEXUS design tools will size this for a given configuration.

Hot-spares

Navigation: Storage Management → Storage Pools → Storage Pool → Add Hot-spare(s) (toolbar)
Add Hot-spare(s) / Recover Storage Pool assigns spare disks to one pool.

All media degrades, so assign at least one hot-spare to every pool and let the appliance heal itself. A spare assigned here is dedicated to this pool. A universal spare, marked from the Physical Disks section instead, is available to any pool in the grid -- see Physical Disks/Devices.

If the pool is already degraded when you add a spare, the repair starts automatically. On a pool that belongs to an HA group the disk list is restricted to devices visible from both nodes, since a spare only one node can see would break the failover.

Remove a spare with Remove Hot-spares from the pool's right-click menu, which returns the disk to the available pool for other uses.

Hot-spare policies

The Hot-Spare Policy on Modify Storage Pool controls how a spare is chosen when a rebuild has to start:

Policy Behaviour
Autoselect best match from assigned or universal spares Default. Any assigned or universal spare of a suitable size may be used.
Autoselect best match from pool assigned spares only Ignores universal spares.
Autoselect exact match from assigned or universal spares Prefers a spare of the same model and capacity as the failed disk before considering anything else.
Autoselect exact match from pool assigned spares only As above, restricted to assigned spares.
Manual Hot-Spare Management Only Disables automatic repair for this pool entirely.

Set Manual Hot-Spare Management Only when an administrator is driving the reconfiguration by hand and does not want the appliance injecting a spare underneath them.

Removing cache devices

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Remove Cache Device
Remove Pool Cache Devices, with separate lists for read cache and write log devices.

This dialog removes read cache and write log devices only; metadata and dedup offload groups do not appear because they cannot be removed. Removing a read cache device raises a confirmation, because a warm cache that is carrying a lot of frequently-read data is doing real work and dropping it will show up as a performance regression until the remaining cache warms up. Write log devices are removed one group per operation.

Removing the last write log device also resets the pool's sync policy to standard, so a pool that was running sync=always against a dedicated log quietly returns to hybrid behaviour rather than forcing every write into the data groups.

Growing and expanding a pool

Grow and Expand solve different problems and are not interchangeable:

  • Grow adds new disks, forming additional device groups. Use it when you have physically added drives.
  • Expand grows the pool into capacity that already exists on its current devices. Use it when the LUNs underneath the pool have been enlarged -- an iSCSI or FC LUN resized on a back-end array, as in an external SAN configuration. It does not apply to a pool built on directly attached disks, whose capacity only changes by adding drives.

Grow

Navigation: Storage Management → Storage Pools → Storage Pool → Grow (toolbar)
Grow Storage Pool adds whole device groups. The number of disks must be an exact multiple of the pool's current group width.

A grow always replicates the pool's existing layout, so the disk count has to be an exact multiple of the current device group width. Selecting a single disk for a mirrored pool is rejected:

Specified group of devices (1) is not a multiple of the Storage Pool's RAID set size.
Must specify groups of '2' devices to grow Storage Pool 'pool-1'.

Adding a device group to a pool that currently has one turns a RAID1 pool into RAID10, a RAIDZ1 pool into RAIDZ1+0, and so on. The new disks must be at least as large as the smallest disk already in the pool.

A grow is refused while the pool is degraded, resilvering or reshaping. If a scrub is running, stop it first. Recover a degraded pool before growing it -- add hot-spares and let the repair finish.

Growing is additive only. Device groups cannot be removed from a pool afterwards, so a pool cannot be shrunk.

Expand

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Expand Storage Pool

Expand takes no disk selection -- pick the pool and click OK. It rescans the underlying devices, extends the partition on each one into the new space, and brings the enlarged devices online.

Expand is not supported on encrypted pools. The menu item is still offered, but the operation is refused:

Expansion of underlying media is only supported on non-encrypted Storage Pools.

Plan capacity growth on an encrypted pool as a Grow with additional devices instead.

Starting, stopping and rescanning

Navigation: Storage Management → Storage Pools → Storage Pool → Stop (toolbar)
Activate/Start Storage Pool. The Current Passphrase field is enabled only for a passphrase-protected encrypted pool.

Stop takes a pool offline for maintenance. Any active client session with a volume in that pool is dropped, so schedule it. Start brings it back. The Current Passphrase field is only enabled for a passphrase-protected pool, and Force is required to start a pool that is degraded.

Rescan scans for hot-plugged disks, then looks for storage pools, and then attempts to activate any offline or newly discovered pool it finds. Run it on the Storage System after physically moving disks in. It is the WUI equivalent of qs pool-scan.

Importing and exporting pools

A pool can be moved between appliances by moving all of its disks. Export it on the old system, move the disks, run Rescan on the new system so the disks are discovered, then import.

Export

Navigation: Storage Management → Storage Pools → Storage Pool → Export (toolbar)
Export Storage Pool. The configuration of the volumes and shares in the pool is saved into the pool before it is detached.

Export detaches the pool from the appliance and removes the pool and its associated objects from the appliance configuration database. Before it does, it writes a fresh metadata archive into the pool's .poolmeta/ directory describing the Storage Volumes, Network Shares and Backup Policies it contained, and an import replays that archive to recreate them. The confirmation prompt warns that the export clears the Replica Association and Schedule Association information held against those volumes and shares, so plan to re-establish replication and snapshot schedules after the import.

Two options are available, both off by default:

  • Release I/O Fence -- drops the SCSI-3 persistent reservations the appliance holds on the pool's disks. Only use this when the disks are being repurposed for a different pool; releasing the fence on disks you intend to re-import removes a protection you want.
  • Clear Encryption Table Entries -- removes the pool's devices from the appliance's crypttab. Relevant only for an encrypted pool.

Import

Navigation: Storage Management → Storage Pools → Storage Pool → Import (toolbar)
Import Storage Pools after clicking Scan. An exported pool is listed under its internal qs-<uuid> name.

Select the Storage System, click Scan, tick the pools to import and click OK. The Passphrase field becomes available when you select a pool that is passphrase protected.

An exported pool appears in the scan list under its internal qs-<uuid> name rather than its friendly name, because the friendly name is a property inside the pool and cannot be read until it is imported. Its own name comes back with it. The scan reports the pool's status, its raw capacity and the number of devices it is built from, which is the quickest way to confirm every disk made the move.

A pool listed as UNAVAIL usually indicates an unclean export; the appliance may need a reboot to clear the stale pool state before the import will succeed. An import can also be refused before it starts if a foreign SCSI-3 reservation is found on the disks -- another appliance still holds them. Powering off the systems and the JBODs is generally the cleanest way to clear stale reservations.

Two options are available from the CLI for a pool whose metadata archive is stale or wrong: qs pool-import --skip-metadata=true imports the volumes and shares without replaying the archive, and qs pool-import --metadata-only=true replays the archive for a pool that is already imported.

Importing an encrypted pool

A software-encrypted pool keeps its key material on the appliance's own boot disk, not on the pool's disks, so moving the disks alone is not enough. Three things live under /etc/cryptconf and all of them have to be present on the target appliance before the pool can be imported:

Directory Holds
/etc/cryptconf/keys The pool's wrapped encryption key and the salt used to wrap it, one pair per pool, named for the pool's UUID. A backup subdirectory keeps copies from before each passphrase change or re-key.
/etc/cryptconf/metadata One file per pool recording which devices make up the pool and their serial numbers. Without this file the pool does not appear in the pre-import scan at all.
/etc/cryptconf/headers A backup of the LUKS header of each encrypted device.

Back up /etc/cryptconf, and keep the backup somewhere other than the appliance. Losing it with no key export, key server copy or HA peer holding the key means the pool is unrecoverable -- the LUKS header backups do not help, because they hold the drive's master key wrapped by the pool key you just lost.

The supported way to move that material between appliances is Export Keys and Import Keys -- see Encryption below. Once the keys are in place, import the pool through the same Import Storage Pools dialog used for unencrypted pools.

Importing a third-party OpenZFS pool

Pools created by other OpenZFS platforms (illumos, FreeBSD, ZFS on Linux) import through the same dialog. QuantaStor identifies pools and volumes by UUID, so the import renames the pool to the qs-<uuid> form and preserves the original name as the pool's friendly name. For more complex cases there is a console-level utility, documented in Console Level OpenZFS Storage Pool Importing.

Scrubbing, trim and health

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Start Pool Scrub

A scrub reads every block in the pool, verifies it against its checksum, and repairs anything that has silently rotted using the pool's redundancy. It is the only mechanism that finds latent corruption in data nobody has read for a year, and it should be run about once a month.

A scrub competes with client I/O. If it is hurting production performance, stop it with Stop Pool Scrub and restart it during a quieter window -- OpenZFS resumes rather than starting over. Progress and the result of the last scrub appear in the pool's properties as the Scan Status.

Do not run a scrub and a grow at the same time; a grow is refused while a scrub is active.

Scheduling scrubs

Nothing schedules a scrub automatically -- creating a pool does not create a schedule for it. Set one up under Schedules as a Maintenance Schedule, or with qs maintenance-schedule-create. A schedule takes a maintenance type of either pool-scrub (the default) or pool-trim, a list of pools, a minimum number of days between runs from 2 to 365 (90 by default), and the day of the week and hour it is allowed to start. One schedule can cover several pools; add and remove them with qs maintenance-schedule-add and qs maintenance-schedule-remove.

Trim

qs pool-trim starts a one-off TRIM pass over the pool's flash media, freeing blocks the filesystem has released so the media can garbage-collect them. Enable SSD Auto Trim does the same thing continuously. Use one or the other on any pool built from SSD or NVMe media; a scheduled pool-trim maintenance schedule is the middle ground when continuous auto-trim is too intrusive.

Health and capacity

qs pool-health-check checks the pool and recursively checks every disk underneath it, which is faster than working down the stack by hand.

Capacity alerting is driven by three thresholds, expressed as the percentage of free space remaining, and they are appliance-wide rather than per pool:

Threshold Default Fires when
Pool Free Space Warning Threshold 30 30% or less of the pool remains free.
Pool Free Space Alert Threshold 10 10% or less remains free.
Pool Free Space Critical Alert Threshold 5 5% or less remains free.

Read them with qs alert-config-get and change them with qs alert-config-set. Alerting only reaches you if a delivery method is configured -- see Alert Manager and Call-home / Alerting.

A thin-provisioned pool can be over-committed, so watch % Provisioned alongside % Utilized in the pool grid. The first tells you what you have promised, the second what has actually been written.

Pool checkpoints

Navigation: Storage Management → Storage Pools → Storage Pool (select + right-click) → Create Pool Checkpoint

A checkpoint saves the whole pool's current state so the pool can be rolled back to that point. It exists for upgrade safety: take a checkpoint immediately before a pool version upgrade, confirm the upgrade is good, then discard it.

Only one checkpoint may exist per pool. A second attempt is refused:

A checkpoint already exists for storage pool '<pool-id>'. Only one checkpoint is allowed per pool;
use pool-checkpoint-modify to change its expiration, or pool-checkpoint-delete to discard it first.

Auto-Expiration sets the number of days after which the appliance discards the checkpoint on its own; the dialog defaults to 7 and 0 disables automatic expiry. Set an expiry rather than relying on remembering, because a checkpoint holds on to every block the pool has freed since it was taken, and that space is only released when the checkpoint goes. While a checkpoint exists, changes to the pool's VDEV layout are blocked, so a grow or a device removal has to wait for it. Change the expiry with Modify Pool Checkpoint and discard it with Delete Pool Checkpoint.

Encryption

Encryption is set at pool creation and cannot be added to, or removed from, an existing pool. To encrypt data that is already on an unencrypted pool, create an encrypted pool and migrate the data across. The rest of the encryption controls manage the keys of a pool that is already encrypted. Software encryption is covered in more depth in Software Encryption.

Operation Where What it does
Change/Clear Pool Passphrase pool right-click menu, encrypted pools only Sets, changes or clears the passphrase that protects the pool's keys. Supply the current passphrase to change it; tick Clear Passphrase to remove it and restore automatic startup.
Export Keys Storage Pool Encryption toolbar group Downloads the pool's key material as a .metakey file. Store it securely and separately -- for a pool with no passphrase this file is enough to unlock the pool.
Import Keys Storage Pool Encryption toolbar group Loads a previously exported key block onto a Storage System. Do this before importing the pool itself.
Re-Key Pool Storage Pool Encryption toolbar group Generates a new encryption key and registers it on the pool's devices, keeping a backup of the previous key material. Software encryption only, and only for pools with no passphrase -- passphrase-protected pools are excluded from the dialog's pool list. Each device needs a free LUKS key slot, and there are eight.

Clearing a passphrase is what restores unattended startup: supply the current passphrase, leave the new one blank, and the pool goes back to unlocking itself at boot. Re-keying is gated on that having been done -- attempting it on a passphrase-protected pool is refused:

Encrypted storage pool 'pool-1' requires a passphrase.
Passphrase must be cleared/removed before the pool may be re-keyed.

Passphrases are 10 to 80 characters long and may use letters, digits and the symbols . - : _ (period, hyphen, colon, underscore).

Two further constraints are worth planning around. A passphrase-protected pool cannot fail over in an HA group unless the passphrase is already cached, because the failover has nobody to prompt. And an idle universal hot-spare is not encrypted while it is idle -- it is encrypted at the moment it is pulled into an encrypted pool.

Deleting a Storage Pool

Navigation: Storage Management → Storage Pools → Storage Pool → Delete (toolbar)
Delete Storage Pool. Every volume and share in the pool goes with it.

Deletion destroys the pool and every Storage Volume and Network Share in it, and releases the disks for reuse. It is final. The dialog is also covered in Storage Pool Delete. Check the pool name in the dialog before you confirm -- the confirmation prompt states the pool name, the appliance, how utilized it is, how much data it holds, and how many volumes and shares are about to go, precisely so a wrong selection is obvious before you commit.

Force is required when volumes or shares in the pool have active client sessions.

Security options

Shred Encryption Keys is enabled and ticked automatically for an encrypted pool. It shreds the pool's key material, which makes the data unrecoverable immediately without having to overwrite it.

Shred Data overwrites the disks as part of the delete, using the mode selected beside it. It is off by default; the mode combo is disabled until you tick it and defaults to Auto. Shredding a large pool can take many hours, and the dialog warns you of that when you confirm.

Mode Procedure
Auto (default) Lets the appliance select the shred method for each device.
4-pass DoD 5220.22-M section 8-306 procedure Overwrites all addressable locations with a character, its complement, a random character, then verifies.
4-pass NNSA Policy Letter NAP-14.1-C procedure Overwrites all locations with a pseudorandom pattern twice, then with a known pattern: random, random, 0x00, verify.
US Army AR380-19 method 0x00, 0xff, random.
Fill-zeros single-pass procedure A single pass of zeros.
Random-data single-pass procedure A single pass of random data.
SED Instant Erase Cryptographic erase on a self-encrypting drive. Effectively instant.
NVME Secure The NVMe device's own secure-erase command.
SSD Secure The SSD's own secure-erase command.

The overwrite-based modes are implemented with the Linux scrub utility, which implements these government standards; the SED, NVMe and SSD modes hand the job to the drive's own firmware and are far faster on media that supports them.

Replacing and repairing devices

Navigation: Storage Management → Storage Pools → Storage Pool Device → Replace (toolbar)

The Storage Pool Device toolbar group operates on one member disk at a time, and its buttons are the manual counterpart to automatic hot-spare repair. Select the device on the Storage Pool Devices tab first.

Button Use it to
Replace Swap a failed or failing device for a new one. The new disk must be on the same Storage System, unused, and -- on a pool using SED encryption -- SED capable.
Attach Add a device to an existing device, joining its group, or creating a mirrored group from a single device.
Detach Permanently remove a device from its mirror. Only valid for a member of a mirror group; the appliance refuses it on a parity group.
Offline Take a working device out of service so a disk that is failing but still online can be swapped. This usually puts the pool into a degraded state. If the pool has no hot-spare the confirmation says so, because nothing will step in automatically.
Online Bring an offline device back into service. The pool resilvers it as needed.

Force on Attach and Replace forces the use of a new device that appears to be in use. Only reach for it after an unforced attempt has failed.

Each of these dialogs asks for confirmation naming the specific device, so read the serial number before agreeing. Physical disk-level operations -- identifying a drive by its LED, formatting, marking a universal spare -- live under Physical Disks/Devices.

Note that the pool-level Identify toolbar button and the Identify Storage Pool menu item open the Identify Hardware Controller Disk Device dialog, which lists disks behind a hardware RAID controller. Scale-up pools are built on HBAs rather than RAID cards, so that dialog has nothing to show and neither entry point does anything useful. To blink the LEDs of every disk in a pool, use qs pool-identify --pool=<pool> --blink-type=on --duration=<seconds>.

The pool context menu

Right-clicking a Storage Pool. Several operations are only available here.

Several pool operations have no toolbar button and are reachable only by selecting a pool and right-clicking it: Expand Storage Pool, Remove Hot-spares, Add Log/Cache/Metadata Devices, Remove Cache Device, Start Pool Scrub, Stop Pool Scrub, Create Pool Checkpoint (and Modify and Delete once one exists), Change/Clear Pool Passphrase on an encrypted pool, and Create Storage Tier.

The menu adapts to the pool. Grow, Expand, hot-spare and cache management, scrub and checkpoints are offered on scale-up (ZFS) pools only; a Ceph-backed pool gets a different subset. The boot pool gets Properties and nothing else.

Troubleshooting

Degraded pool

Degraded means one or more member devices has failed or been disconnected. If the pool has an assigned hot-spare, the repair starts on its own within about a minute. If there are no spares, add a disk of the same type that is the same size or larger, mark it as a hot-spare in the Physical Disks section, and the appliance injects it and starts the resilver.

Rebuild is taking a long time

Rebuild time depends on the media type and speed, the load on the system, and the number of device groups in the pool. Every pool should have at least two groups so it can serve I/O from one while another rebuilds. On an HDD pool, add three SSDs for metadata offload -- rebuilds are metadata intensive, and putting the metadata on flash speeds up both rebuilds and scrubs substantially.

Pool performance is inconsistent

Inconsistent performance often means failing media. The appliance watches predictive failure indicators and marks suspect disks with a Warning state. Check the Physical Disks section first: every device should be Normal. Click any device in Warning or Error and read State Detail in the Properties panel for the specific fault. Replace anything reporting predictive failure. Predictive failures generate alerts, so make sure a delivery method is configured in Alert Manager -- otherwise the first you hear of it is a performance complaint.

If every disk reports Normal, run a read performance test from the Physical Disks section. It reads from each selected device with a basic sequential read. A large gap between one or two devices and the rest points at those devices. Open a support ticket and send logs for help interpreting the results.

Small-file performance issues

On an HDD pool, add three SSDs as metadata offload with a 32K small-block offload size. Beyond that, small-file, database, VM and other random small-block workloads want a short parity stripe (RAIDZ2 with 4 data + 2 parity) or mirroring, and they want many device groups -- at least three and ideally four or more. If the pool does not have enough groups, expand the system so it can.

Failed pool

Because of media connectivity

Failed means more devices have failed within a single device group than that group's redundancy can repair, and the pool has stopped. This is frequently connectivity rather than dead media. If an external JBOD is powered off, every disk in it disappears at once, and a pool that is not striped across several JBODs stops. Check the cabling and power on every external JBOD first.

Because of an HA failover

In an HA configuration, a pool showing Failed on one node can simply mean the other node has taken it over -- this node is reporting that it no longer has write access. The node that held the pool when it transitioned to Failed has to be rebooted, because in-flight write I/O is stuck in the disk I/O stack and nothing else clears it. The appliance has several guards that prevent a failed pool from being re-imported on a system until it has been rebooted.

Because of too many failed devices

If more devices in a single group have failed than the layout can repair -- three devices in a 4d+2p group, for instance -- the pool may not be recoverable without a drive repair service. Before concluding that, power everything off and cold boot the whole stack including the JBODs a few times; occasionally one of the failed devices comes back.

Repeated pool failovers

A pool configured for HA has a High-availability Group with failover policies attached, and the default policy fails over when all data ports go offline. Connect the appliance's Ethernet ports to two separate switches so that one switch outage cannot take every port down. Do the same for Fibre Channel: always put an FC switch between the appliance and the clients, because a directly attached FC initiator rebooting looks to the appliance like all FC ports going down and triggers a failover. The heartbeat network is checked before any failover to confirm the peer is reachable, so a total connectivity loss on both nodes does not cause a needless failover.

Volumes or shares showing as Missing

qs pool-clear-missing removes the Storage Volume and Network Share records that the appliance has marked Missing for a pool. It clears database records only; it does not touch the pool. Use it to tidy up after a pool has been re-imported and some objects genuinely no longer exist, and be aware it is not reversible -- the records are gone, and only re-discovery or a metadata re-import brings them back.

CLI command reference

Every pool operation is available from the qs CLI and the REST API. --disk-list accepts disk names, IDs and filter expressions such as [unused], [gt:4TB] or [tag:value], which makes scripted pool builds practical without hardcoding serial numbers.

# create a six-disk double-parity pool with compression on
qs pool-create --name=pool-1 --disk-list=sdd,sde,sdf,sdg,sdh,sdi \
     --raid-type=RAID60 --raid-set-size=6 --compress=true

# add a mirrored pair of write log devices and a read cache device
qs pool-add-write-log --pool=pool-1 --disk-list=sdx,sdy
qs pool-add-read-cache --pool=pool-1 --disk-list=sdz

# start a scrub, then check its progress in the Scan Status field
qs pool-scrub-start --pool=pool-1
qs pool-get --pool=pool-1

Reference a disk by its device name, its full SCSI ID or its object ID. Prefer the SCSI ID in scripts: device names are not stable across reboots and are not unique across the grid, so sdd is ambiguous on a multi-node grid while a SCSI ID is not.

Command Purpose
qs pool-list, qs pool-get, qs pool-search List, inspect and search pools.
qs pool-create Create a pool. Also carries --ashift, --profile, --encrypt, --encryption-type, --passphrase, --key-server-type and --key-server-profile.
qs pool-modify Change name, description, compression, auto-trim, I/O profile, sync policy, cache policies, hot-spare repair policy and small block threshold.
qs pool-grow Add device groups. --dry-run-only=true reports what would happen without changing anything.
qs pool-expand Grow into enlarged underlying devices.
qs pool-import, qs pool-import-encrypted, qs pool-preimport-scan Scan for and import pools.
qs pool-export Export a pool. --metadata-only=true writes a fresh metadata archive without exporting.
qs pool-start, qs pool-stop, qs pool-scan Start, stop and rescan.
qs pool-destroy Delete a pool. --shred-keys and --shred-mode cover the security options.
qs pool-add-read-cache, qs pool-add-write-log, qs pool-add-metadata-offload, qs pool-add-dedup-offload Add the offload tiers.
qs pool-remove-read-cache, qs pool-remove-write-log Remove read cache and write log devices. There is deliberately no equivalent for metadata or dedup offload.
qs pool-add-spare, qs pool-remove-spare Manage pool-assigned hot-spares.
qs pool-scrub-start, qs pool-scrub-stop, qs pool-trim Scrub and trim.
qs pool-checkpoint-create, qs pool-checkpoint-modify, qs pool-checkpoint-delete Manage the pool checkpoint.
qs pool-set-passphrase, qs pool-rekey, qs pool-key-export, qs pool-key-import Encryption key management.
qs pool-device-list, qs pool-device-get, qs pool-device-group-list, qs pool-device-group-get Inspect device groups and their members.
qs pool-device-attach, qs pool-device-detach, qs pool-device-replace, qs pool-device-online, qs pool-device-offline Per-device operations. These are newer than the generated CLI reference and do not have entries there yet; run qs <command> --verbose for the argument list.
qs pool-health-check, qs pool-clear-missing, qs pool-identify, qs pool-portals-list Health check, clear missing records, blink the pool's disk LEDs, list the pool's portals.

Three operations have no dialog in the web interface and are CLI or API only: qs pool-trim, qs pool-health-check and qs pool-clear-missing.

Related pages


Verified against QuantaStor 6.9.0.