Provisioning Tiers

A Storage Provisioning Tier is a named group of Storage Pools that can be used in place of a specific pool when provisioning a Storage Volume or a Network Share. You hand QuantaStor the tier, and QuantaStor picks the pool inside it -- which is what makes tiers useful to automation frameworks such as OpenStack Cinder, where the caller has no view of how full each pool is.

Provisioning Tiers are not auto-tiering. QuantaStor also has a feature that moves cold files out to a Cloud Container and leaves a stub behind; that is the Auto-tier Files mode of a Backup Policy, and it has nothing in common with a Provisioning Tier -- different objects, different dialogs, no shared code. A Provisioning Tier never moves data. It only chooses where new data is placed at the moment it is created. Object storage also has its own unrelated Storage-Class rules for auto-tiering S3 objects between data pools.

Section	Purpose
What a tier solves	Why grouping pools helps, and what a tier is made of
Tiers and Storage Pools	Membership rules, grid scope, and what happens when a member pool goes away
How QuantaStor picks a pool	The selection algorithm, in order, with a worked example
Capacity, Max Provisionable and over-provisioning	What the tier's capacity figures mean and where a request fails
Creating a tier	Create Storage Tier, field by field
Modifying a tier	Changing the name and attributes
Adding and removing pools	Changing membership
Deleting a tier	Removing a tier without touching its pools
Provisioning from a tier	From the web interface and from the CLI
Automation and the REST API	What a framework such as OpenStack Cinder does with a tier
Performance and tuning considerations	Mixing pool types and tuning profiles in one tier

What a tier solves

Tiers appear at the top of the Storage Pools section of the tree, with their member pools nested beneath them. The pools also remain listed in their own right.

In a grid with many storage systems and many pools, choosing where to put a new volume or share means answering several questions at once: which pool still has room, which pool is least over-provisioned, which pool already carries the most volumes, and which pool has the right media behind it. An administrator can answer those from the Storage Pools grid. A provisioning script, a self-service portal or a cloud framework generally cannot, because it has no way to inspect the grid before it asks for storage.

A tier moves that decision into QuantaStor. You group the pools that are interchangeable for a given class of work -- all the all-flash pools, say, or all the low-cost archive pools -- give the group a name, and let callers ask for the name. QuantaStor selects the pool.

A tier carries a small set of descriptive attributes: a Classification string, a Storage Type, and a Performance Rating. These are labels for the benefit of whoever is choosing a tier. They do not influence which pool QuantaStor selects inside the tier -- the selection uses capacity and provisioning figures only. Treat them as the answer to "which tier should I provision from", never as a filter applied within a tier.

Naming carries more weight than the attributes do, because the tier name is what an automated caller passes. There is no location attribute on a tier, so encode anything a caller needs to discriminate on into the name: east-campus-gold and west-campus-archive let a workload provision from storage local to it, while gold and archive alone do not.

Tiers and Storage Pools

A tier holds one or more pools, and the rules are looser than they first appear:

• A tier can span storage systems. The pool list in the Create and Add/Remove dialogs shows every pool in the grid with the system that owns it, and a volume provisioned from a tier is created on whichever system owns the pool that was selected.
• A pool can belong to several tiers at once. Nothing prevents it and no warning is raised. A pool that is a member of both a "gold" and a "general" tier is a legitimate configuration.
• A tier is a grid-wide object. Create, modify, delete and membership changes are all executed on the grid master, and the resulting tier is visible from every member system.
• Deleting a tier never deletes its pools. The tier is only a grouping.
• Destroying a member pool removes it from the tier. The membership record goes with the pool; the tier is left with one fewer member and needs no repair.
• A tier can be left with no pools at all. The web interface refuses to save an empty selection, but qs storage-tier-pool-remove will happily remove the last pool. Provisioning from an empty tier then fails with Specified storage tier '<name>' has no storage pools associated with it.

A member pool that is offline is still a candidate. Selection reads the size and free-space figures recorded in the grid configuration database and does not consult the pool's State, so an offline pool keeps its last known capacity and can still be picked. The provisioning request then fails against that pool -- Storage Pool '<name>' is currently offline, cannot create Network Shares at this time on this pool for a share, or Specified storage pool '<name>' is offline, volume creation of '<volume>' failed for a volume. Remove a pool from its tiers while it is out of service, or expect provisioning from those tiers to fail intermittently depending on which pool the algorithm lands on.

For the same reason a pool that was created moments ago may not be selected until its capacity has been recorded in the configuration database. This resolves itself; if a brand-new pool is being skipped, wait and retry rather than changing the tier.

How QuantaStor picks a pool

This is the part worth understanding before you build a tier, because the behaviour is not "the emptiest pool wins".

For a request of a given size, QuantaStor walks the tier's member pools and applies the following, in order:

1. Discard members that are no longer present. A membership record whose pool has gone from the configuration database is skipped with a warning in the service log.
2. Shortlist the pools with room to spare. A member reaches the shortlist only if its recorded size is non-zero, its free space is more than half its size, and its free space is more than twice the requested size. Both thresholds have to hold. A pool that is 60% full never reaches the shortlist, and neither does a pool whose free space is merely a little larger than the request.
3. From the shortlist, take the least over-provisioned pool. The comparison is on % Provisioned -- the column shown in the Storage Pools grid -- not on free space.
4. Break a near-tie on volume count. If a pool is less over-provisioned than the current best but by less than 20 percentage points, the pool holding fewer Storage Volumes wins instead. A gap of 20 points or more is decisive on its own and the volume count is not consulted.
5. If the shortlist is empty, fall back to the pool with the most free space. This fallback ignores both thresholds, so a request that no pool comfortably fits is still placed rather than refused.

Two consequences are easy to trip over. Where two shortlisted pools have exactly the same % Provisioned -- two freshly built pools, most obviously -- neither displaces the other, so the winner is simply whichever the configuration database returned first. That order is not stable and differs between grid members, so do not build a scheme that depends on it. And because step 3 compares percentages rather than bytes, a small pool and a large pool that are equally provisioned are equally attractive, and the smaller pool may well be chosen.

A worked example

A tier holds Pool A with 9.2 GiB free and Pool B with 18.4 GiB free, both newly built and both 0% provisioned. Creating four 1 GiB thin volumes from the tier one after another places them A, B, A, B:

• The first request finds both pools shortlisted and tied at 0%, so the first-returned pool -- A -- takes it.
• The second finds A at 11% and B at 0%. The gap is under 20 points, so volume count decides: B holds none and takes it.
• The third finds A at 11% with one volume and B at 5% with one volume. B is less provisioned, the gap is under 20 points, and the volume counts are equal, so nothing displaces A and A takes it.
• The fourth finds A at 21% with two volumes and B at 5% with one. The gap is still under 20 points, B holds fewer volumes, and B takes it.

Adding a third empty pool to the same tier then takes the next several requests outright, because at 0% it sits more than 20 points below the others and step 4 never runs. The net effect over many requests is a rough balance rather than a strict rotation, and it is driven by provisioning ratio rather than by capacity.

Capacity, Max Provisionable and over-provisioning

Create Storage Volume from Storage Tier. The Storage Tier Information panel reports Free Space as the sum of the member pools and Max Provisionable as the largest single member.

The Storage Tier Information panel in the volume dialog reports two capacity figures and they mean different things:

• Free Space is the sum of the free space of every member pool.
• Max Provisionable is the free space of the largest single member pool.

Max Provisionable is the figure that matters, because a volume or share is created inside one pool and is never spread across several. The size slider in the dialog spans the tier's Max Provisionable rather than its Free Space for exactly this reason.

% Utilized on the same panel is derived from space actually consumed, not from space promised. A tier whose pools are heavily thin-provisioned can read 0% Utilized while its member pools show several hundred percent provisioned.

Over-provisioning through a tier works, with one boundary worth knowing:

• A thin request larger than any member pool's free space still succeeds. It is placed by the step 5 fallback into the member with the most free space, and that pool's % Provisioned climbs past 100%. A 25 GiB thin volume provisioned into a tier whose largest member had 18.4 GiB free left that pool at 211% provisioned.
• A request that reserves space fails once the reservation exceeds the selected pool's free space, with Insufficient free space in storage pool '<name>' for this operation, operation cancelled. The error names the pool the tier selected, not the tier.

So a tier does not lift the single-pool ceiling; it just does not enforce it for thin allocations. If you need a volume larger than any one member pool, grow a pool or add a larger one to the tier.

Creating a tier

Navigation: Storage Management → Storage Pools → Storage Pool or Storage Tier (select + right-click) → Create Storage Tier...

Every tier operation is on the right-click menu. There are no Storage Tier buttons on the toolbar.

All tier operations are right-click only. The Storage Pools toolbar has no Storage Tier group, so the only way to reach these dialogs is to select a pool or an existing tier in the tree and right-click it. Right-clicking a pool offers Create Storage Tier...; right-clicking a tier offers the full set.

Create Storage Tier. The pool grid lists every pool in the grid together with the storage system that owns it.

The Create Storage Tier dialog has these fields:

• Name -- required, and pre-filled with the next free name in the form storage-tier-N. Names may contain letters, digits and the characters - _ . only, and must be unique among tiers.
• Description -- free text, optional.
• Classification -- free text describing the primary use case, defaulting to General Purpose. Anything is accepted; Backup/Archive and High-Performance are the other conventional values. It is a label, not a setting.
• Storage Type -- the media behind the member pools: SCSI (the default), SSD, SATA, SSHD, Multiple for a tier that mixes media, and Other where none of them fit. Again a label.
• Performance Rating -- a relative score from 0 to 100, defaulting to 50, set either by typing in the field or by dragging the slider beside it. It exists so a caller can rank tiers against each other; QuantaStor itself does not read it.
• Select Pools for the Tier -- the pool grid, with a checkbox per row and a select-all checkbox in the header. It lists every pool in the grid with its owning Storage System, Pool Type and state, so a tier that spans systems is built here. At least one pool must be ticked or the dialog reports No Pools Selected.

The equivalent command takes the pool list as a comma-delimited argument:

qs storage-tier-create --name=gold-tier --pool-list=pool1,pool2,pool3 \
    --desc="All-flash pools" --storage-class="High-Performance" \
    --storage-type=SSD --perf-level=90

See qs storage-tier-create for the full argument list. Note that --storage-type and --perf-level are optional and that omitting --storage-type stores Unknown, which the web interface renders as Other.

Modifying a tier

Navigation: Storage Management → Storage Pools → Storage Tier (select + right-click) → Modify Storage Tier...

Modify Storage Tier changes the name and the descriptive attributes. Membership is changed in a separate dialog.

Modify Storage Tier edits the Name, Description, Classification, Storage Type and Performance Rating of an existing tier. The Storage Tier combo at the top selects which tier to edit and re-reads the fields when you change it; if you opened the dialog by right-clicking a tier, that tier is already selected.

Membership is not editable here -- use Add/Remove Storage Pools for that. Renaming a tier to a name another tier already holds is rejected.

From the command line each property is an optional argument, and anything you leave out keeps its current value:

qs storage-tier-modify --tier=gold-tier --name=gold --perf-level=95

See qs storage-tier-modify.

Adding and removing pools

Navigation: Storage Management → Storage Pools → Storage Tier (select + right-click) → Add/Remove Storage Pools...

Add/Remove Pools to/from Storage Tier. Current members are already ticked, and OK sets membership to whatever is ticked when you press it.

Add/Remove Pools to/from Storage Tier shows the same grid of every pool in the grid, with the tier's current members already ticked. It replaces the membership with the selection rather than adding to it, so tick a pool to add it and untick a member to remove it, then press OK. At least one pool must remain ticked.

The CLI splits the same operation into two commands, both of which take a list and both of which are additive or subtractive rather than absolute:

qs storage-tier-pool-add    --tier=gold-tier --pool-list=pool4,pool5
qs storage-tier-pool-remove --tier=gold-tier --pool-list=pool1

Adding a pool that is already a member, or removing one that is not, is skipped with a log warning rather than treated as an error. See qs storage-tier-pool-add and qs storage-tier-pool-remove.

Deleting a tier

Navigation: Storage Management → Storage Pools → Storage Tier (select + right-click) → Delete Storage Tier...

Delete Storage Tier. The member pools, and everything provisioned from them, are untouched.

Delete Storage Tier removes the grouping only. Select the tier in the combo -- the Storage Tier Information panel shows its name and description so you can confirm you have the right one -- press OK, and confirm the prompt. The member pools, their Storage Volumes and their Network Shares are all unaffected, including volumes and shares that were originally provisioned through the tier: once created, they belong to a pool and have no further relationship with the tier.

qs storage-tier-delete --tier=gold-tier

See qs storage-tier-delete. There is no dependency check, so a tier can be deleted at any time.

Provisioning from a tier

From the web interface

Navigation: Storage Management → Storage Pools → Storage Tier (select + right-click) → Create Storage Volume...

Create Storage Volume from Storage Tier is the volume create dialog with the Storage Pool picker replaced by a Storage Tier picker and the pool information panel replaced by Storage Tier Information. Everything else -- the General Settings, Security Settings and Advanced Settings tabs, the I/O Profile, the batch count, the block size -- behaves as it does when creating from a pool, and is covered on Storage Volumes. The chosen I/O Profile is applied to the new volume whichever pool the tier selects.

The web interface provisions volumes from a tier but not shares. The Create Network Share dialog has no Storage Tier picker; it is reached from a pool and writes to that pool. To place a share through a tier, use the CLI or the REST API, both of which accept one.

From the CLI

Where a command's --pool argument identifies somewhere to allocate from -- volume-create, share-create and volume-clone -- you may pass the name or UUID of a tier instead of a pool, and QuantaStor selects the pool:

qs volume-create --name=vol1 --size=100G --pool=gold-tier
qs share-create  --name=share1 --pool=gold-tier
qs volume-clone  --volume=vol1 --name=vol1-clone --pool=gold-tier

See qs volume-create, qs share-create and qs volume-clone. The --pool help text mentions only pools; a tier is accepted all the same.

This does not extend to every command that names a pool. Operations on a pool as an object -- qs pool-get, qs pool-modify, qs pool-start, qs pool-destroy and the rest -- require a real pool and reject a tier. The substitution applies only where the argument identifies somewhere to allocate from.

Prefer the UUID over the name in scripts. A tier name and a pool name occupy the same namespace at the point of substitution, and a provisioning quota is resolved before either, so a name collision between the three resolves in a way you would rather not depend on. qs storage-tier-list prints the names with their UUIDs, and qs storage-tier-get prints one tier in full, including its member pool IDs.

Automation and the REST API

Tiers exist mainly so that something other than a person can provision sensibly, and the REST API is where that happens.

The tier itself has a full set of operations of its own, parallel to the pool operations: storageTierCreate, storageTierModify, storageTierDelete, storageTierEnum, storageTierGet and storageTierPoolAddRemove. storageTierGet and storageTierEnum both return the tier's storageTierAssocList, so one call is enough to learn a tier's members.

Two API operations have no CLI equivalent:

• storageTierAssocEnum takes an optional storagePool and answers the reverse question -- which tiers does this pool belong to. Called with an empty pool it returns every tier/pool pairing in the grid.
• storageTierAssocGet tests one specific tier/pool pairing.

To provision, pass the tier's UUID as the provisionableId of storageVolumeCreate, storageVolumeCreateEx or networkShareCreate exactly where a pool UUID would go. The field is named provisionableId rather than storagePoolId precisely because it accepts either.

One difference from pools is worth planning around: a tier reports no capacity of its own. storageTierGet returns the attributes and the member list, but no size or free-space field -- the aggregate figures shown in the web interface are computed client-side by summing the member pools. A framework that wants to publish tier capacity has to read the member pools and add them up itself, and must decide for itself whether to report the sum or the largest member. For placement decisions the largest member is the meaningful number, as it is for Max Provisionable.

OpenStack Cinder

The QuantaStor Cinder volume driver shipped at /opt/osnexus/quantastor/cinder/QuantaStor.py is the worked example of this pattern. Its qs_pool_id setting takes the ID of a QuantaStor pool or tier, and the driver passes whatever it is given straight through as the provisionableId of the volume create and clone calls. Point one Cinder backend at a tier UUID rather than a pool UUID and every volume Cinder creates is placed by QuantaStor; define a Cinder volume type per tier -- gold, silver, archive -- and Cinder's own scheduling picks the tier while QuantaStor picks the pool.

Any other framework that drives the REST API works the same way, and the only integration work is storing a tier UUID where a pool UUID would otherwise go.

Performance and tuning considerations

Because the selection is driven by capacity and provisioning ratio and never by the tier's own Storage Type or Performance Rating, a tier should only contain pools that are genuinely interchangeable for the workload. Mix an all-flash pool and a RAIDZ2 archive pool in one tier and a volume will land on whichever of them the capacity rules favour at that moment, which is very often the archive pool, since it is usually the emptier of the two. Build one tier per performance class instead and let the caller choose the tier. The Multiple Storage Type exists for tiers that deliberately mix media, but selecting it changes nothing beyond the label.

The same caution applies across storage systems. Pool layout, the write log and read cache devices, and the system-level tuning applied through Storage System Optimization and described on Performance Tuning are all properties of a pool and of the system that owns it, not of the tier. A tier that spans two systems running different tuning profiles will deliver noticeably different results depending on where a volume lands, and nothing in the tier records or corrects for that. If tiers span systems, keep the tuning profiles of those systems aligned.

Two smaller points:

• Because step 2 of the selection shortlists only pools that are less than half full, a tier whose members are all past 50% falls through to the free-space fallback and behaves as a plain "emptiest pool wins" chooser. That is a reasonable signal that the tier needs more capacity.
• The 20-point rule means a newly added empty pool absorbs a run of consecutive requests rather than sharing them out. Expect a burst onto new capacity, not a gradual ramp.

Related pages

• Storage Pools -- creating and managing the pools a tier groups
• Storage Volumes -- the volume create dialogs, including Create Storage Volume from Storage Tier
• Network Shares -- shares, which can be placed through a tier from the CLI and API
• Backup Policies -- the Auto-tier Files backup mode, which is a different feature despite the name
• Storage System Optimization -- system tuning profiles, which apply per storage system rather than per tier
• Performance Tuning -- where pool layout and system tuning actually come from
• QuantaStor CLI Command Reference -- the storage-tier-* commands
• QuantaStor REST API Reference Guide -- the storageTier* operations

---

Verified against QuantaStor 6.9.0.