Proxmox Storage Plugin: Difference between revisions
m Plugin 1.1.0: iSCSI multipath, API token auth, err=493 retry, validated HA fencing, upgrade/downgrade notes, QuantaStor 6.9 recommended |
m Plugin 1.1.1: install with apt (multipath-tools dependency), no manual multipath config, least-privilege role adds PendingOperationRequest:create, separate networks / ARP note for multipath |
||
| Line 107: | Line 107: | ||
* '''Storage Volume ACL''' -- add, remove | * '''Storage Volume ACL''' -- add, remove | ||
* '''Host''' -- add, addInitiator, modify, remove, assignVolume, unassignVolume | * '''Host''' -- add, addInitiator, modify, remove, assignVolume, unassignVolume | ||
* '''Pending Operation Request''' -- create (deletes are submitted as a pending operation, so a role without it cannot delete volumes or snapshots) | |||
The role can be built in the web interface, or in one command with <code>[[QuantaStor CLI Command Reference#role-add|qs role-add]]</code> followed by <code>[[QuantaStor CLI Command Reference#user-add|qs user-add]]</code>: | The role can be built in the web interface, or in one command with <code>[[QuantaStor CLI Command Reference#role-add|qs role-add]]</code> followed by <code>[[QuantaStor CLI Command Reference#user-add|qs user-add]]</code>: | ||
| Line 118: | Line 119: | ||
StorageVolumeAcl:add:system,StorageVolumeAcl:remove:system,\ | StorageVolumeAcl:add:system,StorageVolumeAcl:remove:system,\ | ||
Host:add:system,Host:addInitiator:system,Host:modify:system,Host:remove:system,\ | Host:add:system,Host:addInitiator:system,Host:modify:system,Host:remove:system,\ | ||
Host:assignVolume:system,Host:unassignVolume:system" | Host:assignVolume:system,Host:unassignVolume:system,\ | ||
PendingOperationRequest:create:system" | |||
qs user-add --name=pve-plugin --password=<password> --role=pve-plugin-role | qs user-add --name=pve-plugin --password=<password> --role=pve-plugin-role | ||
| Line 130: | Line 132: | ||
<pre style="font-size: smaller"> | <pre style="font-size: smaller"> | ||
apt install ./pve-storage-quantastor_1.1.1-1_all.deb | |||
</pre> | </pre> | ||
Use {{Code|1=apt}} rather than {{Code|1=dpkg -i}}: it also installs the packages the plugin depends on ({{Code|1=open-iscsi}}, {{Code|1=multipath-tools}}). {{Code|1=dpkg -i}} does not, and on a node that lacks one it stops with "dependency problems - leaving unconfigured"; run {{Code|1=apt -f install}} to finish. | |||
The package automatically enables the iSCSI initiator service ({{Code|1=iscsid}}), registers the storage type with the Proxmox services, and adds the QuantaStor panel to the web interface. Refresh the browser after installing and the '''QuantaStor''' entry appears in the Add Storage menu. | The package automatically enables the iSCSI initiator service ({{Code|1=iscsid}}), registers the storage type with the Proxmox services, and adds the QuantaStor panel to the web interface. Refresh the browser after installing and the '''QuantaStor''' entry appears in the Add Storage menu. | ||
| Line 139: | Line 143: | ||
=== Upgrades === | === Upgrades === | ||
Install the new package version with {{Code|1= | Install the new package version with {{Code|1=apt install ./<package>.deb}} on each node. From 1.0.0 or later, install the current package directly; there are no intermediate versions to step through and no migration steps. Existing storage definitions, running guests, and iSCSI sessions are unaffected; the Proxmox services are restarted automatically by the package. | ||
'''Downgrading needs one step first if multipath is on.''' Older releases do not know the {{Code|1=multipath}} option. Before installing a 1.0.x package over 1.1.0 or later, stop the guests on every storage that uses multipath and turn it off: | '''Downgrading needs one step first if multipath is on.''' Older releases do not know the {{Code|1=multipath}} option. Before installing a 1.0.x package over 1.1.0 or later, stop the guests on every storage that uses multipath and turn it off: | ||
| Line 303: | Line 307: | ||
With it on, the plugin logs each volume in on every portal QuantaStor advertises for it, builds a device-mapper multipath map for that one volume, and gives guests the map ({{Code|1=/dev/disk/by-id/dm-uuid-mpath-<wwid>}}) instead of a single-path device. Deactivating, deleting, rolling back, templating, and cloning a volume tear down or bring up every path, and an online resize rescans every path and grows the map. | With it on, the plugin logs each volume in on every portal QuantaStor advertises for it, builds a device-mapper multipath map for that one volume, and gives guests the map ({{Code|1=/dev/disk/by-id/dm-uuid-mpath-<wwid>}}) instead of a single-path device. Deactivating, deleting, rolling back, templating, and cloning a volume tear down or bring up every path, and an online resize rescans every path and grows the map. | ||
Nothing needs setting up on the nodes. The plugin package depends on {{Code|1=multipath-tools}} and installs this device section for QuantaStor volumes as {{Code|1=/etc/multipath/conf.d/quantastor.conf}}: | |||
<pre style="font-size: smaller"> | |||
devices { | devices { | ||
device { | device { | ||
| Line 319: | Line 322: | ||
</pre> | </pre> | ||
Keep the {{Code|1=no_path_retry}} setting if you edit the file (local edits are kept across upgrades). multipathd shortens each iSCSI session's recovery timeout to 5 seconds so that a dead path is dropped quickly; without queueing, a moment when '''every''' path is down -- an HA pool failover, a switch reboot -- fails guest I/O instead of pausing it. With this setting the map holds I/O for up to about two minutes until a path returns. With the Debian defaults, multipathd only claims the volumes the plugin registers for storages with multipath on, never local disks or other storage, so the section has no effect on storages without multipath. A copy of this section added to {{Code|1=/etc/multipath.conf}} for plugin 1.1.0 is now redundant and can be removed. | |||
Turn multipath on with the '''Multipath''' checkbox in the dialog's advanced options, {{Code|1=--multipath 1}} on {{Code|1=pvesm add}}, or on an existing storage: | Turn multipath on with the '''Multipath''' checkbox in the dialog's advanced options, {{Code|1=--multipath 1}} on {{Code|1=pvesm add}}, or on an existing storage: | ||
| Line 335: | Line 338: | ||
Expect an {{Code|1=OSNEXUS,QUANTASTOR}} map with one {{Code|1=active ready}} line per portal. When a path fails, I/O pauses for about 15 seconds while the path is failed out, then continues on the remaining paths; the path is reinstated automatically when it returns. | Expect an {{Code|1=OSNEXUS,QUANTASTOR}} map with one {{Code|1=active ready}} line per portal. When a path fails, I/O pauses for about 15 seconds while the path is failed out, then continues on the remaining paths; the path is reinstated automatically when it returns. | ||
'''Give each path its own network''' -- separate subnets, ideally separate switches or VLANs -- so one failure takes out one path. If a node's iSCSI NICs share one layer-2 segment, Linux can answer ARP for one NIC's address on the other, and losing one NIC can then interrupt both paths: guests keep running, but I/O pauses until the path recovers (measured 43 s with two NICs on one segment). On such nodes set {{Code|1=net.ipv4.conf.all.arp_ignore = 1}} and {{Code|1=net.ipv4.conf.all.arp_announce = 2}} in a file under {{Code|1=/etc/sysctl.d/}} and run {{Code|1=sysctl --system}}; the same failure then keeps one path (measured 15 s pause). | |||
QuantaStor advertises every port that has iSCSI enabled, including ports on networks the Proxmox nodes are not on. The plugin checks each advertised portal with a 3-second TCP connection and skips the ones that do not answer, logging a "not reachable on TCP, skipped" warning, rather than stalling activation for about two minutes per unreachable portal. To keep the warning out of the log, enable iSCSI only on the ports the Proxmox nodes use: <code>[[QuantaStor CLI Command Reference#network-port-modify|qs np-modify]] --port=<port> --iscsi-enable=false</code>. A path that is down when a volume is activated joins the next time the volume is activated. | QuantaStor advertises every port that has iSCSI enabled, including ports on networks the Proxmox nodes are not on. The plugin checks each advertised portal with a 3-second TCP connection and skips the ones that do not answer, logging a "not reachable on TCP, skipped" warning, rather than stalling activation for about two minutes per unreachable portal. To keep the warning out of the log, enable iSCSI only on the ports the Proxmox nodes use: <code>[[QuantaStor CLI Command Reference#network-port-modify|qs np-modify]] --port=<port> --iscsi-enable=false</code>. A path that is down when a volume is activated joins the next time the volume is activated. | ||
| Line 434: | Line 439: | ||
---- | ---- | ||
<small>''Plugin release 1.1. | <small>''Plugin release 1.1.1. Verified against QuantaStor 6.9.0.''</small> | ||
Latest revision as of 20:31, 20 October 2026
The QuantaStor Proxmox VE storage plugin integrates QuantaStor as a native iSCSI block storage backend for Proxmox VE virtual machines and LXC containers. This page covers installing the plugin, configuring a QuantaStor-backed storage in Proxmox, and the operational behavior to expect.
| Section | Purpose |
|---|---|
| Overview | What the plugin is and where to get it |
| Architecture | How Proxmox, the plugin, and the appliance interact |
| Requirements | Supported Proxmox VE versions, QuantaStor requirements, credentials |
| Installation | Installing, upgrading, and removing the plugin package |
| Configuration | Adding the storage, all configuration options, API tokens, HA pools, multipath, SSL |
| Container (LXC) support | Running container root filesystems on QuantaStor volumes |
| Usage | Snapshots, templates, clones, resize, and migration behavior |
| Troubleshooting | The most common errors and their fixes |
| Known limitations | Constraints to know before deploying |
| Reporting issues | Where to report bugs and ask for help |
Overview
The plugin registers a first-class storage type named quantastor in Proxmox VE. It appears in the Proxmox web interface under Datacenter → Storage → Add like any built-in storage type, and it works with the standard pvesm, qm, and pct tooling. It does not patch or replace any Proxmox source files, so Proxmox package upgrades do not break it.
Each Proxmox disk is a dedicated QuantaStor storage volume exported over iSCSI. The plugin provisions volumes through the QuantaStor REST API and logs into their iSCSI targets on the Proxmox node, handing Proxmox a stable local block device. Virtual machine disks and container root filesystems are both supported.
The plugin is open source and distributed as a Debian package:
- GitHub repository: OSNEXUS/pve-quantastor-plugin
- Releases: pve-storage-quantastor .deb packages
The README in the repository carries the full module reference and developer documentation; this page is the operator-facing guide.
Architecture

The plugin separates management traffic from storage traffic:
- Control plane -- all provisioning operations (create, delete, snapshot, clone, resize, access control) are REST calls to the QuantaStor API over HTTPS on port 8153, authenticated with HTTP Basic credentials.
- Data plane -- guest I/O flows over iSCSI (port 3260) between the Proxmox node's initiator and the QuantaStor target. The plugin drives the standard
open-iscsiinitiator withiscsiadm: it discovers the target, logs in, and waits for the device to appear before handing it to Proxmox.
Each storage volume is exported through its own dedicated iSCSI target, always presented at LUN 0. The plugin resolves devices through stable /dev/disk/by-path/ip-<portal>:<port>-iscsi-<target-iqn>-lun-0 paths rather than ephemeral /dev/sdX names; with multipath enabled, it logs in on every portal instead and resolves the volume's device-mapper multipath device. QEMU attaches the raw block device directly for VM disks; for containers, Proxmox formats the same block device with ext4 and mounts it as the container root filesystem.
On first activation the plugin registers each Proxmox node with QuantaStor as an iSCSI host entry, using the node's initiator IQN from /etc/iscsi/initiatorname.iscsi. Before a volume is used on a node, the plugin assigns that host to the volume's access control list; when the volume is released, the assignment is removed. Volumes are therefore only visible to the nodes actively using them.
Volume naming
Volumes on the QuantaStor pool follow the standard Proxmox naming convention, so the mapping between a Proxmox disk and its QuantaStor volume is direct:
| Type | Name pattern | Example |
|---|---|---|
| VM or container disk | vm-<vmid>-disk-<N> |
vm-100-disk-0
|
| Template base disk | base-<vmid>-disk-<N> |
base-100-disk-0
|
| Template snapshot | template-base-<vmid>-disk-<N> |
template-base-100-disk-0
|
| Snapshot | <volume>_<snapshot> |
vm-100-disk-0_snap1
|
Requirements
Supported Proxmox VE versions
| Proxmox VE version | Supported | Notes |
|---|---|---|
| 9.2.x | Yes | Loads with a cosmetic "implementing an older storage API" log message; see Known limitations |
| 9.1.x | Yes | Loads silently |
| 8.4.x and older | No | The plugin does not load; the quantastor type will not appear
|
Nodes running 9.1 and 9.2 can be mixed in one cluster and storage operations work transparently across both. Do not include 8.x nodes -- any storage operation routed to one (for example a migration target) fails because the storage type is not registered there.
QuantaStor requirements
- A QuantaStor system with a storage pool to provision from. QuantaStor 6.9 is the recommended release; builds without its fixes refuse to delete freshly created snapshots for several minutes and can wedge volumes on template conversion (see Troubleshooting, err=493).
- Network reachability from every Proxmox node to the appliance on TCP 8153 (REST API) and TCP 3260 (iSCSI).
- No configuration is required on the QuantaStor side beyond the pool: the plugin creates volumes, host entries, and access control assignments itself.
API credentials
The plugin authenticates to the QuantaStor REST API with a username and password, or with an application token in the same two fields (see Authenticating with an API token). The admin account works, but we recommend creating a dedicated user so the credentials stored on the Proxmox cluster carry only the permissions the plugin needs.

Create a role that grants view access to everything plus the specific write operations the plugin performs, then create a user with that role (see User Add).
The write operations the plugin needs are:
- Storage Volume -- create, delete, modify, resize, createSnapshot, deleteSnapshot, rollback, clone
- Storage Volume ACL -- add, remove
- Host -- add, addInitiator, modify, remove, assignVolume, unassignVolume
- Pending Operation Request -- create (deletes are submitted as a pending operation, so a role without it cannot delete volumes or snapshots)
The role can be built in the web interface, or in one command with qs role-add followed by qs user-add:
qs role-add --name=pve-plugin-role --desc="Role for the Proxmox VE storage plugin" \ --permissions="*:view:system,\ StorageVolume:create:system,StorageVolume:delete:system,StorageVolume:modify:system,\ StorageVolume:resize:system,StorageVolume:createSnapshot:system,StorageVolume:deleteSnapshot:system,\ StorageVolume:rollback:system,StorageVolume:clone:system,\ StorageVolumeAcl:add:system,StorageVolumeAcl:remove:system,\ Host:add:system,Host:addInitiator:system,Host:modify:system,Host:remove:system,\ Host:assignVolume:system,Host:unassignVolume:system,\ PendingOperationRequest:create:system" qs user-add --name=pve-plugin --password=<password> --role=pve-plugin-role
The full storage lifecycle -- storage activation, volume create, iSCSI access control, snapshot create and delete, and volume delete -- works under a user with exactly this permission set.
Installation
Install the plugin package on every node in the cluster. A node without the plugin cannot activate the storage, which breaks migration to that node and leaves the storage inactive there.
apt install ./pve-storage-quantastor_1.1.1-1_all.deb
Use apt rather than dpkg -i: it also installs the packages the plugin depends on (open-iscsi, multipath-tools). dpkg -i does not, and on a node that lacks one it stops with "dependency problems - leaving unconfigured"; run apt -f install to finish.
The package automatically enables the iSCSI initiator service (iscsid), registers the storage type with the Proxmox services, and adds the QuantaStor panel to the web interface. Refresh the browser after installing and the QuantaStor entry appears in the Add Storage menu.
The package installs dpkg triggers that re-apply the web interface integration whenever the pve-manager or pve-storage packages are upgraded, so routine Proxmox updates need no plugin reinstall.
Upgrades
Install the new package version with apt install ./<package>.deb on each node. From 1.0.0 or later, install the current package directly; there are no intermediate versions to step through and no migration steps. Existing storage definitions, running guests, and iSCSI sessions are unaffected; the Proxmox services are restarted automatically by the package.
Downgrading needs one step first if multipath is on. Older releases do not know the multipath option. Before installing a 1.0.x package over 1.1.0 or later, stop the guests on every storage that uses multipath and turn it off:
pvesm set <storage-id> --delete multipath
Uninstalling
apt remove pve-storage-quantastor
Removing the package does not remove storage entries from /etc/pve/storage.cfg. Delete or disable QuantaStor storage entries first if the plugin is being removed permanently; otherwise those entries error once the plugin is gone.
Configuration
Adding the storage in the web interface

| Field | Description |
|---|---|
| ID | Proxmox storage identifier. Auto-fills with the first free qs-storage-N; edit as needed.
|
| API Host | QuantaStor appliance IP address or hostname. For an HA pool, use the failover group's virtual interface -- see QuantaStor HA storage pools. |
| Username | QuantaStor API user. |
| Password | QuantaStor API password, or QSAPPTKN:<token> for an application token. Stored in a root-only file on the cluster filesystem, never in storage.cfg.
|
| Pool | QuantaStor storage pool to provision volumes from. Type a pool name, or click the magnifier button to connect to the appliance and pick from its active pools. |
| Content | Disk image for VM disks, Container for LXC root filesystems. Both are selected by default.
|
| Nodes | Restrict the storage to specific Proxmox nodes; leave empty for all nodes. |
| iSCSI Portal (Advanced) | Portal address for iSCSI logins. Defaults to the API Host when left blank. |
| Multipath (Advanced) | Log in on every portal QuantaStor advertises and use the multipath device. Off by default; see Multipath for the prerequisites. |
| SSL Verify (Advanced) | Verify the appliance's TLS certificate against the node's system CA store. Leave off for self-signed certificates. |
API Host, Username, and Pool are fixed once the storage is created. The other fields can be changed later through the Edit dialog or pvesm set.

Adding the storage from the command line
pvesm add quantastor qs-storage-1 \ --api_host 10.0.0.20 \ --username admin \ --password '<password>' \ --pool_id <pool-name-or-uuid> \ --content images,rootdir
Single-quote the password so the shell does not interpret special characters ($, !, backticks).
The resulting /etc/pve/storage.cfg entry contains no password:
quantastor: qs-storage-1
api_host 10.0.0.20
pool_id <pool-name-or-uuid>
username admin
content images,rootdir
shared 1
ssl_verify 0
All options:
| Option | Required | Default | Description |
|---|---|---|---|
api_host |
yes | -- | QuantaStor appliance IP or hostname (HA pools: the failover group virtual interface) |
username |
yes | -- | QuantaStor API user |
password |
on create | -- | QuantaStor API password; stored outside storage.cfg
|
pool_id |
yes | -- | QuantaStor pool name or UUID |
content |
no | images |
images (VM disks), rootdir (containers), or both
|
portal |
no | api_host |
iSCSI portal address when it differs from the API endpoint |
api_port |
no | 8153 | QuantaStor REST API port |
multipath |
no | 0 | Use every advertised portal through DM-MPIO (see Multipath) |
ssl_verify |
no | 0 | Verify the appliance TLS certificate (see SSL verification) |
sparse |
no | 1 | Thin provisioning; set 0 for thick (see below)
|
shared |
no | 1 | Set automatically on create; marks the storage cluster-shared so migration works |
nodes |
no | all | Restrict to specific Proxmox nodes |
Change settings with pvesm set (or the Edit dialog), not by removing and re-adding the storage. The remove path deletes the stored password file cluster-wide, and a re-add that fails to capture the password leaves every operation failing until the password is set again.
Thin and thick provisioning
Volumes are thin provisioned by default: they consume pool space as data is written, not their full size at creation. Set sparse 0 on the storage for thick provisioning, where each volume reserves 100% of its size in the pool at creation time. Note this default is the opposite of Proxmox's built-in ZFS storage types, where sparse defaults to off.
How the plugin stores credentials
The API password is never written to storage.cfg. It is kept in a root-only file under the cluster filesystem's private area (/etc/pve/priv), following the same convention Proxmox uses for CIFS and Proxmox Backup Server credentials, and it replicates to every node automatically.
If the stored password is missing or empty on a node, every plugin operation on that node fails with an explicit error naming the affected file and the pvesm set command that repairs it.
Authenticating with an API token
Instead of a user's password, the plugin can authenticate with a QuantaStor user application token, so the Proxmox cluster never holds a login that also works in the web interface. No plugin option is involved: enter the token in the same two fields.
- On the appliance, create a token for the account the plugin uses -- ideally the dedicated user from API credentials:
qs user-app-token-add --user=pve-plugin --name=pve-cluster
The result shows an app key (a UUID, used as the username) and a 64-character access token (used as the password). Copy the access token now; it is not shown again. - Add or edit the storage with Username set to the app key and Password set to
QSAPPTKN:followed by the access token. The token name also works in place of the app key.
Everything else is unchanged: the pool scan, pvesm add, and every volume operation run under the token, and the credential is stored exactly like a password. A token inherits the role of its user, so pair it with a least-privilege user rather than admin, and do not mark it read-only -- the plugin creates and deletes volumes. To rotate a token, delete it with qs user-app-token-delete, create a new one, and run pvesm set <storage-id> --password 'QSAPPTKN:<new token>'.
SSL verification
By default (ssl_verify 0) the plugin accepts the appliance's certificate without verification, which suits the self-signed certificate QuantaStor ships with. To enable verification, install the QuantaStor CA certificate into the system trust store on every Proxmox node, then enable the option:
cp qs-ca.crt /usr/local/share/ca-certificates/ update-ca-certificates pvesm set <storage-id> --ssl_verify 1
The value configured as API Host must appear in the certificate's Subject Alternative Name list -- verification checks the hostname as well as the trust chain, so a certificate without the configured IP address or hostname in its SAN entries fails even with the CA trusted.
QuantaStor HA storage pools
If the pool is a QuantaStor HA storage pool -- one owned by a storage pool high-availability group that fails over between two controllers -- configure the storage against the failover group's virtual interface, not against either controller's own address:
quantastor: qs-ha
api_host 10.0.0.20 # the failover group's floating VIF
portal 10.0.0.20 # the same VIF
pool_id my-ha-pool
username admin
content images,rootdir
shared 1
Both api_host and portal must be that virtual interface (see High-availability VIF Management). The address moves with the pool on failover, so the REST endpoint and the iSCSI portal follow the active controller on their own. Pointed at a controller's fixed address instead, the storage stops working the moment the pool moves to the other controller.
Nothing else changes -- the plugin does not need to be told the pool is HA, and no extra options are required. During a controller failover, guests pause rather than fail; see Known limitations for the measured behavior and the one initiator setting that must not be changed.
Multipath
When QuantaStor serves iSCSI on more than one network port, the storage can use every portal at once and keep guest I/O running when one path fails. Multipath is off by default.
With it on, the plugin logs each volume in on every portal QuantaStor advertises for it, builds a device-mapper multipath map for that one volume, and gives guests the map (/dev/disk/by-id/dm-uuid-mpath-<wwid>) instead of a single-path device. Deactivating, deleting, rolling back, templating, and cloning a volume tear down or bring up every path, and an online resize rescans every path and grows the map.
Nothing needs setting up on the nodes. The plugin package depends on multipath-tools and installs this device section for QuantaStor volumes as /etc/multipath/conf.d/quantastor.conf:
devices {
device {
vendor "OSNEXUS"
product "QUANTASTOR"
path_grouping_policy "multibus"
path_checker "tur"
failback "immediate"
no_path_retry 24
}
}
Keep the no_path_retry setting if you edit the file (local edits are kept across upgrades). multipathd shortens each iSCSI session's recovery timeout to 5 seconds so that a dead path is dropped quickly; without queueing, a moment when every path is down -- an HA pool failover, a switch reboot -- fails guest I/O instead of pausing it. With this setting the map holds I/O for up to about two minutes until a path returns. With the Debian defaults, multipathd only claims the volumes the plugin registers for storages with multipath on, never local disks or other storage, so the section has no effect on storages without multipath. A copy of this section added to /etc/multipath.conf for plugin 1.1.0 is now redundant and can be removed.
Turn multipath on with the Multipath checkbox in the dialog's advanced options, --multipath 1 on pvesm add, or on an existing storage:
pvesm set <storage-id> --multipath 1
Change the setting only while no guest on that storage is running. Then start a guest and confirm that its volume has one map with a path per portal, and that QEMU opened the map:
multipath -ll qm showcmd <vmid> | grep -o 'dm-uuid-mpath-[0-9a-f]*'
Expect an OSNEXUS,QUANTASTOR map with one active ready line per portal. When a path fails, I/O pauses for about 15 seconds while the path is failed out, then continues on the remaining paths; the path is reinstated automatically when it returns.
Give each path its own network -- separate subnets, ideally separate switches or VLANs -- so one failure takes out one path. If a node's iSCSI NICs share one layer-2 segment, Linux can answer ARP for one NIC's address on the other, and losing one NIC can then interrupt both paths: guests keep running, but I/O pauses until the path recovers (measured 43 s with two NICs on one segment). On such nodes set net.ipv4.conf.all.arp_ignore = 1 and net.ipv4.conf.all.arp_announce = 2 in a file under /etc/sysctl.d/ and run sysctl --system; the same failure then keeps one path (measured 15 s pause).
QuantaStor advertises every port that has iSCSI enabled, including ports on networks the Proxmox nodes are not on. The plugin checks each advertised portal with a 3-second TCP connection and skips the ones that do not answer, logging a "not reachable on TCP, skipped" warning, rather than stalling activation for about two minutes per unreachable portal. To keep the warning out of the log, enable iSCSI only on the ports the Proxmox nodes use: qs np-modify --port=<port> --iscsi-enable=false. A path that is down when a volume is activated joins the next time the volume is activated.
Verifying the setup
The password file should exist on every cluster node, and the storage should activate:
ls -la /etc/pve/priv/storage/ pvesm status --storage <storage-id>
Expect the <storage-id>.pw file present on each node and the storage reported active with sizes populated. On the QuantaStor side, each Proxmox node appears as a host entry once the storage has been activated there -- verify with qs host-list or in the web interface under Hosts.
Container (LXC) support
LXC containers can place their root filesystem (and additional mount points) on QuantaStor storage. Enable the Container content type on the storage -- tick it in the dialog, or:
pvesm set <storage-id> --content images,rootdir
A container volume is the same raw iSCSI LUN as a VM disk. Proxmox's container stack formats the LUN with ext4, mounts it, and grows the filesystem on resize -- the plugin's job is only to provide the block device, the same model Proxmox uses for containers on Ceph RBD and LVM-thin storage.
The full container lifecycle is supported: create, start, stop, and destroy (privileged and unprivileged), snapshots of running containers (filesystems are frozen first, so snapshots are consistent), snapshot rollback (container stopped), online and offline resize, templates with linked or full clones, restart-mode migration between nodes, extra mount points (one dedicated LUN per mount point), and pct fstrim (freed space returns to the thin volume through iSCSI UNMAP).
Container limitations:
- Container templates (
vztmpl) cannot be stored on this storage -- template tarballs need a file-based storage such aslocal. Create containers from a template onlocalonto QuantaStor storage. This mirrors Ceph RBD, where template storage is likewise not possible on block storage. - Containers cannot live-migrate -- migration is restart mode, a Proxmox-wide constraint for containers on block storage (the same applies to RBD and LVM-thin).
- Proxmox HA fencing with mounted container filesystems is safe under Proxmox's default watchdog fencing. Validated on a two-node cluster with a corosync QDevice by partitioning the node running a container from the cluster while its storage path stayed reachable: the partitioned node lost quorum, its watchdog reset it, and only then did the surviving node attach and mount the volume, with no window in which both nodes wrote to it. The plugin provides no storage-level exclusion, so this rests on Proxmox's watchdog timing -- do not shorten the watchdog or fence-delay defaults.
Usage
VM disks on QuantaStor storage behave like disks on any shared Proxmox block storage: create them when creating a guest or add them later, and the plugin provisions the backing volume on the pool automatically. The operations below have behavior worth knowing about.
Snapshots and rollback
Snapshots are QuantaStor volume snapshots taken through the API. Snapshots of running containers freeze the filesystem first, so they are filesystem-consistent.
Rollback requires the guest to be stopped, and only the most recent snapshot can be rolled back to (Proxmox enforces the stop; the plugin enforces the ordering). A rollback issued right after stopping a guest can take up to a few minutes: the appliance's session tracking releases the volume on a cycle of up to three minutes after the initiator disconnects, and the plugin waits for that release before asking the appliance to roll back. This wait is normal, not a hang.
Templates and clones
Converting a guest to a template renames its volumes to base-* and takes a template snapshot; clones are created from that snapshot on the appliance, so creating a clone does not copy the full volume up front. Destroy clones before destroying their template.
Resizing
Disks can be grown online -- the volume grows on the appliance, the plugin refreshes the kernel's view of the LUN, and the guest sees the new size without a restart. Shrinking is not supported.
Migration
Live and offline VM migration between nodes works over the shared storage: the same volume is activated on the destination and released on the source, with no data copied. Every node involved needs the plugin installed and the storage active -- see Troubleshooting if a migration fails.
Troubleshooting
The README troubleshooting section carries the full decision trees; these are the most common cases.
Authentication check failed ... [err=26] -- QuantaStor rejected the request's credentials. Check, in order:
- The password file exists and is non-empty on the node running the operation:
ls -la /etc/pve/priv/storage/<storage-id>.pw. If missing, repair withpvesm set <storage-id> --password '<password>'from any node. - The credentials are actually valid -- test with curl, which bypasses the plugin:
curl -k "https://<api_host>:8153/qstorapi/storagePoolGet?storagePool=<pool>" -u admin:'<password>' - A password containing shell metacharacters was mangled by an unquoted
pvesm add-- re-set it single-quoted.
Failed to delete storage volume ... [err=493] on destroy or snapshot delete -- the backing volume was busy when the appliance tried to destroy it. Current QuantaStor releases eliminate the known causes; on older appliance builds, a delete of a freshly created snapshot can be refused for a few minutes. The plugin retries a refused snapshot delete for about three minutes, and the task log shows each attempt. If it still fails, the guest is left locked: unlock it (qm unlock <vmid> or pct unlock <ctid>) and retry after a few minutes. A persistently failing delete on a current release warrants a support ticket with /var/log/qs/qs_service.log from around the failure.
Container fails to start with run_buffer: ... Script exited with status 255 -- check that the storage still advertises the Container content type. Proxmox's pre-start hook refuses to start a container whose root storage lacks rootdir, and the real error is swallowed. Restore with pvesm set <storage-id> --content images,rootdir.
Storage shows active on some nodes and inactive on others -- the inactive node's error says which case it is: a "no password configured" message means the password file is missing there (repair as in err=26 above); a network or SSL error means that node cannot reach the API host on port 8153 -- check firewalls and routing, not the plugin.
Migration or qm move-disk to another node fails -- verify on the destination node that the plugin package is installed, pvesm status shows the storage, and the password file exists. If the plugin and storage are healthy, check for QEMU version skew between the nodes (pin the VM's machine type to the older node's version) and confirm the storage has shared 1 set.
Plugin ... is implementing an older storage API, an upgrade is recommended in the logs -- cosmetic; see Known limitations.
Plugin errors surface in the Proxmox task log of the operation that failed, and in the journal of the PVE service that ran it (journalctl -u pvedaemon, journalctl -u pvestatd).
Known limitations
- QuantaStor HA pool failover pauses guest I/O for roughly 30-45 seconds. When an HA pool moves between controllers, guests on that storage block until the pool is imported on the other controller, then continue with no errors and no data loss -- measured guest-visible pauses were 26-34 seconds, independent of how many volumes the pool serves. Guests and their filesystems ride through it without tuning: the initiator holds queued I/O while the session recovers, so the guest never sees an error. Do not lower
node.session.timeo.replacement_timeoutin/etc/iscsi/iscsid.conf: its 120-second default is the budget that makes the pause survivable, and multipath tuning advice that shortens it does not apply here because there is no alternate path. On a storage with multipath on, multipathd does shorten it, and theno_path_retryqueueing carries the pause instead. This is QuantaStor pool failover, separate from Proxmox's own HA -- for Proxmox HA fencing see Container (LXC) support. - Snapshot rollback is limited to the most recent snapshot and requires the guest stopped; rollback can wait up to a few minutes for the appliance to release the volume (see Usage).
- Volumes cannot be shrunk, only grown.
- All volumes present at iSCSI LUN 0 on their dedicated target; the
lunfield the QuantaStor API reports on volume objects is an internal value and does not reflect the wire LUN. - The "older storage API" log message on Proxmox VE 9.2 is cosmetic. The plugin declares storage API version 13 so that one build supports both 9.1 and 9.2; PVE 9.2 logs a recommendation to upgrade at every plugin scan. Behavior is identical on both versions. If the log noise is a problem, filter it with an rsyslog rule:
:msg, contains, "implementing an older storage API" stopin/etc/rsyslog.d/. - Proxmox storage replication is not supported (
pvesr); QuantaStor's own remote replication can protect the pool instead.
Reporting issues
Report bugs and feature requests through GitHub Issues on the plugin repository, or by email to eng-proxmox@osnexus.com. Include the Proxmox VE version, the plugin version, and the failing task's log output.
Related pages
- Storage Pools
- Storage Volumes
- Create Storage Pool High-Availability Group
- High-availability VIF Management
- ISCSI Initiator Setup
- User Add
Plugin release 1.1.1. Verified against QuantaStor 6.9.0.