Proxmox Storage Plugin

From OSNEXUS Online Documentation Site
Jump to navigation Jump to search

Proxmox Storage Plugins

A Proxmox Storage Plugin is a Perl module that integrates an external storage system into Proxmox VE so it can be managed natively through the Proxmox UI and CLI.

  • Storage plugins allow Proxmox to:
  • Create, delete, and resize VM disks
  • Report storage capacity and usage
  • Manage snapshots (where supported)
  • Coordinate locks and concurrency across a cluster

From Proxmox’s perspective, the plugin is the storage backend.

QuantaStor PVE Plugin

Github Repo: here

Overview

What the plugin does

The QuantaStor Proxmox plugin uses ZFS over ISCSi protocol to provide block storage for PVE using QuantaStor APIs.

Supported Proxmox versions

The plug-in supports PVE versions 8.4 and 9.1. Installation script allows a “patch” mode and a “full copy” mode. After an upgrade to a PVE system, users will need to reinstall the plugin. Major Minor version updates may break installation and will require developers to update the repository to generate the latest patch files.

Supported backends

The current version of the plugin extends the functionality of the ZFS over iSCSI storage configuration by adding another option for ISCSi provider. QuantaStor RestAPI calls are then used to manage block storage provisioning.

Current maturity

Plugin is currently in pre-release phase

Architecture

How Proxmox interacts with the plugin

After installing the plugin, users can create ZFS over ISCSi storage configurations using QuantaStor as the ISCSi provider. Proxmox will then use the plugin’s QuantaStor API functionality to facilitate ISCSi connection to Proxmox node or cluster. QuantaStor creates and wires storage objects, hands Proxmox a block device and then steps out of the way.

  • Proxmox VE Layer (Orange) - User interface and storage API
  • Plugin Layer (Blue) - QuantaStor plugin
    • ZFSPlugin.pm (modified to support QuantaStor)
    • QuantaStorPlugin.pm (LUN command handler)
  • QuantaStor Layer (Green) - Storage management system
  • Storage Layer (Yellow) - Actual ZFS pools and block devices
  • VM Layer (Purple) - Virtual machines using the storage

External dependencies

After installation of the plugin on all nodes, there should be no other external dependencies required. You will need a QuantaStor storage server with a ZFS pool. See QuantaStor documentation for installation and storage pool setup.

Requirements

Network requirements

A stable network connection is needed between Proxmox and QuantaStor nodes to ensure API call reliability and statistics gathering.

Permissions / API credentials

Recommended to create a proxmox user/role in QuantaStor so you don’t have to use the admin credentials which are saved on the Proxmox systems. The user needs permissions that will allow it to provision hosts initiators, storage volumes, snapshots, and clones. Additionally at minimum view permissions for storage pools.

Copy over system monitor permission for viewing and then apply 'System' permissions for 'storagePool' operations.

Apply 'System' permissions for 'storageVolume' operations.

Apply 'System' permissions for 'host' operations.

Create a new user using the plugin role.

Installation

QuantaStor PVE Plugin README
Installation instructions can be found in the README on the github repository. Plugin needs to be installed on all nodes in the cluster or storage configurations will break. Support for 2 installation modes: patch mode and full copy mode. Using patch mode is a more gentle approach that will add QuantaStor plugin patches to the existing PVE source files and requires the ‘patch’ cli dependency. Full copy mode will fully overwrite the PVE source files (for developement).

Upgrade

To upgrade the QuantaStor Proxmox VE storage plugin, re-run the installation script on each Proxmox node. This applies to both patch-level updates and full upgrades.

During a Proxmox VE upgrade, existing QuantaStor storage configurations may temporarily lose management functionality until the installation script is executed again. Reinstalling the plugin restores full integration.

Currently supported Proxmox VE versions:

  • 8.4.x
  • 9.1.x

Uninstall / Rollback

The installation script supports a rollback mechanism using the --rollback option. When executed, the script restores previously backed-up Proxmox source files, effectively removing the plugin and returning the system to its pre-installation state.

A file-based backup of modified Proxmox source files is automatically created during each installation or upgrade to ensure safe rollback between releases.

Configuration

Storage definitions in Proxmox

  • ID – Storage configuration identifier.
  • Portal – IPv4 iSCSI portal address.
  • Pool – QuantaStor ZFS pool used to provision storage volumes.
  • Block Size – Block size (in KB) used when creating ZFS volumes.
  • Target – Required to be non-empty; currently not used by the plugin.
  • QS Username – QuantaStor API user name.
  • iSCSI Provider – Must be set to QuantaStor API to enable the QuantaStor storage plugin.
  • Thin provision –
    • sparse=1: Thin-provisioned volume with 0% space reserved.
    • sparse=0: Thick-provisioned volume with 100% space reserved.
  • QS Host – QuantaStor API endpoint. When SSL is enabled, the configured CA root certificate must include this value (IP address or hostname) in its SAN list.
  • QS Password – QuantaStor API password (confirmation required).

Enable SSL Verification

To enable SSL verification for QuantaStor API communication, the QuantaStor Root CA certificate must be installed on each Proxmox node in the cluster.

Copy the QuantaStor Root CA certificate to the following location: /etc/ssl/certs/qs-ca-certificates.crt

After copying the certificate, update the system certificate store to ensure the CA is trusted by the operating system.

Proper SSL certificate configuration is required for successful API communication. The value specified in QS Host must match a Subject Alternative Name (SAN) entry in the QuantaStor API server certificate. If the API endpoint (IPv4 address or hostname) is not present in the SAN list, SSL verification will fail and API calls will be rejected.

Example storage.cfg entry

zfs: qs-shared-storage
        blocksize 4k
        iscsiprovider quantastor
        pool qs-52fa39c2-64b0-1830-5e80-050f469bb21d
        portal 10.0.26.31
        target qs-iscsi
        content images
        nowritecache 1
        qs_apiv4_host 10.0.26.31
        qs_password password
        qs_user admin
        sparse 0
        zfs-base-path /dev/zvol

Common misconfigurations

  • Invalid or non-existent QuantaStor storage pool (e.g. qs-<uuid>).
  • iSCSI portal or QuantaStor API host unreachable due to network or firewall issues.
  • Incorrect QuantaStor API credentials.
  • Empty Target field or target value containing whitespace.
  • SSL verification failures caused by missing or incorrect SAN entries for the configured QS Host.

Validation commands

You can use the pvesm storage CLI to see the status of your storage configuration 

 pvesm status
 Name                Type     Status     Total (KiB)      Used (KiB) Available (KiB)        %
 local                dir     active        14173644         5316720         8115140   37.51%
 local-lvm        lvmthin     active        12406784               0        12406784    0.00%
 qs-storage           zfs     active       236716032         1074352       235641680    0.45%
 qs-storage-2         zfs     active       236716032         1074352       235641680    0.45%

From QuantaStor you can verify that a host has been created for each node in your PVE cluster. 

qs host-list

Name                 IP Address       Host OS Type   
-----------------------------------------------------
pve-0-proxmox-host                    0              
pve-1-proxmox-host                    0

Usage

Creating VM disks

VM disks can be created during VM creation or added to an existing VM. When using QuantaStor-backed storage, disk provisioning behavior (thin vs thick) is controlled by the storage configuration.

VM to Template

Virtual machines stored on QuantaStor-backed storage can be converted to templates using standard Proxmox workflows. Template creation does not immediately consume additional storage capacity.

Cloning and Snapshots

Cloning and snapshot operations are supported for VM disks provisioned on QuantaStor storage. These operations leverage ZFS snapshot and clone functionality on the QuantaStor backend.

Moving and Resizing VM disks

Disk resize operations are supported and will update the backing ZFS volume size on the QuantaStor system. Disk move operations between QuantaStor-backed storage and other storage types are supported, subject to available capacity and network performance.

Reassign Hard Disk Owner

Disk ownership reassignment is supported for QuantaStor-backed volumes and may be required after cloning, restoring, or manual disk operations.

Creating Containers (CTs)

Support for LXC containers on QuantaStor-backed storage has **not been tested**. Users attempting to provision containers should proceed with caution and validate behavior before using in production.

VM Migration

Live and offline VM migration is supported when the destination node has access to the same QuantaStor-backed storage configuration.

Limitations and Unsupported Operations

Certain operations may be restricted or behave differently due to the ZFS over iSCSI architecture. Refer to this section before performing large-scale migrations or automation.

Troubleshooting

Logging locations

By default, logging is disabled. To enable debug logging:

  • Open the QuantaStor plugin Perl module:
 /usr/share/perl5/PVE/Storage/LunCmd/QuantaStorPlugin.pm
  • Set the debug variable to 1:
 our $QS_DEBUG = 1;
  • Save the file and restart the Proxmox service (see Installation instructions).

Once enabled, logs will be written to: /var/log/pve-quantastor-plugin.log on the node where the plugin is installed.

Logging provides detailed information about API calls, storage operations, and potential errors, which is useful for troubleshooting configuration or connectivity issues.

Common errors

  • Permission denied - invalid PVE ticket (401)
 *Cause:* Plugin commands require a valid Proxmox session.  
 *Fix:* Ensure the plugin is installed on **all nodes** in the cluster. If using snapshots or reverting VMs, make sure sessions are refreshed. Restart the node if necessary.
  • Unable to verify host / SSL errors
 *Cause:* SSL verification fails if the QuantaStor API endpoint is not present in the root CA certificate’s Subject Alternative Name (SAN) list.  
 *Fix:* Ensure the QuantaStor Root CA is installed on all nodes:  
   /etc/ssl/certs/qs-ca-certificates.crt  
   Update the system certificate store. The QS Host must match a SAN entry (IPv4 or hostname).
  • Storage locked / command timed out
 *Cause:* Proxmox sometimes locks storage during operations such as VM creation or migration.  
 *Fix:* Do **not manually delete lock files**. Wait for operations to finish or check for hung processes. Restart the node if locks persist. Ensure ZFS or iSCSI targets are healthy and reachable.
  • Portal or storage pool unreachable
 *Cause:* Network issues or incorrect iSCSI portal/IP configuration.  
 *Fix:* Verify network connectivity between Proxmox nodes and QuantaStor. Ensure the pool exists and is online in QuantaStor.
  • Invalid QuantaStor credentials
 *Cause:* Username/password mismatch or insufficient API permissions.  
 *Fix:* Confirm the API user exists in QuantaStor with permissions to manage hosts, volumes, snapshots, and clones. Avoid using admin credentials directly if possible.
  • Low reported usage / thin-provisioned volumes
 *Cause:* Thin-provisioned volumes only consume physical storage as data is written.  
 *Fix:* This is expected behavior; usage will increase as VMs write data. Thick-provisioned volumes show reserved storage immediately.

Additional tips

  • Restart Proxmox services after installing or updating the plugin.
  • Verify that ZFS pools and iSCSI targets are healthy in QuantaStor before creating VMs.
  • Use the debug log for tracking API requests and responses when troubleshooting errors.
  • For SSL issues, test connectivity using:
 openssl s_client -connect <QS Host>:8153 -showcerts  
 to confirm the certificate chain is trusted.

Roadmap

Planned features

  • Standalone QuantaStor plugin – Allow the plugin to operate independently without patching core PVE files.
  • Support for Proxmox VE Replication – Enable replication of VMs and storage volumes using QuantaStor storage backend.
  • QuantaStor API tokens – Move from password-based authentication to secure API token usage.
  • CHAP support – Implement CHAP authentication for iSCSI targets to improve security.

Known gaps

  • Proxmox HA and Replication – High Availability and cluster replication with QuantaStor-managed volumes are not yet fully supported. Use caution when deploying mission-critical workloads in a HA cluster. QuantaStor storage plugin does not support Proxmox native HA and replication, however QuantaStor has its own replication and HA systems for ZFS storage pools.

Support & Contributing

Reporting Issues

The QuantaStor Proxmox Storage Plugin is an actively developed project and requires ongoing maintenance to remain compatible with new Proxmox VE releases.

Issues, bugs, and feature requests can be reported through the following public channels:

  • GitHub Issues – The preferred method for reporting bugs, requesting enhancements, and tracking development work.

Please submit issues through the project’s GitHub repository and include relevant details such as Proxmox version, plugin version, error messages, and logs when available.

  • OSNEXUS Engineering Email – Public support and feedback are also accepted via email at: eng-proxmox@osnexus.com

To help us diagnose issues efficiently, please include clear reproduction steps, environment details, and any relevant log output (if plugin logging is enabled).

Retrieved from "https://wiki.osnexus.com/index.php?title=Proxmox_Storage_Plugin&oldid=26965"