QuantaStor Shell Utilities

Revision as of 05:50, 3 September 2026 by Qadmin (talk | contribs) (Restore the verified detail that qs client packages are downloadable for Debian/Ubuntu, RedHat/CentOS and Windows (QSTOR-12352))

Every QuantaStor appliance ships a set of command-line utilities under /usr/bin/qs-* alongside the main qs management CLI. They cover the work that is easier from a shell than from the web interface -- log collection, triage, upgrades, kernel and driver management, I/O fencing, Ceph maintenance and low-level pool repair. This page describes each of them, what it is for, and how to invoke it.

Almost all of these commands require root. Log in as qadmin and use sudo, or run them from a root shell.

Area Utilities Purpose
Management CLI qs The full management API as a command-line tool -- everything the web interface can do.
Status and logs qs-version, qs-status, qs-showlog, qs-audit What is running, what version, and what the service has been doing.
Support logs qs-sendlogs, qs-trunclog, qs-download-link Collect a log bundle, keep /var/log from filling, hand a file to support.
Upgrades and packages qs-upgrade, qs-distupgrade, qs-apt-channel, qs-package-check, qs-kernel, qs-kernelupgrade, qs-dkms-install Product upgrades, repository channel selection, kernel and driver installs.
Performance qs-iostat, qs-zfstunings, qs-perftest, qs-ramdisk Measure device and cache behaviour, compare tunables against their defaults, run baseline benchmarks.
General maintenance qs-util The large general-purpose toolbox: networking, ZFS memory, encryption keys, iSCSI/FC, SNMP, hardening.
Pools, shares, encryption qs-zconvert, qs-zvolutil, qs-zpoolscrub, qs-luksutil, qs-fsmon, qs-shareusage Import foreign pools, repair bit-rot, tune encrypted pools, scan share capacity.
High availability qs-crm, qs-iofence, qs-scstdlm, qs-pcsnodealert Corosync/Pacemaker state, SCSI-3 persistent reservations, clustered reservation locking.
Scale-out (Ceph) qs-ceph, qs-ceph-journalgroup, qs-osd-compact, qs-cephfsutil, qs-s3util, qs-rgw-autotier, qs-rgw-monitor, qs-rgw-moveobjects Cluster health, OSD and journal maintenance, CephFS and S3 object tooling.
Certificates qs-sslcert Inspect, generate, install and reset the grid and web certificates.
Statistics qs-statsdb Query and maintain the InfluxDB time-series database behind the dashboards.
Replication keys qs-sshkey Manage the SSH keys remote replication uses.
Enclosures qs-seagate, qs-wd Vendor-specific enclosure log collection and zoning.
Boot repair qs-bootefi-fix Repair an EFI boot partition entry in /etc/fstab.

Getting help from a utility

The utilities do not all take --help. Most of the shell-based ones print their usage block when they are run with no arguments, or when they are given an operation they do not recognise -- so --help produces the usage text followed by an ERROR: Specified operation '--help' is not supported line. That error is harmless; the usage above it is the real output.

Two exceptions are worth knowing:

  • qs-iofence --help prints only the flag list. Run qs-iofence with no arguments to get the full command list.
  • qs-zfstunings has no help at all. Its optional first argument is a configuration file path, so --help is read as a filename and rejected.

The Python-based utilities -- qs-audit, qs-iostat, qs-sendlogs, qs-upgrade, qs-kernelupgrade, qs-dkms-install, qs-perftest -- take -h / --help in the usual way. The compiled utilities -- qs-s3util, qs-cephfsutil, qs-shareusage -- take a help operation.

qs -- the management CLI

qs is the full QuantaStor management interface as a command-line tool. It exposes the same operations as the web interface, and it is the tool to reach for when provisioning, licensing or cluster configuration needs to be scripted -- which is why it is the basis of most MSP and service-provider automation around QuantaStor.

It is installed on every appliance, and it also runs remotely: point it at an appliance with --server, the QS_SERVER environment variable, or a ~/.qs.cnf file holding ipaddress,username,password. Client packages for Debian/Ubuntu, RedHat/CentOS and Windows are on the OSNEXUS downloads page, so the same commands can be run from an administrator's own workstation.

The command set is large -- around two thousand commands and aliases -- so discovery matters more than memorisation:

qs help                      # full help
qs help pool                 # help for every command whose name contains 'pool'
qs help --min                # one line per command: name plus required arguments
qs pool-list --verbose       # every argument of one command, with descriptions
qs imode                     # interactive mode

Output can be rendered as XML, JSON or CSV with --xml, --json or --csv, which is what makes it usable from scripts.

A malformed qs command exits zero, so check the printed error rather than the exit status when scripting against it.

The complete command reference is the QuantaStor CLI Command Reference; the equivalent for PowerShell is the QuantaStor PowerShell Command Reference, and the same operations are available over HTTP through the REST API Reference Guide. See also the CLI Guide Overview.

System status and logs

qs-version

Prints the CLI, service and web manager versions, then lists the version of every member of the storage grid. The grid listing needs root; run as an unprivileged user it prints the local versions and warns that it cannot report on grid members.

# qs-version
## CLI Version Info ##
OSNEXUS QuantaStor CLI 6.9.0

## Service Version Info ##
OSNEXUS QuantaStor Service 6.9.0

## Web Manager Version Info ##
OSNEXUS QuantaStor Web Manager 6.9.0

## Storage Grid Server List ##
...

Note that a few of the utilities on this page print a version of their own in their banner which does not track the product version. qs-version is the authoritative answer.

qs-status

Prints a one-line systemd state for each QuantaStor and protocol service, grouped into four sections -- core service, internal services, web services and protocol services. A service whose unit is not installed prints missing rather than an error, so the report is readable on an appliance that does not run every protocol.

The internal services covered are qs-snmpagentd, qs-zfseventd, qs-jsonrpcd, qs-rest-soap-relay, qs-statsd, qs-scriptd and qs-sso-authd; the protocol services are iscsi-target, nfs-server (with a count of running nfsd threads), nfs-ganesha, smb, nmb and winbind. The web tier is nginx.

# qs-status

[core service]
          quantastor:       active (running) since Thu 2026-09-03 04:53:21 UTC; 39min ago

[internal services]
         qs-jsonrpcd:       active (running) since Thu 2026-09-03 04:53:12 UTC; 40min ago
  qs-rest-soap-relay:       active (running) since Thu 2026-09-03 04:53:12 UTC; 40min ago
...

qs-showlog

A front end to the log files under /var/log/qs/. It saves remembering which file holds what, and it knows which logs are toggled on rather than always written.

qs-showlog is a helper script for printing various QuantaStor service logs

Usage:

  Service Log:
    qs-showlog -a                  : Dump QuantaStor service log to the console (cat /var/log/qs/qs_service.log)
    qs-showlog -f                  : View service log (tail -F /var/log/qs/qs_service.log)
    qs-showlog -e                  : Print just the errors and warnings in the qs_service.log.
  Service Internals:
    qs-showlog -audit [-f]         : View the QuantaStor service audit log (tail /var/log/qs/qs_audit.log)
    qs-showlog -api [--clear]      : View API command log (tail /var/log/qs/qs_apicall.log)
    qs-showlog -boot               : View the QuantaStor service boot log (tail /var/log/qs/qs_boot.log)
    qs-showlog -crypto             : View crypto log traces (tail /var/log/qs/qs_crypto.log)
    qs-showlog -event [<seconds>]  : View grid events being processed and generated by the QuantaStor service.
                                     Each capture writes a new /var/log/qs/qs_event_<timestamp>_<window>.log and stops
                                     itself when the window elapses, defaulting to 600 seconds
                                     (10 minutes). Pass a window in seconds to override it.
    qs-showlog -event --clear      : Stop event logging now, leaving the capture in place.
    qs-showlog -execcmd [--clear]  : View commands being run by the core service (tail /var/log/qs/qs_exec.log)
    qs-showlog -gs [--clear]       : View grid stats (tail /var/log/qs/qs_gridstats.log)
    qs-showlog -running            : View currently running commands (tail /var/log/qs/qs_service.log)
    qs-showlog -shutdown           : View the QuantaStor system shutdown log (tail /var/log/qs/qs_shutdown.log)
    qs-showlog -snmp               : View the QuantaStor SNMP agent log (tail /var/log/qs/qs_snmpagent.log)
    qs-showlog -scst [--clear]     : VIew SCST sysfs commands (tail /var/log/qs/qs_scst.log)
    qs-showlog -vols [--clear]     : View iSCSI/FC volume sessions (tail /var/log/qs/qs_volsession.log)

qs-showlog -e is the usual first move on a reported problem: it filters the service log down to just its errors and warnings, with the source line number of each entry.

Event logging is bounded rather than left on. qs-showlog -event starts a capture that writes its own timestamped file and stops itself after ten minutes; pass a number of seconds to change the window, or --clear to stop the capture early and keep what has been written.

qs-showlog -shutdown reports that the file is missing on a system that has not been shut down since the log was last rotated. That is expected -- the log is written during shutdown.

qs-audit

Reads /var/log/qs/qs_audit.log, the JSON record of every API call the service has handled, and prints it with syntax colouring. It runs without root.

Its most useful capability is translation: -c converts each logged call back into the equivalent qs CLI command, and -u / -j into an equivalent curl call using query parameters or a JSON body. That turns "do it once in the web interface, then read back the command" into a practical way of writing automation, and it is the fastest way to find the argument set a dialog actually sends.

# qs-audit -c -q -n 5
Displaying 5 of 64 entries:
Thu Sep 03 05:36:56 2026 qs_audit.log entry converted to CLI:
qs share-modify --share "sales-share" --name "sales-share" --active "true" --cifs-enable "true" ...

Arguments that matter:

Argument Effect
-n <count> Number of matching entries to show. Default 100, or 5 in follow mode.
-a Show every entry in the log.
-f Follow the log in real time.
--filter <string> Show only entries matching a search term. Comma-separated terms are allowed.
--filter-apis Restrict the filter to the method_api field, expanded through the CLI name mapping. Cannot be combined with -v.
-c / -u / -j Convert entries to a qs CLI command, a query-parameter curl call, or a JSON-body curl call.
-q Suppress the raw JSON and print only the conversion. Only valid with a conversion option.
-v Include entries that carry no method_api field, which are hidden by default.
--file <path> Read a different log file -- for example one unpacked from a support bundle.

The web interface offers the same log through the Audit Log Analyzer.

Support log collection

qs-sendlogs

Collects a support log bundle and uploads it to OSNEXUS support. The web interface uses this same script, so the bundle is the same either way. Requires root.

Scrubbing and uploading are both on by default, which is the opposite of what older documentation said. There is no --scrub argument, because personally identifiable information is removed unless you ask for it to be kept:

Argument Effect
--no-upload Collect the bundle but do not upload it. Use on a system with no outbound internet access.
--no-scrub Keep personally identifiable information in the bundle. Off by default.
--no-trunc Do not truncate large log files.
--proxy <url> Upload through an HTTP proxy, as http://proxyserver:port.
--filepath <path> Send only the single file at this path instead of collecting a bundle.
--no-cleanup Keep the temporary collection directory, so the bundle can be reviewed locally.

For a permanently disconnected site, create the touchfile /var/opt/osnexus/quantastor/touchfiles/qs_logs.no_upload rather than remembering --no-upload on every run -- it disables uploading for good, including for collections started from the web interface.

Collection from the web interface, and what the bundle contains, is covered on Send System Log Report.

qs-trunclog

Truncates oversized files in /var/log when the filesystem is close to full. It looks for files larger than 1 GiB and acts only when usage on the log or root filesystem is above 85%. Truncation preserves the inode, so processes with the file already open keep writing without needing a restart. Requires root.

# qs-trunclog scan
{Thu Sep  3 05:34:31 2026, INFO, sh:qs_trunclog} Log/Root filesystem usage: / = 25%
{Thu Sep  3 05:34:32 2026, INFO, sh:qs_trunclog} No files in /var/log exceed 1024 MiB.

Three operations: scan reports without changing anything, truncate acts if the threshold is crossed, and cronsetup installs an hourly job under /etc/cron.hourly to run truncate. Add --quiet for cron use and --debug when working out why it did or did not act.

Run scan first. It is the safe way to see what truncate would take.

qs-download-link

Publishes a file over HTTPS on port 8889 behind a short-lived random token, so a log bundle or a diagnostic capture can be pulled off an appliance without opening SSH access or setting up a share. Requires root.

# qs-download-link create /var/tmp/qs_logs.zip 30
https://10.0.8.141:8889/669936a2aeab5fd644c2deca?name=qs_logs.zip

# qs-download-link list
TOKEN                       EXPIRES_IN    FILE
669936a2aeab5fd644c2deca          1799s   /var/tmp/qs_logs.zip
Command Purpose
auto [minutes] Pick the newest /var/tmp/*.zip and the best local address automatically. This is the one to use straight after qs-sendlogs --no-upload.
create <file> [minutes] [--host <ip>] [--port <port>] Publish a specific file, optionally on a chosen address and port.
list Show the active tokens, the file each maps to, and the seconds left on each.
cleanup Remove expired and stale tokens.

Links default to 30 minutes. Tokens live in /var/tmp/nginx_tokens; cleanup is what removes them once they lapse.

Upgrades, packages and kernels

qs-upgrade

The product upgrade driver. It refreshes the package catalog and upgrades the QuantaStor packages and platform security updates from the configured repository, using the platform's own apt tooling. An upgrade started from the web interface runs this script, so the two paths are equivalent.

The arguments that matter in practice are --dryrun (do the setup work but do not upgrade), --coreOnly (QuantaStor core packages only), --includeKernel (also take kernel and driver packages, which needs a reboot), --force (discard cached repository metadata and re-download the package lists), --repoUrl (upgrade from a specific repository) and --getConfigValues (print the configuration it would use and do nothing else). Progress is written to /var/log/qs/qs_upgrade.log.

Note that these are Python argparse options that each take a value, so they are written --dryrun 1 rather than as bare flags.

Driving upgrades from the web interface, and the release-to-release notes, are on Upgrade Manager. Security-only package updates are covered on QuantaStor Security Updates.

qs-distupgrade

Upgrades the underlying Ubuntu release. A routine QuantaStor upgrade deliberately does not do this, because a distribution upgrade replaces drivers and platform packages and needs a much longer maintenance window. Reserve a maintenance window and contact OSNEXUS support before running it, so your logs can be reviewed on both sides of the upgrade.

The command itself is a dispatcher: it reads VERSION_CODENAME from /etc/os-release and hands off to the per-platform upgrade script for that release, preserving arguments and stdin. If no upgrade path exists from the running platform it says so and exits rather than doing anything. Run it inside screen so the upgrade survives a dropped session.

See Upgrade Manager for the surrounding procedure.

qs-apt-channel

Selects which QuantaStor apt repository channels the appliance uses, with validation and automatic rollback. Requires root.

Several channels can be enabled at once. Because apt resolves an overlapping package by taking the newest version, the script works out which of the enabled repositories will actually win for QuantaStor packages and reports that before it applies the change.

# qs-apt-channel --status
Current QuantaStor apt repositories (/etc/apt/sources.list.d/osnexus.list):
  release          deb http://packages.osnexus.com/packages quantastor-jammy main

# qs-apt-channel --list
Channels (from /opt/osnexus/quantastor/conf/qs_apt_channel.conf + ...):
  http://packages.osnexus.com/packages
        [x] release         quantastor-jammy                 default, production
        [ ] optional        quantastor-jammy-optional        add-on/optional packages ...
        [ ] rc              quantastor-jammy-rc              staged stable build (release candidate), not yet GA
        [ ] preview         quantastor-jammy-preview         TESTING ONLY - next-release preview; NOT for production use
...
Argument Effect
<channel> Enable exactly that channel and nothing else.
--channels a,b,c Enable exactly this set of channels.
--select Choose interactively.
--url <base-url> Use a custom repository base. Combinable with --channels; on its own it becomes the only source.
--status Show the repository entries currently in place.
--list List the configured channels and which are enabled.
--dry-run Validate the target selection and change nothing.
--non-interactive Skip the confirmation prompts, and roll back automatically if post-switch validation fails.
--force Keep the new repositories even if post-switch validation fails.
--repair-pins Regenerate the apt pin file when an enabled suite has no pin stanza.

The channel definitions come from /opt/osnexus/quantastor/conf/qs_apt_channel.conf, extended additively by drop-in files under /var/opt/osnexus/quantastor/conf/qs_apt_channel.conf.d/. Production systems belong on release; the rc and preview channels are for testing, and optional carries add-on packages and is meant to be enabled alongside a main channel rather than on its own.

Exit codes are meaningful: 0 for success, 1 for a usage or environment error, and 2 when the target selection failed validation.

qs-package-check

Compares the packages installed on the appliance against the reference package list captured for that QuantaStor version and platform, which makes an unexpected package install or a missing dependency visible. Runs without root.

# qs-package-check get-tag
qs-pkglist-ubuntu-jammy-6.8.0-90-generic-qs-6.9.0.274

check diffs against the best-matching baseline ignoring versions; check-withver includes versions. Both accept a path to a specific package list file. capture-list prints the current package set as a sorted list and log-list writes it under /var/log/qs/, which is the right thing to do before a change you may need to compare against later. list-all, list-matching and list-best show which baselines are available, and get-tag prints the baseline tag for the local system.

qs-kernel

Installs a specific Ubuntu kernel version and builds the QuantaStor DKMS drivers against it. Requires root. Intended for systems that do not have Secure Boot enabled, since the DKMS modules it builds are unsigned.

qs-kernel --list
sudo qs-kernel --custom-kernel 6.8.0-90-generic --reboot
sudo qs-kernel --purge-old --yes

--list shows the most recent signed generic kernels. --custom-kernel <version> installs that kernel's headers, image, modules and extras, and --reboot reboots when it is done. --purge-old removes kernels that are not in the shipped list for the platform release, with --yes to skip the prompt.

Around the install it turns on DKMS source builds with a touchfile, temporarily lifts QuantaStor's apt pinning so the kernel packages can install cleanly, and puts the pinning back afterwards. It runs the driver install a second time deliberately, to cover the case where the kernel and headers were already present and the DKMS hook was skipped. Follow the build with tail -f /var/log/qs/qs_dkms.log.

qs-kernelupgrade

Upgrades the kernel and driver packages to the current shipped versions. This is the path qs-upgrade --includeKernel takes, rather than something normally run by hand. --repoUrl selects a repository and --targetDist a target platform release; --taskId is used by the service to attach progress to an upgrade task.

Use qs-kernel instead when the goal is a specific kernel version rather than the shipped one.

qs-dkms-install

Builds and installs the QuantaStor DKMS drivers. It normally runs from the kernel package's post-install hook rather than by hand, which is why its first two positional arguments are a kernel version and a boot path. Requires root. Logs to /var/log/qs/qs_dkms.log.

--dryrun performs the setup steps without installing, --getConfigValues prints the configuration and exits, --buildDrivers builds from source when no pre-built driver is available, --skipDownload uses only what is already local, and --repoUrl names the repository to pull driver packages from.

Driver support and the shipped driver set are covered on QuantaStor Driver Upgrades.

Performance measurement and tuning

qs-iostat

A front end to iostat and the ZFS kernel statistics, for looking at where I/O time is going. Runs without root.

Argument Shows
-c Globally averaged CPU statistics.
-d I/O statistics for all devices.
-z Extended I/O statistics for the devices backing ZFS storage pools only.
-a ZFS ARC, ZIL, prefetch and L2ARC statistics.
-f Repeat every 2 seconds.
--extra "<args>" Pass additional arguments straight through to iostat.
# qs-iostat -a
ZFS Adaptive Replacement Cache (ARC) / read cache statistics

Name                              Data
---------------------------------------------
hits                              1582909
misses                            4
c_min                             260208768
c_max                             5692719104
size                              16686912
l2_hits                           0
...

-z is the one to reach for on a pool that feels slow: it restricts the device view to the pool members, so a single slow disk stands out instead of being averaged away. Use qs-iostat -z -f while the workload is running.

What the ARC and ZIL numbers mean, and what to do about them, is on Performance Tuning; see also IO Performance Tuning and Performance Monitoring.

qs-zfstunings

Prints every ZFS and system tunable QuantaStor manages, with its current value beside its default and a state column saying whether the two agree. Runs without root, and changes nothing.

# qs-zfstunings
SECTION                         | TYPE       | STATE | TITLE                            | CURRENT                     | DEFAULT
sst_resilver_min_time_ms        | range      | MATCH | Resilver Priority (msec/TXG)     | 3000                        | 3000
sst_cache_size                  | percentage | MATCH | Cache Size (% of RAM)            | 68.37% (5.30 GiB)           | 70% (5.43 GiB)
sst_prefetch_disable            | boolean    | MATCH | Prefetch Disable                 | 0                           | 0
sst_resilver_prio               | -          | SKIP  | deprecated                       | <deprecated>                | <none>
...

The state is MATCH when the running value agrees with the default, DIFF when it does not, and SKIP for tunables that are retained but deprecated. Percentage tunables are compared with a small tolerance, so a value derived from installed RAM is not reported as a difference just because the arithmetic rounds differently.

This is the quickest way to answer "has anything on this system been tuned away from stock", which is usually the first question when a pool underperforms. Definitions come from /opt/osnexus/quantastor/conf/qs_systemtunables.conf; pass a different file as the first argument to read one from elsewhere, such as a copy taken from another appliance.

Each of these tunables is settable from the web interface -- see Storage System Optimization for the full option reference and Storage System Tunable Set for the dialog.

qs-perftest

Baseline disk, pool and share benchmarking, wrapping fio, dd, elbencho and iozone. It can read every disk in the system, read only the disks backing one pool, write and re-read a file on a pool, or create temporary shares and benchmark them. Requires root.

Full coverage of the operations, arguments and how to read the results is on Performance Testing.

qs-ramdisk

Creates transient RAM-backed SCSI disks, so a benchmark can be run against memory and the storage transport measured on its own without the pool in the way. Optionally adds a synthetic per-command latency, which is how you calibrate how much latency a transport is contributing.

These devices are volatile: everything on them is lost on destroy and on reboot. They are for measurement only -- never place data on them and never build a production pool from them.

Operations and worked examples are on Performance Testing.

qs-util -- general maintenance

qs-util is the largest of these utilities: one command with around 150 operations covering Linux-level maintenance that has no natural home elsewhere. Some of it is support tooling, but a good deal of it is the sort of thing an administrator needs during a network change, a hardware swap or a capacity problem.

Run qs-util with no arguments for the full usage block.

Almost every operation requires root. Ten do not: lastreboot, lastshutdown, iscsiiqn, monitorinstall, distro, nvmelist, nvmecompression, mexec, fixosrelease and usage. Everything else prints ERROR: Use 'sudo' with this command, operations must be run as root. and stops.

Note that the usage block does not list every operation the command accepts; a number of internal operations are dispatched but undocumented.

System information

Operation Purpose
lastreboot / lastshutdown When the system last came up, and when it last went down cleanly. The pair distinguishes a clean shutdown from a crash.
distro The platform release codename, e.g. jammy.
zfsversion The installed ZFS package and driver versions.
checkports Which process is listening on each QuantaStor service port.
showservices Every listening socket on the system, including NFS and SMB.
showclients Currently connected clients across NFS, SMB and the other protocols.
devicemap Each /dev/sdX beside its stable /dev/disk/by-id/ path, make, model and serial. The one to use when a log names a device that has since been renumbered.
devinfo <sdN> The I/O tuning settings in effect for one device.
nvmelist PCIe paths of the NVMe devices, in the form the slot map configuration file expects.
nvmecompression Compression achieved on NVMe media that reports it.
iscsiiqn The local system's iSCSI IQN.
checkpass Whether the qadmin console account is still on its default password.
# qs-util devicemap
/dev/sdb    /dev/disk/by-id/scsi-SVMware_Virtual_disk_6000c295..., VMware, Virtual disk, 6000c295...

ZFS memory and swap

Operation Purpose
zfsparams Print every ZFS tuning parameter currently in effect.
zfsarcsummary The ZFS subsystem report -- ARC sizing, hit rates and breakdown.
setzfsarcmax <pct> / setzfsarcmin <pct> Set the ARC ceiling and floor as a percentage of system RAM.
zfsarcfix [-c] Drop the ARC if it has grown too large. -c only checks.
clearcache Drop the system page cache.
checkswap [-c] Clear the cache if swap use is above 90%. -c only checks.
addswap <GB> / resizeswap <GB> Add a swap file, or resize the existing swap.
importall Import every available ZFS storage pool.
zpooldisks zpool status annotated with enclosure, slot, serial, make and model for each disk -- so a faulted member maps to a physical slot.

zpooldisks is the one to run before pulling a disk. Plain zpool status gives a device name; this gives the enclosure and slot to walk to.

Networking

Operation Purpose
flusharp Flush the ARP cache.
arping Send gratuitous ARP request and reply packets, to make switches relearn a moved address.
addgw <ip> <nic> / delgw <ip> <nic> Add or remove a default gateway.
addnet <net> <mask> <nic> / delnet <net> Add or remove a route.
ifdown <nic> Bring an interface down, with extra checks first.
pingtest <ip>,<ip>,... [timeout] Ping a list of addresses, with an optional timeout in milliseconds.
npblink <nic> <seconds> Blink an interface's port LED, to identify the physical cable.
gppset <uuid> <ip>, gpplist, gppclear Override, list and clear the grid preferred port -- the address a grid member is reached on.

iSCSI initiator and Fibre Channel target

Operation Purpose
iscsiinstall Install the open-iscsi initiator software.
iscsidiscover <ip> Discover iSCSI targets at an address.
iscsilogin <ip> Discover and log in to every target at an address.
iscsirelogin <ip> Log back in to targets already established at an address.
alua Print the ALUA configuration state.
listfcclients List the connected Fibre Channel clients.
issuelip Issue a LIP to every FC port, forcing a loop re-initialisation.
enabledualmode / disabledualmode Put the QLogic FC driver into combined initiator+target mode, or back to target-only.

Encryption keys and LUKS devices

These operate on the LUKS layer under an encrypted storage pool. Key material is handled directly, so treat them as support-assisted operations.

Operation Purpose
cryptgenkey <outfile> Generate a new 256-bit encryption key file.
cryptwrapkey <keyfile> <kwfile> <saltfile> Wrap a key and write the key-wrap and salt files. Prompts for the passphrase; for a pool with no passphrase, use the storage pool ID as the passphrase.
cryptdecryptkey <kwfile> <saltfile> [outfile] Unwrap a wrapped key using its salt file.
cryptrecoverkey <pooluuid> Unwrap a pool's key-wrap and salt files and place the plaintext key in /run/quantastor/cryptconf/keys.
cryptformat <device> <keyfile> LUKS-format a device with a given key.
cryptopen <device> [keyfile] Open a LUKS device. With no key file, every key under /run/quantastor/cryptconf/keys is tried.
cryptclose <device> / cryptcloseall Close one or all LUKS devices.
crypttabopenall Open every device listed in /etc/crypttab.
crypttabrepair Rebuild /etc/crypttab by trying every available key against every LUKS device.
cryptrekeydevice <oldkey> <device> <newkey> Add a new key slot to a device using the existing key.
cryptswap <device> Encrypt the swap device, updating /etc/fstab and /etc/crypttab.

Pool key export and import from the web interface are on Storage Pool Export Encryption Keys and Storage Pool Import Encryption Keys.

Web access, ciphers and hardening

Operation Purpose
disablehttp / enablehttp Restrict web management to HTTPS, or allow plain HTTP as well.
disablehttpgzip / enablehttpgzip Turn HTTP gzip compression in nginx off or on.
setweblogin "admin" Set the username prefilled on the login page; pass "" to clear it.
ciphers [HIGH|ALL] Set the OpenSSL cipher selection the core service uses.
wuicustomcerts Allow custom certificate and cipher settings for the web interface, from /var/opt/osnexus/quantastor/ssl/nginx_quantastor_ssl_custom.conf.
wuicustomcertsrestart Restart nginx after changing those files, checking for configuration errors.
wuicustomcertsdefault Revert to the shipped certificates and cipher settings.
cacertusedefault / cacertuselegacy Use the standard 2048-bit RSA CA certificates, or the legacy 1024-bit set for compatibility with very old clients.
enablefips / disablefips Turn FIPS mode on or off.
ssh-sha1-audit Detect and remediate SHA-1 HMAC algorithms offered by OpenSSH.

Certificate replacement, cipher policy and the web access model are covered in full on Security Configuration.

Protocol services

Operation Purpose
enablesmbmulti / disablesmbmulti Turn SMB multi-channel support on or off.
nfsganesha Give the scale-out NFS Ganesha service the primary NFS port, 2049.
nfsstandard Move Ganesha to the secondary port, 2249, leaving 2049 to the kernel NFS server.
nfsrdma Enable NFS RDMA mode.
snmprestart Restart the SNMP service and agent.
snmpmib Print the contents of the QuantaStor SNMP MIB.
snmpwalkall, snmpwalkvolumes, snmpwalkalerts Walk the whole MIB, or just the volume or alert subtrees.

Replication and clone bandwidth

Operation Purpose
rlinkcheck <ssid> <ip> Verify that a remote system is reachable for replication over a given address.
clratelimitget The current maximum clone throughput, shared across all active clone operations.
clratelimitset <MB/s> Set that shared maximum.
clraterebalance Rebalance active clone operations across the shared limit.

The default shared clone limit is 200 MB/sec, and QuantaStor rebalances active clone streams every minute on its own unless /etc/clratelimit.disable exists -- so clraterebalance is only needed by hand when that file is in place.

Replication bandwidth is no longer set here; it is a per-link setting on the Storage System Link. See Remote-replication / Disaster Recovery Setup.

Triage, simulation and recovery

Operation Purpose
loadlog Append top -bn 1 output to /var/log/qs/qs_load.log.
loadlogenable / loadlogdisable Add or remove a cron job doing that every 2 minutes -- the way to catch a load spike that only happens overnight.
enablesysstats Enable sysstat collection, viewable with sar.
simstart <osn.db> Put the service into simulation mode against a configuration database from another system, including one fetched over HTTP from a log server. Support use.
hwsimstart <url> Simulate hardware controllers from information in a service log. Support use.
simstop Leave simulation mode.
dbtables List the tables in the configuration database.
dblist
Dump one object table. Support use.
dbdiff Differences between osn.db and osn.db.preupgrade -- what an upgrade changed.
dbremove <id>
Delete a row by ID. Support use only, and destructive.
resetids Reset the storage system UUID on next reboot. Needed after cloning an appliance image, so the copy does not collide with its source.
resetadmin Reset the QuantaStor admin user's password.
expandboot Grow the boot device logical volume and filesystem to fill the device.
ipmireset Refresh IPMI information. Needed after a power supply is added.
ipmicapture Capture IPMI information for hardware certification.
ntpserver / ntpfix Show the primary NTP server, and correct clock skew.
forcereboot / forceshutdown Reboot or power off immediately via the sysrq trigger, bypassing a graceful shutdown.

forcereboot and forceshutdown do not flush anything or stop services. They are for a system that will not shut down any other way.

Repository and platform

Operation Purpose
aptdefault Point apt at the default packages.osnexus.com repository.
aptpreview / aptstablepreview Point apt at a preview repository. Testing use.
aptpin / aptnopin Re-enable or disable apt package pinning. Development and QA use.
aptoffline <iso-file> Serve upgrades from a QuantaStor ISO instead of the network, for a disconnected site.
aptonline Return to the online repositories and remove the offline ISO.
monitorinstall Watch an apt install or upgrade in progress.
updateinitramfs Rebuild the initramfs.
noapic Add noapic to the GRUB boot parameters.
haproxyconfig Create the HAProxy configuration directory.
apcupsenable / apcupsdisable Enable or disable apcupsd for APC UPS monitoring.
containerrefresh <uuid> Invalidate the local FUSE cache for a cloud container.
lsistorcligetdata <n> Collect LSI MegaRAID controller data into text and tar files for one controller.
seagate-ap-ha / seagate-ap-split Configure a Seagate AP system for shared-SAS HA, or split for scale-out. Also available as qs-seagate operations.
ramcopy [size] [-keepfiles] [-y] Create a file in a ramdisk and copy it repeatedly into the working directory -- a crude write-throughput check on the current filesystem.

aptoffline deserves its own mention: point it at a QuantaStor ISO and the appliance upgrades from the ISO's package repository, which is the supported way to upgrade an appliance with no route to the internet.

For a repeatable, validated channel change, prefer qs-apt-channel over the apt* operations here -- it validates the result and rolls back on failure.

Storage pools, shares and encryption

qs-zconvert

Imports a foreign OpenZFS pool and renames it into the QuantaStor convention, so it can be managed as a QuantaStor Storage Pool with its data intact. QuantaStor names pools qs-<UUID> and volumes by UUID, so a third-party pool has to be renamed to be managed -- and the original names are preserved as custom ZFS properties on the datasets rather than discarded, so nothing is lost.

# qs-zconvert list
no pools available to import
Operation Purpose
list Pools available to import.
listall The same, in detail.
import <poolname> [-noscan] Import and convert the pool and its volumes.
convertvols <poolname> Convert only the volumes to UUID naming.
importhg <file> Create host groups and hosts from stmfadm list-hg -v output.
importlumap <file> Assign logical units to host groups.

Back up the data before converting. The tool is reliable in practice, but not every combination of ZFS implementation and version has been tested.

You do not normally need to run this by hand: it is what the pool import path in the web interface uses, and the same operation is available as qs pool-import and over the REST API. See Storage Pool Import.

qs-zvolutil

Repairs unreadable blocks inside a storage volume by overwriting the bad 4K sectors with zeros, which updates the ZFS checksum so the volume can be read again.

This exists for a specific situation. A pool laid out as RAID0 at the ZFS layer on top of hardware RAID still detects bit-rot -- the checksums do not match -- but has no redundancy to repair it from, so the bad block returns an I/O error forever. Putting ZFS on top of hardware RAID is not recommended for exactly this reason.

qs-zvolutil scan   dev=/dev/zd0
qs-zvolutil repair dev=/dev/zd0

scan is non-destructive and prints . for a good block and e for a bad one. repair overwrites each bad sector, printing r for a repair and x for a repair that failed.

This does not recover data -- it makes the volume readable again by zeroing what could not be read. Afterwards, run a pool scrub to confirm no errors remain, zpool clear to clear the pool state, and then a filesystem check inside the guest filesystem on the volume to find out which files were affected. You may still need to restore those files from backup. Use it with OSNEXUS support.

qs-zpoolscrub

Starts, stops or schedules a scrub across all QuantaStor pools. A scrub reads everything in the pool, detects bit-rot and repairs it where the pool has redundancy.

qs-zpoolscrub --startall
qs-zpoolscrub --stopall
qs-zpoolscrub --cron

--cron installs a crontab entry that scrubs monthly, on the last Saturday of the month at 1am.

Note the usage text this command prints names zpoolscrub rather than qs-zpoolscrub; use the qs- form shown above.

Prefer scrub schedules for anything ongoing. They are managed as objects, so they report their history and can be modified and triggered without editing a crontab -- see Storage Pool Scrub Schedule Create and Storage Pool Scrub.

qs-luksutil

Adjusts the dm-crypt options on the LUKS devices under an encrypted storage pool, and converts LUKS1 headers to LUKS2. Requires root, since it talks to the device-mapper.

# qs-luksutil show-config
INFO: Current 'dmsetup table' lines referencing enc-:
No enc- devices found.
Operation Purpose
show-config The dmsetup table lines for the encrypted devices.
convert-luks2 --pool <id> Convert the pool's devices from LUKS1 to LUKS2, interactively.
disable-workqueue --pool <id> Add no-read-workqueue,no-write-workqueue, taking the kernel work queues out of the crypto path.
enable-workqueue --pool <id> Remove those flags.
enable-same-cpu --pool <id> Add same-cpu-crypt,submit-from-crypt-cpus, keeping encryption on the submitting CPU.
disable-same-cpu --pool <id> Remove those flags.
enable-discard --pool <id> Enable discard/TRIM pass-through on an encrypted pool.

The tuning operations share a set of modifiers: --live applies the change to the running devices, --persistent makes it survive a reboot, --force skips the safety prompts, --skip-kernel skips the kernel version check, and --trim (on enable-discard) issues a TRIM as well as enabling it.

Bypassing the crypto work queues can improve throughput on fast NVMe media and hurt it elsewhere -- measure before and after with qs-iostat -z rather than applying it on principle.

qs-fsmon

Controls the filesystem monitor instances, which handle the immutability expiration scans behind WORM network shares. Requires root. It is normally driven by the service rather than by hand.

start, stop and status manage the instances -- status prints the running scan for each pool, with its command line. refresh reloads the monitor configuration, and scan <sharepath> starts an immediate recursive scan of one share.

Note the usage banner this command prints names a different script; the installed command is qs-fsmon.

qs-shareusage

Scans a network share and writes per-user and per-group capacity statistics as JSON, which is what backs the share quota reporting. Requires root.

qs-shareusage scan-share --path=/mnt/storage-pools/qs-<pool-uuid>/<share> \
    --share-id=<share-uuid> --output-path=/mnt/storage-pools/qs-<pool-uuid>

--path is the share root to scan, --share-id the share UUID (used as the output filename), and --output-path the storage pool mountpoint under which the .usage/ report directory lives. --top-n sets how many top users and groups to record, default 10, and --threads the concurrency, default 20.

The reports it produces are what the Network Share Users and Group Quota Manager displays.

High availability and I/O fencing

qs-crm

Reports and manipulates the Corosync and Pacemaker configuration behind cluster virtual interfaces. On a system with no site cluster configured it says so and exits, which is itself a useful check.

# qs-crm status
ERROR: Corosync service is not running..
Operation Purpose
status Node and resource status.
watch Monitor resource management and pool status continuously.
show The resource configuration.
detail Ring configuration details.
siteid The site cluster UUID of the active Corosync configuration.
mcast Multicast settings for the local node.
online / offline Bring the local node online, or put it into standby.
start / stop / restart Start, stop or restart Pacemaker and Corosync.
start_gridip Start the grid IP resource and leave maintenance mode.
move_gridip <node> Move the grid IP resource to a node.
move_resource <resource> <node> Move one resource to a node.
move_resource_group <group> <node> Move a whole resource group to a node.
add_resource <resource> <group> <ip> <nic> <cidr> <node-a> <node-b> Create an HA cluster resource and add it to a group.
add_resource_to_group <resource> <group> Add an existing resource to a group.
rm_resource <resource> Stop, delete and clean up a resource.
enable_logging / disable_logging Turn Corosync logging to /var/log/corosync.log on or off.
pcs_bind_localhost / pcs_unbind_localhost Bind pcsd to 127.0.0.1, or let it listen on all addresses. Restarts pcsd either way.

status and watch are the triage operations. The resource-manipulation operations change live cluster state -- use them with support, and prefer the HA group operations in the web interface. See Setup Guide for Clustered HA Storage Pools and Clustered HA SAN/NAS (ZFS based) Storage Pool Setup.

qs-iofence

The I/O fencing engine QuantaStor uses to protect highly available ZFS storage pools, exposed as a command. It speaks SCSI-3 persistent reservations and NVMe reservations over SAS, iSCSI, Fibre Channel, NVMe and NVMe-oF. Requires root -- it refuses to run otherwise, since reservation commands need elevated privileges.

Run qs-iofence with no arguments for the full command list; --help prints only the flags.

# qs-iofence devstatus
/dev/sda 6000c295cd1c5a1b1681d69a78bb082c (NOT-SUPPORTED) [NOT-SUPPORTED] <>
/dev/sdb 6000c29533831d8fb680dded32555cf4 (NOT-SUPPORTED) [NOT-SUPPORTED] <>

devstatus is the operation to reach for. With no arguments it reports the fencing state of every device; give it a comma-separated list of serial numbers to narrow it down. NOT-SUPPORTED means the device does not implement persistent reservations, which is normal for virtual disks and for media that is not part of a shared-storage HA configuration.

Operation Purpose
devstatus [<sn>,...] Fencing status of all, or of the named, devices.
devregister <sn>,... <regkey> Register with the named devices.
devreserve <sn>,... <regkey> Reserve the named devices.
devrelease <sn>,... <regkey> Release a reservation held by a key.
devpreempt <sn>,... <regkey> Preempt the current reservation; the given key takes ownership.
devreregister <sn>,... <regkey> Change the reservation key without changing who holds the reservation.
devclearall <regkey> Clear registrations -- not reservations -- for devices registered with a key.
devreleaseall <regkey> Release every reservation held by a key.
devscrub <sn>,... Scrub all SCSI-3 persistent reservation state on every path to the named devices.

Useful flags: -m for machine-readable output, -verbose to print the logs to stdout, -log to write /var/log/qs/qs_iofence.log, -sid to use mode page 83 device IDs, -itid for a unique reservation key per I_T nexus, and -wear to use Write Exclusive All Registrants instead of the default Write Exclusive Registrants Only.

Everything except devstatus modifies reservation state on shared media and can fence a running node off its own pool. Use those only with OSNEXUS support.

qs-scstdlm

Sets up the SCST Distributed Lock Manager integration, which is what makes SCSI-3 persistent reservations consistent across the nodes of an HA cluster. Requires root.

Nothing here normally needs running by hand: QuantaStor configures the DLM on every node itself when an HA group has distributed locking enabled. The commands exist for verification and for support-assisted repair.

Operation Purpose
check Report whether this node is ready, changing nothing. The first thing to run.
status Cluster status, the same as pcs status.
inspect Every device that can carry clustered reservations, with its cluster mode, USN, device ID, backing path, whether its lockspace is joined on this node, and the reservations it currently holds. --no-prs prints the table without the reservations.
install-pkgs Install the required packages. One-time.
setup-dlm Prepare configfs on this node. One-time, after install-pkgs.
configure-dlm Create the DLM resource in Pacemaker. Run on one node; it replicates to the rest.
start-dlm Start the DLM resource on this node. Automatic on cluster startup after the first time.
cleanup-dlm Remove the DLM resource, take this node's devices out of cluster mode, and reclaim the phantom devices holding their lockspaces. Run on every node before uninstalling packages or changing configuration.
auto-enable / auto-disable Used by the service to converge this node when an HA group's distributed locking setting changes. Idempotent.
# qs-scstdlm check
NOTREADY: the Pacemaker 'dlm' resource has not been created in the cluster.

The setting that controls all of this is Enable SCSI3-PR Distributed Locking on the HA group, and it has three values rather than two:

  • enabled -- clustered reservations on for that pool.
  • disabled -- off. This is the only value that will take clustered reservations away from a running pool.
  • auto -- nobody has said. It resolves to enabled while the pool is already carrying clustered reservations or the node has a Fibre Channel ALUA HBA, and to disabled otherwise.

That auto behaviour is what stops an upgrade silently switching a working cluster off: an HA group created before the setting existed reads as auto, as does any API call that omits the field. New HA groups are created enabled, for iSCSI and Fibre Channel alike.

There is also a support override. Creating the touchfile /var/opt/osnexus/quantastor/touchfiles/tf_dlm_cluster_pr.enable forces clustered reservations on for every HA pool on the node regardless of the HA group setting, which is useful while HA groups are being created and deleted. Nothing in the product writes or removes it. Deleting it returns to per-HA-group control, but does not strip a pool that is still carrying clustered reservations, because an HA group left on auto resolves to enabled in that case.

qs-pcsnodealert

The Pacemaker alert handler that turns node membership changes and node attribute changes into QuantaStor alerts, plus the helpers to install and verify it. Requires root. Pacemaker invokes it with no arguments; the operations below are for administration.

register creates or updates the qs_node_alert entry in the cluster information base for this script, verify (with optional --verbose) checks the configuration, show prints it, and delete removes it. test node <member|lost> [node [nodeid]] and test attr <name> <value> [node [nodeid]] fire a synthetic alert so the path can be exercised without disturbing the cluster, and standby on|off [node] toggles standby through attrd_updater, which emits a real attribute alert.

Set QS_DEBUG=1 for extra logging to /var/log/qs/qs_pcsnodealert.log.

Scale-out (Ceph) utilities

qs-ceph

The general-purpose Ceph helper. It calls the ceph tools underneath and gathers information on monitors, pools, OSDs, users and overall cluster health, and carries a set of one-shot maintenance fixes. Requires root. On a system with no Ceph cluster it reports an error initialising the cluster client.

Group Operations
Status status, mons [--update|--status], pools, osds, user --username <name>
Compression check-compression, check-mode, update-mode [--algorithm snappy|lz4|none] [--pool <name>]
Backups mon-db-backup (stops the monitor safely, backs up the monitor database, restarts it), config-backup (the cluster configuration files, not the monitor database)
Maintenance fixownership, fixpermissions, updatediscard, updateosdmax, balance, fix-rgw-conf, cephfs-mount-switch --mount kernel|fuse, enablecephfs-top, enable_grafana_iframe
Recovery osdrmlist --osd <n> -- scans an offline OSD and produces a file of object-removal commands
Destroy clearconfig, cleanup

balance enables the placement group balancer so data spreads evenly across OSDs, and updateosdmax raises osd_max_pg_per_osd_hard_ratio and mon_max_pg_per_osd to the recommended values -- both are worth knowing when a cluster reports too many placement groups per OSD.

osdrmlist exists for the specific case of a full OSD that will not start: it produces the commands to remove objects so the OSD can come up. Use it with great care, and with support.

cleanup destroys all Ceph configuration data on the system and clearconfig removes the configuration files. These are irreversible. They exist to wipe a cluster configuration from a node, and should only be used by OSNEXUS support.

Cluster setup and day-to-day management belong in the web interface -- see Scale-out Block Setup (ceph), Scale-out File Setup (ceph) and Scale-out Object Setup (ceph).

qs-ceph-journalgroup

Replaces a failed drive in a Ceph journal group. Requires root.

jglist lists the journal groups across the storage grid, jgcheck reports the status of the local journal devices, vglist lists the local volume groups, and udevices lists the local unused devices -- so the replacement candidate can be picked. Then:

qs-ceph-journalgroup replace -vg ceph-vg-cea4d812-1f2e-d29f-3b91-a6a551cc43fb -nd sdb

-vg is the volume group UUID of the faulted journal group and -nd the new device; both are required. -d adds debug output.

The equivalent web interface operations are on Ceph Journal Group Create and Ceph Journal Migrate.

qs-osd-compact

Runs an online RocksDB compaction on the OSDs of the local node, which recovers space and latency on an OSD whose metadata database has grown. Requires root, and only operates on local OSDs. Logs to /var/log/ceph/qs_osd_compact.log.

qs-osd-compact compact -c ssd
qs-osd-compact compact -o osd.1,osd.2
qs-osd-compact stopcompact

-c|--class selects a storage class (ssd, hdd or nvme) and compacts every local OSD in it; -o|--osds names specific OSDs. The two can be combined. stopcompact kills the active compaction jobs, optionally narrowed by the same two arguments. -v and -d add verbose and debug output.

Compaction is I/O intensive. Run it outside a busy period, and use stopcompact if it starts affecting client latency.

qs-cephfsutil

Analyses and optimises CephFS directories, moving file data between Ceph data pools by size or age so that small files and large files land on appropriate media. Runs highly concurrently -- 20 threads by default.

qs-cephfsutil analyze-directory --path=/mnt/cephfs/projects --recursive --human
qs-cephfsutil optimize-directory --path=/mnt/cephfs/projects --target-datapool=cephfs-data-nvme \
    --min-filesize=10MB --max-age=30d --dry-run

analyze-directory (alias ad) reports the file size distribution of a directory and takes --path, --recursive and --threads.

optimize-directory (alias od) does the movement. --path and --target-datapool are required; --min-filesize / --max-filesize and --min-age / --max-age select what to move (sizes as 10MB or 1GB, ages as 7d, 24h or 30m), --checksum verifies the first and last 4K of each copied file against the source before the rename, --skip-errors keeps going past failures, and --command-err-log records what failed.

Always run it with --dry-run first. Both operations also take --csv, --json, --human and --noheader for the report format.

qs-s3util

Object storage analysis and test-data generation against an S3 gateway. Every operation takes either --accesskey and --secretkey, or a --credsfile -- the credentials file is written the first time keys are supplied and reused afterwards, which keeps keys out of the shell history.

Operation Purpose
analyze-buckets (ab) Size composition and storage-class placement of one or more buckets.
list-buckets (lb) List buckets, optionally for one --tenant.
list-objects (lo) List the objects in a bucket.
inspect-object (io) Deep inspection of one object -- the RADOS objects behind it and where they live.
move-objects (mo) Move objects between storage classes, filtered by size and date.
create-buckets (cb) Create a batch of buckets. Test-data generation.
create-objects (co) Create a batch of objects of a given size. Test-data generation.
qs-s3util analyze-buckets --endpoint=http://10.0.8.200:7480 \
    --accesskey=aAbBcCdDeEfF01234567 --secretkey=aAbBcCdDeEfF0123456789aAbBcCdDeEfF012345 \
    --bucket-name=archive-2026 --human --csv --reportfile=archive.csv

analyze-buckets and move-objects share the selection and reporting arguments: --bucket-name (a name or comma-separated list) or --bucket-list (a file of names), --date-start / --date-end, --object-size-min / --object-size-max, and --csv, --json, --human, --noheader, --reverse-sort and --reportfile for output. A bucket in an S3 tenant is named tenant/bucket.

Two things matter on a large bucket. --interim prints progress to stdout every 30 seconds by default, so a long analysis is not silent. And move-objects threads twice -- --threads across buckets (default 10) and --sub-threads within each bucket (default 10), for 100 concurrent operations by default; --command-err-log records the moves that returned errors, which is how you find out why objects did not move.

qs-rgw-autotier

Manages the S3 auto-tiering Lua script in the RADOS gateways. Requires root, and must be run on a node with a local RGW instance. Rules come from /etc/ceph/qs_rgw_autotier.prop.

enable injects the Lua script into the gateways for the default tenant and every named tenant -- it is safe to re-run, and re-running is how an updated script is rolled out. disable removes it. status reports the rules in effect and whether the script is installed per tenant, and is read-only.

For diagnosis: show prints the gateway's current preRequest Lua handler, enable-debuglog turns on RGW debug logging in /etc/ceph/ceph.conf and restarts the local gateways, showlog filters /var/log/radosgw/ down to the Lua operations, and disable-debuglog turns it back off. Run enable-debuglog before showlog or there will be nothing to see.

objcount reports the number of RADOS objects in each bucket data pool -- the way to confirm tiering is actually moving data -- bucketlist lists the buckets in the object pool group, and garbagecollect runs garbage collection immediately.

qs-rgw-monitor

Monitors a RADOS gateway by writing to a bucket through it, and restarts the gateway if that stops working. Requires root.

qs-rgw-monitor monitor -e https://10.0.8.80:7480 \
    -a aAbBcCdDeEfF01234567 -s aAbBcCdDeEfF0123456789aAbBcCdDeEfF012345 -n bucket-verify

-e is the gateway endpoint, -a and -s the S3 access and secret keys, and -n the bucket to write to (default bucket-1). -f|--credsfile writes a credentials file the first time keys are given and reads it afterwards, so the keys need not be repeated on the command line. -d and -v add debug and verbose output.

Use a bucket created for the purpose. The check writes to it.

qs-rgw-moveobjects

Moves objects between S3 storage classes in bulk. Requires root.

qs-rgw-moveobjects -b bucketlist.txt -s STANDARD_IA -e http://10.0.8.80:7480 \
    -k aAbBcCdDeEfF01234567 -x aAbBcCdDeEfF0123456789aAbBcCdDeEfF012345 -r us-east-1 -m 60000

-b is a bucket name or a file listing one bucket per line, and accepts both bucket and tenant/bucket. -s is the destination storage class, -e the gateway endpoint, -k and -x the S3 access and secret keys, and -r the zone or region. -m sets a minimum object size in bytes -- only objects at or above it are moved, defaulting to 32768.

qs-s3util move-objects does the same job with date filtering, per-bucket concurrency control and an error log, so prefer it for anything large. Storage classes and lifecycle rules from the web interface are on Ceph Object Storage.

Certificates

qs-sslcert

Inspects, generates, installs and resets the certificates QuantaStor uses for grid communication, the CLI, the web interface and the REST API. Requires root. qs-sslcertgen is the same command under a second name.

printcerts is the read-only operation, and the one to start with. For each of the certificate files it prints the shipped default and any custom override, with an MD5 sum and the certificate subject, so a mismatch between grid members is immediately visible:

# qs-sslcert printcerts
INFO: Checking 'qscacert.pem' this is the CA Certificate used to generate/sign pem files.
        Default:         [1c88282f... /opt/osnexus/common/lib/qscacert.pem] [subject=...]
        Custom Override: [1c88282f... /var/opt/osnexus/quantastor/ssl/qscacert.pem] [subject=...]
INFO: Checking 'qsserver.pem' this is the service key and certificate used for grid communication.
        Default:         [e7042617... /opt/osnexus/common/lib/qsserver.pem] [subject=...]
        Custom Override: [NO OVERRIDE] []
...

The files it reports on are qscacert.pem (the CA that signs the rest), qsserver.pem (grid communication), qsclient.pem (grid and CLI communication), qsrestsrv.pem (the web interface and REST API), qskeypass (the CA passphrase -- NO DEFAULT is normal) and qsciphers.

Operation Purpose
printcerts Report the certificates in use. Read-only.
generatecerts Generate the whole grid certificate chain -- CA, client, server, REST -- and build a tar installer. Run it from the directory you want the chain created in.
installcerts Install the certificates from an extracted tar.
resetcerts Reset to the certificates shipped with QuantaStor.
fipsrewrap [passphrase] Re-encrypt installed private keys that FIPS mode cannot read to AES-256-CBC.

fipsrewrap covers a specific FIPS problem: a private key encrypted with 3DES, or in traditional PKCS#1 form whose key derivation uses MD5, cannot be read once FIPS mode is on. It re-encrypts the key to AES-256-CBC without changing the key material or the passphrase, and takes a backup first. The passphrase is read from the qskeypass file unless one is given on the command line.

There is also a set of advanced operations for building a chain step by step -- createca, createclient, createserver, setpempassword, certcreatetar, certtarmkdir, certtarrm, convertcustom and resetlegacycerts. Use those with OSNEXUS support; generatecerts and installcerts wrap the normal path.

Replacing certificates across a grid, including the ordering that avoids locking yourself out, is on Security Configuration. See also Update Certificates.

Statistics database

qs-statsdb

Queries and maintains the InfluxDB time-series database that the real-time charts in the web interface read from. Requires root. It is a triage tool: when a dashboard is empty or a chart has a gap, this is how you find out whether the data was collected.

# qs-statsdb count cpu
INFO: Querying record count for series 'cpu' based on field 'usage_guest'
9124

# qs-statsdb showrp
name                duration   shardGroupDuration replicaN default
----                --------   ------------------ -------- -------
autogen             0s         168h0m0s           1        false
quantastor          6h0m0s     1h0m0s             1        true
quantastor_rp_3w    504h0m0s   24h0m0s            1        false
quantastor_rp_78w   13104h0m0s 168h0m0s           1        false
quantastor_rp_usage 2016h0m0s  24h0m0s            1        false
Operation Purpose
print [mm] [limit] Print the data for a measurement or series. Default limit 10; 0 for no limit.
print_net [limit] / print_cpu [limit] The network and CPU series, filtered.
print_zfs_<name> A ZFS series, where <name> is arc, zil, prefetch, arc_mfu, arc_mem or l2arc.
count [mm] Number of entries in a series. The quickest check that collection is working.
showmm The measurements being collected.
showseries The series being collected, tagged by host.
showtags [mm] [key] / showfields [mm] The tags and fields of a measurement.
showdb The configured databases.
showrp The retention policies.
showcq The continuous queries that downsample the raw data.
setup Initialise the retention policies and continuous queries.
createdb <name> Create a database.
dropmm [mm], dropcq [cq], droprp <policy> Drop a measurement, continuous query or retention policy.
alter_mon_duration Set the monitor duration on the _internal retention policy to 2 days.
show_internal_rps Retention policies on _internal.
resetpassword [--password <pw>] Reset the InfluxDB quantastor user's password, generating one if none is given.

Retention is tiered: raw samples are kept briefly and continuous queries downsample them into longer-retention policies, which is why showcq and showrp are the two to look at when history is shorter than expected. The drop* operations delete collected data -- there is no undo.

Building your own dashboards on this data is covered on Grafana Ceph Dashboard & Prometheus Integration.

Replication SSH keys

qs-sshkey

Manages the SSH keys remote replication uses, and the ssh-agent instances holding them. Requires root. Keys live in /var/opt/osnexus/quantastor/replication-keys/.

Operation Purpose
listkeys The key files on disk.
agentkeys The keys currently loaded into the agents.
addkeys Load every key from the replication-keys directory into an agent.
addkey -k <keyfile> / removekey -k <keyfile> Load or unload one key.
generatekey -k <keyfile> -c <comment> Generate a key and load it.
checklink -k <keyfile> -i <remote-ip> Verify that SSH access to a remote system works for replication with that key.
stopagent / restartagent Stop the agents, or restart them and reload the keys.

checklink is the diagnostic for a replication link that will not establish -- it separates an SSH or key problem from a replication problem. Comparing listkeys against agentkeys finds the other common cause, a key present on disk but never loaded, which restartagent fixes.

Replication setup is on Remote-replication / Disaster Recovery Setup.

Enclosure helpers

qs-seagate

Collects canister logs from Seagate enclosures and sets their zoning mode. Requires root. Logs are written to /var/log/qs/qs_seagate_logs_<timestamp>.log.

qs-seagate get-logs --list
qs-seagate get-logs --device 4
qs-seagate seagate-ap-ha --dry-run

get-logs lists the SES devices and gathers a ddump from both canisters. --list prints the detected SES devices and exits, which is the safe first step; --device <id> forces the dump for one SES target; --dry-run skips the ddump commands; and --ammend-log appends to /var/log/qs/qs_seagate.log instead of writing a timestamped file.

seagate-ap-ha configures a Seagate AP system in HA shared-SAS mode (zone 1), for a scale-up ZFS HA cluster. seagate-ap-split configures split mode (zone 5), for a scale-out deployment. Both accept --dry-run, which prints the fwdownloader command without running it -- use it first, since these change enclosure zoning.

qs-wd

Manages device reservation support on Western Digital Data24 enclosures, which is what I/O fencing needs in order to work on that hardware. Requires root.

qs-wd 10.0.0.5 iofencing-status
qs-wd -p <admin-password> 10.0.0.5 iofencing-enable
qs-wd iofencing-device-status Y1M0A00XFM16,Y1M0A01MFM16

The <IP> is the management address of either Data24 IOM. Reservations must be enabled on both IOMs, so run iofencing-enable once against each IOM address, not just one. iofencing-status reports the current state and iofencing-disable turns it off.

Omit -p and the enclosure admin password is read from the terminal, which keeps it out of the process list and the shell history. Prefer that to passing it on the command line.

iofencing-device-status is different from the other three: it reports the local host's SCSI-3 persistent reservation and NVMe reservation state through qs-iofence, does not contact the enclosure, and so takes no address and no password. It optionally takes a comma-separated list of serial numbers.

Enclosure management from the web interface is on Hardware Controllers & Enclosures.

Boot repair

qs-bootefi-fix

Repairs the EFI System Partition entry in /etc/fstab on an appliance that fails to boot, or fails to update its bootloader, because the ESP is referenced by a device path that has changed. Requires root.

It takes no arguments and acts immediately. It backs up /etc/fstab to a timestamped file first, then labels the EFI partitions on SATA and NVMe devices EFI, replaces the /boot/efi line in /etc/fstab with a LABEL=EFI mount using nofail and a short device timeout, and remounts. Referencing the partition by label rather than by device path is what makes the entry survive the disks being renumbered.

On a system that is not booted in EFI mode it detects that and exits without changing anything.

Run it only when you are working a boot problem, and with OSNEXUS support. It rewrites /etc/fstab as soon as it is invoked, with no confirmation prompt.

Related pages


Verified against QuantaStor 6.9.0.