Upgrade Manager

The Upgrade Manager upgrades QuantaStor itself -- the core packages, the platform security updates, and optionally the kernel, the storage drivers and the Ceph daemons -- across one appliance or every member of a Storage Grid, from a single dialog. This page covers what each update class contains, every field in the dialog, how a grid upgrade is staged, what requires a reboot, and what to check before you start.

Section	Purpose
What the Upgrade Manager updates	The four update classes and which control selects each one
The General tab	Choosing systems, and reading the two version grids
Advanced Settings	Kernel and driver, Ceph, security exclusion and force -- every field
Where updates come from	The repository channel model and how availability is determined
Upgrading a Storage Grid	Staging order, version skew, and what a failed member does
Reboots and the WARNING state	What stages a reboot and how a pending restart is signalled
Watching an upgrade	The task, the alerts, the disconnect, and the log
Before you upgrade	Pre-upgrade checks worth doing
If an upgrade fails	Recovery options, and the absence of a package rollback
Upgrading by re-install or boot drive swap	The supported procedure for moving to new boot media
From the CLI	qs system-upgrade and the shell utilities

What the Upgrade Manager updates

Navigation: Storage Management → Storage Systems (section) → Storage System (toolbar group) → Upgrade Manager

It is also on the right-click context menu for a Storage System, as Upgrade Manager....

The Upgrade Manager manages four distinct classes of update. Which classes a run applies depends entirely on which boxes you tick in Advanced Settings -- with nothing ticked you get the first two.

Update class	What it contains	Selected by
Core QuantaStor packages	The core service, the web management interface, the REST server, and the base installer package, together with their dependencies	Always
Platform security updates	The security-relevant platform packages QuantaStor depends on, followed by a general platform package upgrade	Always, unless Exclude security updates is ticked
Kernel and drivers	The Linux kernel modules for the target kernel and the QuantaStor SCSI target driver package	Kernel + Driver Updates (REBOOT REQUIRED)
Ceph daemons	The Ceph packages on every member of a selected Ceph cluster, restarted daemon type by daemon type	Upgrade Ceph Cluster

Two consequences are worth knowing before you plan a window:

• Only the kernel and driver class needs a reboot. Core and security updates restart the QuantaStor services, which does not interrupt iSCSI, FC or NFS data access. New drivers are staged into place but are not loaded until the system restarts.
• The upgrade does not touch the underlying Ubuntu release. Moving between platform releases is a separate, much longer operation -- see qs-distupgrade. The Upgrade Manager never performs one.

Security patching cadence and the security notices behind it are covered on Security Updates; user, role and authentication settings are on Security Configuration.

The General tab

The General tab. Latest Version Information is what the configured repository offers; System Version Information is what each system is running now.

Select the systems to upgrade, then press Start Upgrade. The dialog opens on this tab and immediately runs an update check against the configured repository, showing Fetching latest package version information... while it works. Start Upgrade is disabled until that finishes. The check refreshes the platform package catalogue, so on a slow or distant repository it can take a minute or more.

Two grids sit on this tab, and they answer different questions.

• Latest Version Information -- a single row named Versions, showing what the repository the appliance is pointed at currently offers for Core, Kernel, Drivers and Ceph. This is the target, not something you select.
• System Version Information -- one row per Storage System, with the same four columns showing what that system is running now, and a checkbox to include it in the upgrade. Tick the header checkbox to select every system.

Comparing the two rows tells you what a run will actually change. Where a system's column already matches the Versions row, that class has nothing to install.

Note that the Core column reports the running service version, not the installed package version. The two normally agree, but they diverge on a system where the core package has been installed and the service has not yet restarted onto it. If a system looks behind after a successful upgrade, restart the service or reload the interface before concluding the upgrade failed.

Pressing Start Upgrade raises a confirmation listing the systems you selected. The wording differs by what you selected, and the kernel and driver case is explicit that reboots are involved: "Upgrading kernel, drivers or the OS distribution requires a system reboot. The following systems will automatically restart during the upgrade process." Nothing happens until you confirm.

Selecting no systems is rejected with a prompt to select a Storage System.

Advanced Settings

Advanced Settings in its default state. The Ceph field set is greyed out because this grid has no Ceph cluster; Allow Auto Reboots is greyed out until Kernel + Driver Updates is ticked.

Everything on this tab is optional. With nothing ticked, Start Upgrade applies core packages plus platform security updates and needs no reboot. The tab's own header repeats the recommendation that matters most: when executing advanced upgrades, we recommend reaching out to your support representative in order to avoid loss of service and ensure there is an adequate maintenance window.

Note that the systems are still chosen on the General tab. This tab only decides what is applied to them.

Kernel and Driver

• Kernel + Driver Updates (REBOOT REQUIRED) -- adds the kernel modules and the SCSI target driver package to the run. Tooltip: "Upgrade core packages along with the latest Kernel and Drivers." Ticking this forces Exclude security updates off and disables it, and disables the whole Ceph field set, because a Ceph daemon upgrade and a kernel upgrade cannot run in the same operation.
• Allow Auto Reboots -- greyed out until the box above is ticked. With it, a system that needs a restart to finish the upgrade reboots itself as soon as the packages are staged. Without it, the upgrade stops at the point a reboot is needed and parks the system in a warning state until you restart it yourself -- see Reboots and the WARNING state.

With Kernel + Driver Updates ticked, Allow Auto Reboots becomes available and Exclude security updates is forced off.

Ticking Kernel + Driver Updates when no kernel or driver update is available does nothing. The service checks the installed kernel and driver versions against what the repository offers, and if neither has moved it drops the kernel step and runs an ordinary core and security upgrade instead. No reboot is staged in that case. This is why a run you expected to require a restart sometimes completes without one.

Ceph

The whole field set is disabled when the grid contains no Ceph cluster, which is how it appears on a system that only serves scale-up storage.

• Upgrade Ceph Cluster -- tick it and choose a cluster from the combo box beside it. Tooltip: "Select a Ceph Cluster for the Ceph package upgrade." Ticking it selects every member of the chosen cluster in the General tab grid and then locks that grid, because a Ceph upgrade must cover the whole cluster. It also ticks and disables Exclude security updates, and disables the Kernel and Driver field set.
• Timeout Padding (min) -- enabled only while Upgrade Ceph Cluster is ticked. Default 0. Tooltip: "Add additional time before timeout in minutes. Useful when cluster has many OSDs and PGs." Each daemon restart phase has a timeout that already scales with the OSD count; this adds a flat margin on top for clusters where recovery and peering take longer than usual.
• Enable Alerts -- default on. Tooltip: "Check the box to have INFO level alerts generated at the end of each ceph daemon restart phase." On a long cluster upgrade those per-phase alerts are the main progress signal, so leaving it on is usually what you want.

A Ceph cluster upgrade is refused outright, before anything is applied, if any of the following is true. These are the common reasons a Ceph upgrade will not start:

• Not every member of the cluster is selected. All members must be in the run.
• The selected systems are not all members of the same Ceph cluster, or one of them is not a cluster member at all.
• The members are not all running an identical QuantaStor service version. Upgrade the cluster's QuantaStor packages first, then run the Ceph upgrade as a second operation.
• The cluster has MDS daemons but no Ceph filesystem. Create the filesystem or remove the surplus MDS daemons first.
• The appliances run an RHEL-family platform. Ceph daemon upgrades are supported on the Ubuntu-based platform only.

Exclude security updates and Force

• Exclude security updates -- restricts the run to the core QuantaStor packages, skipping both the security package set and the general platform package upgrade. It is greyed out whenever Kernel + Driver Updates is ticked, and forced on whenever Upgrade Ceph Cluster is ticked. Use it when you want the QuantaStor version moved and nothing else changed on the platform.
• Force (flushes package repository metadata and forces all package files to redownload) -- discards the cached repository metadata and downloaded package files before starting, so everything is fetched again. Use it when a previous attempt failed partway through, or when the repository content changed under a cached index. It makes the run considerably longer.

Where updates come from

Each appliance decides what is available from a single repository entry, /etc/apt/sources.list.d/osnexus.list. The update check reads that entry, refreshes the package catalogue from it, and reports the newest version each package has there. Nothing else determines what the Upgrade Manager offers -- there is no version selector in the dialog, so the channel the appliance points at is the choice of what it upgrades to.

Four channels are published:

Channel	Purpose
release	The default. Production.
optional	Add-on and optional packages, including patched platform packages. Enable it alongside a main channel, not instead of one.
rc	A staged stable build -- a release candidate, not yet generally available.
preview	Testing only. A preview of the next release; not for production use.

Read and change the selection with qs-apt-channel, which validates the result and rolls back if the new selection does not resolve. qs-apt-channel --status shows what is in effect and is the first thing to check when the Upgrade Manager offers an unexpected version, or offers nothing at all:

# qs-apt-channel --status
Current QuantaStor apt repositories (/etc/apt/sources.list.d/osnexus.list):
  release          deb http://packages.osnexus.com/packages quantastor-jammy main

Two things about this file are worth knowing:

• Only the first repository entry in it is used by the update check. If several are enabled, edit the selection with qs-apt-channel rather than by hand, so the entries and their pin priorities stay consistent.
• A companion pin file, /etc/apt/preferences.d/osnexus.pref, must be present. An upgrade started without it fails early with "APT pinning is not configured". qs-apt-channel maintains it; qs-util aptpin restores it if it has been removed.

An upgrade also refuses to start if the repository entry is missing or unreadable, with "Failed to retrieve package repository url, upgrade cannot proceed. Please verify your package repository is configured correctly."

Upgrading a Storage Grid

Select several systems in the General tab and the upgrade becomes a coordinated, serial operation rather than a parallel one. Grid membership itself is covered on Grid Configuration.

• The request is handled by the grid master. Whichever member you started from, a multi-system upgrade is handed to the grid master, which then drives the run. A Ceph cluster upgrade is handed to the cluster's manager node instead.
• The grid master upgrades first, then the remaining members one at a time. Nothing runs concurrently.
• The remaining members are ordered by how disruptive their upgrade is, least first: systems with no Storage Pools, then systems with pools but no HA groups, then systems in HA groups that are not currently hosting an active HA pool, and last the systems actually hosting active HA pools.
• Each member is followed by a short settling pause before the next one starts, so the grid can re-synchronise.
• A member that fails does not stop the run. The failure is logged and the coordinator moves to the next system, so after a grid upgrade check every member's version rather than only the task result.

Do grid members have to match versions?

For an ordinary upgrade, no. Members may run different QuantaStor versions, and a grid stays functional while an upgrade works through it -- which is what makes the serial, one-at-a-time staging usable. There is no requirement to bring the whole grid to the same version in one window.

For a Ceph cluster upgrade, yes, exactly. Every member of the cluster must be on an identical QuantaStor service version before the Ceph daemon upgrade will start. In practice that means two operations: upgrade the QuantaStor packages across the cluster first, confirm the Core column matches on every member, then run the Ceph upgrade.

Note that joining a grid is a different matter from upgrading one: a system below the minimum version for the grid it is joining is refused, with an error naming the version it needs. See Grid Configuration.

One upgrade at a time

A system will not start a second upgrade while one is still in progress; the attempt is refused with a message naming the system that is busy. The Health Check also reports Local Upgrade in Progress while one is running, and HA pool failover is blocked for the duration -- which is a good reason to keep the window tight and not leave a stalled upgrade sitting.

Reboots and the WARNING state

Core and security updates never need a restart. A kernel or driver upgrade always does, because the new modules are staged but not loaded.

When such an upgrade reaches the point where a restart is required and Allow Auto Reboots was not ticked, the system is deliberately parked rather than left looking healthy:

• Its state changes to WARNING, and its state detail is set to "System reboot required. Please reboot the system at your earliest convenience to finish upgrades."
• A WARNING-severity alert, Reboot After Upgrade Requested, is raised. See Alert Manager for where alerts are delivered.
• A Requires Reboot flag is set on the Storage System object, visible from the CLI:

qs system-get
...
                        State: Normal
                 State Detail:
              Service Version: 6.9.0
               Kernel Version: 6.8.0-90-generic
              Requires Reboot: false

The WARNING does not clear on its own. It persists across service restarts and is cleared only when the appliance actually reboots. A grid member in this state also reports "Please restart the appliance" to the rest of the grid. The Health Check reports System Requires Reboot, and raises it to critical where an HA pool is involved, because an appliance pending a driver restart is not a safe failover target.

With Allow Auto Reboots ticked, the state detail instead reads "System reboot required to finish upgrades, initiating system reboot now." and the appliance restarts itself. The upgrade task survives the reboot and resumes monitoring on the way back up, so it finishes on its own rather than needing to be restarted.

Watching an upgrade

The upgrade runs as a task, visible in the Tasks pane at the bottom of the interface, with a description that names the stage in progress -- updating the web management interface package, updating the core service package, applying security patches, updating the kernel and driver package, and so on.

Three things about the progress are worth expecting rather than worrying about:

• The web interface disconnects part way through. An INFO alert says so: "Core Services upgrading, please allow up to 3 minutes for the upgrade to complete. Web User Interface will disconnect part way through the upgrade." Clear your browser cache and reload once it finishes -- the interface is served from a package that has just changed underneath your session.
• Progress can appear to pause around the halfway mark. The upgrade deliberately runs in two passes, the second re-checking and completing anything the first pass could not, and the first pass ends at roughly 50%. A task sitting at 50% for a while is normal.
• The task is designed to resume. A QuantaStor upgrade restarts the core service more than once, and the task re-attaches to the running upgrade each time rather than being abandoned.

For anything the task does not explain, the upgrade writes /var/log/qs/qs_upgrade.log, and per-run progress files under /var/tmp/upgrade/. Both are collected by Send Support Logs, which is what OSNEXUS support will ask for.

Before you upgrade

Worth doing every time:

1. Schedule a maintenance window. The confirmation prompt says why: "Upgrades should be scheduled during a maintenance window as some protocol services may be momentarily interrupted."
2. Resolve existing faults first. A degraded Storage Pool, an unhealthy Ceph cluster or failing hardware is better fixed before an upgrade than during one.
3. Send a log report with Send Support Logs before you start. It gives support a known-good baseline to compare against, and it is the first thing they ask for if the upgrade goes wrong.
4. Check which channel the appliance is on with qs-apt-channel --status, and confirm the version in Latest Version Information is the one you intend to move to.
5. Confirm no upgrade is already in progress on any system you are about to select.
6. Run the Health Check and clear what it reports.

Additionally, for a kernel and driver upgrade:

1. Verify remote console access -- iDRAC, iLO, IPMI, CIMC or equivalent -- before you start. A driver or network change that goes wrong is recoverable from the console and often not from the network.
2. Check free space in /boot. The kernel step needs a few hundred megabytes there and fails cleanly rather than partway if it is short.
3. Plan the restart. Either tick Allow Auto Reboots or be ready to reboot each system yourself; a system left pending a restart sits in WARNING and cannot host an HA failover.

And for a Ceph cluster upgrade, confirm every member is on an identical QuantaStor service version and that the cluster is healthy, then select all of its members.

If an upgrade fails

Recovery is forward, not backward. QuantaStor has no automated package rollback: there is no mechanism that reinstalls the previous QuantaStor packages, and no snapshot of the previous install is taken before an upgrade begins. Nor does the upgrade take a configuration database backup of its own -- the backups you have are the periodic ones described on Recovery Manager. Plan on that basis rather than expecting to undo an upgrade.

What is available, in order of what to try:

1. Re-run the upgrade with Force. Most partial failures are a bad or interrupted download, and flushing the repository metadata and package files clears them. The second of the upgrade's two passes exists for the same reason.
2. Check the channel. If the Upgrade Manager is offering something unexpected, or an upgrade resolves to nothing, qs-apt-channel --status and qs-apt-channel --list will show why. Pointing back at release and re-running is a legitimate remedy.
3. Read /var/log/qs/qs_upgrade.log, then send a log report with Send Support Logs and contact OSNEXUS support at support@osnexus.com.
4. Restore the configuration database with the Recovery Manager if the configuration, rather than the packages, is what is damaged. That page covers what the database does and does not hold -- notably that your pools, volumes and shares do not depend on it.

One thing not to expect: the configuration database schema migrates forward on the first start of a new build, and that migration is one-way. Reverting to an older QuantaStor build against a database a newer build has already opened is not a supported recovery path. If you need the ability to return to the previous release, keep the old boot media, as in the procedure below.

Upgrading by re-install or boot drive swap

When jumping several major releases at once it can be faster to install the current release onto new boot media and restore the configuration afterwards, rather than upgrading in place. Keeping the old boot media also gives you a genuine roll-back: the previous system is intact on a drive you can put back.

These guidelines are provided for insight into the procedure and are intended to be carried out with OSNEXUS Support (support@osnexus.com) assistance.

QuantaStor migrates its internal database forward from any older version, so jumping several releases in one step is expected and supported.

Step 1. Back up the configuration

Send a log report for every server to be upgraded, with Send Support Logs. This lets the support team check the hardware configuration and flag anything that could complicate the upgrade.

Then, on each host, archive these directories to a share elsewhere on the network:

/etc/
/var/opt/osnexus/
/var/lib/ceph/

Step 2. Log review and health check

Address degraded pools and degraded Ceph clusters, and replace faulty hardware, before upgrading.

Step 3. Document and verify the network configuration

The new install starts with a default network configuration, so you need the old one written down before you begin.

Per server: hostname, DNS addresses, NTP addresses, domain suffix, and the ports used for cluster heartbeat.

Per port: IP address, MAC address, subnet mask, MTU, gateway, parent and child ports for VIFs, VLANs and bonded ports, and pool association for HA and site VIFs.

Step 4. Verify remote console access

The procedure involves multiple reboots, and a network configuration problem leaves the console as the only way in. Verify iDRAC, iLO, IPMI, CIMC or your equivalent access method before starting the install.

Step 5. Power off

Power off both systems in a cluster pair -- this needs a maintenance window. Power off any attached JBODs as well and leave them off until later.

Step 6. Replace the boot media

QuantaStor boots from a mirrored SSD system device. Remove the old media, install the new, and create a new RAID1 mirror in the controller BIOS. Use two datacenter-grade SATA SSDs of 200GB or more. Avoid SATADOM boot media, which is not reliable enough for this role.

Step 7. Install the current release

Always install the latest release. The current ISO is at osnexus.com/downloads and works for both network installs and USB media. Do one system at a time; the pool import is the last step.

Step 8. Re-apply the network settings

Applying network configuration needs a license, so apply a temporary Trial Edition key first -- available from the form on the osnexus.com/downloads page. Then re-apply everything from Step 3 except the HA cluster VIFs, which come later.

• Storage System Modify -- restore the hostname from the installer default back to its original name.
• Storage System Modify -- apply the NTP server settings.
• Storage System Modify -- apply the DNS server settings.
• Network Ports Modify -- restore every port to its configuration from Step 3.

Step 9. Restore the configuration database

Restore the QuantaStor internal database with the Recovery Manager, which is the supported route and finds the backups on the imported pools for you.

Then clear the stale cluster configuration. HA cluster data does not carry forward between major releases and has to be rebuilt, so it must be cleared out of the restored database. Log in at the console or over SSH as qadmin and run:

sudo -i
systemctl stop quantastor
cd /opt/osnexus/quantastor/bin
./qs_service --reset-pool-ha --reset-cluster
systemctl start quantastor

The internal database itself lives at /var/opt/osnexus/quantastor/osn.db; Recovery Manager documents restoring it by hand if the interface is unavailable.

Step 10. Verify the configuration

Log in to the web interface and check the configuration. It is normal at this stage for pools to be missing and for various warnings to be present, since the JBODs are still powered off, and for other servers to show as disconnected because they are still off. Both are safe to ignore for now.

Step 11. Power on JBODs, reboot, import pools

Power on the JBODs, then reboot the system you have just rebuilt. Import and start the pool if it has not come up on its own. Verify access to shares and volumes, verify the network settings, and run the Configuration Analyzer for any configuration issues worth addressing.

Step 12. Repeat for the remaining systems

Power the rebuilt server and its JBODs back off and repeat Steps 6 through 10 on the other server or servers in the HA cluster, until every system is rebuilt.

Step 13. Power everything on and import pools

Power on all systems and confirm that the members are communicating with each other in the grid. See Grid Configuration if a member stays disconnected.

Step 14. Re-create the site cluster and heartbeat rings

Step 9 cleared the HA cluster configuration, so rebuild it:

• Create the site cluster in High-availability VIF Management.
• Add a second cluster heartbeat ring to the site cluster.
• Create an HA group on each HA Storage Pool.
• Re-create the VIFs for each HA Storage Pool from the information collected in Step 3.

See HA Cluster Setup (JBODs).

Step 15. Verification and testing

• Fail each pool over to confirm both systems can reach the back-end devices.
• Verify that client systems and users can reach every Storage Volume and Network Share.
• Send a log report again for final review by OSNEXUS Support.

Fixing grid communication after an upgrade

An upgrade that changes management addressing can leave grid members unable to find each other, because each member's copy of the grid database still holds the old address for the others. The general fix is to pin grid traffic to a chosen interface with Use Preferred Grid Port IP on Storage System Modify, choosing the same subnet on every member; Grid Configuration covers members that show offline more broadly.

Where the addresses have changed wholesale and the members cannot link up to synchronise the correction, the addresses can be overridden on disk from the grid master. This avoids deleting and re-creating the grid, which is disruptive when clusters are configured.

Log in to the grid master over SSH or the console, and list the grid to get each member's UUID:

qs system-list

Then write one override file per remote member, named for that member's UUID, containing the address to reach it on:

sudo -i
echo "10.0.10.11" > /var/opt/osnexus/quantastor/<member-uuid>.gpp

Once the grid has synchronised and the members hold accurate addresses again, remove the override files -- they are a temporary measure, not a configuration setting:

ls /var/opt/osnexus/quantastor/*.gpp
rm /var/opt/osnexus/quantastor/*.gpp

From the CLI

The same operation is available as qs system-upgrade (alias qs upgrade). The arguments map directly onto the dialog:

Argument	Dialog equivalent
--system-list	The systems ticked in System Version Information. Required -- the command refuses to run without it.
--include-kernel	Kernel + Driver Updates (REBOOT REQUIRED)
--allow-reboot	Allow Auto Reboots
--core-only	Exclude security updates
--ceph-upgrade	Upgrade Ceph Cluster. Implies --core-only=true, so you do not need to pass both.
--timeout-padding-sec	Timeout Padding, in seconds here rather than the dialog's minutes.
--distro-upgrade	No equivalent. Not offered in the dialog; use qs-distupgrade for a platform release change.
--flags	force is the Force checkbox.

Every argument takes an explicit value, so write --include-kernel=true rather than a bare flag. Upgrading the core packages on two systems, taking kernel and drivers, and rebooting without prompting:

qs system-upgrade --system-list=qs-node1,qs-node2 --include-kernel=true --allow-reboot=true

The CLI always returns immediately; it does not block until the upgrade finishes. Follow the task it creates, or watch qs system-get for the version and the Requires Reboot flag.

There is no CLI command for the update check. The available-version query the dialog runs when it opens is not exposed as a qs command, so to see what a system would upgrade to from a shell, compare qs system-list against the repository.

Below the API, the appliance ships shell utilities that the Upgrade Manager itself drives. They are documented on QuantaStor Shell Utilities rather than here:

• qs-upgrade -- the upgrade driver the service invokes. Running it directly is equivalent to starting an upgrade from the interface.
• qs-apt-channel -- select the repository channel, with validation and rollback.
• qs-distupgrade -- upgrade the underlying Ubuntu release. Contact support and reserve a long window first.

Related pages

• Storage System -- system-level configuration, renaming, and the preferred grid port setting
• Grid Configuration -- grid membership, the grid master, and members that show offline
• Recovery Manager -- restoring the configuration database, used by the re-install procedure above
• QuantaStor Shell Utilities -- qs-upgrade, qs-apt-channel and qs-distupgrade in detail
• Security Updates -- security patch cadence and the security notices behind it
• Security Configuration -- users, roles and authentication
• Send System Log Report -- sending a log report before and after an upgrade
• Alert Manager -- where the upgrade and reboot-required alerts are delivered
• QuantaStor Version ChangeLog -- what changed in each release

---

Verified against QuantaStor 6.9.0.