Guides:Durability vs Availability, Designing Storage for Reliability and Uptime
By Steve Umbehocker, CTO, OSNexus · Updated October 31, 2026
Durability and availability are two different reliability promises. Durability is the probability that data written to a storage pool is never lost. Availability is the fraction of time clients can actually reach that data. A storage layout can be extremely durable and still be unavailable during a power or network outage, and a highly available cluster can still lose data if its RAID layout is too thin. QuantaStor gives you direct control over durability through the pool's RAID layout, and it gives you the building blocks for availability: HA failover groups, dual heartbeat rings, enclosure-redundant pools and multipath JBOD cabling. This guide explains why OSNexus quotes nines of durability but not nines of availability, shows the Markov model behind the durability numbers, and compares five layouts for an 84-drive pool of 24 TB NL-SAS drives.
Why it matters
Buyers ask "how many nines does it have?" and usually mean two things at once: will I lose data, and will my applications stop. Those questions have different answers and different owners.
Availability depends on the whole environment. Whether clients keep access depends on much more than the storage software: network redundancy (LACP bonds, redundant switches), network services such as firewalls, NTP and DNS, directory services such as Active Directory, correct cabling, other equipment on the network, cooling, power stability, humidity, server stability and firmware maintenance. None of those are under the storage vendor's control, so OSNexus does not publish a nines-of-availability figure for QuantaStor. Any such number would be a statement about your data center, not the product.
Durability can be calculated. Given a drive failure rate, a rebuild time and a RAID layout, the probability of losing data in a year follows from a standard Markov model. That makes durability a design choice: pick the layout whose nines meet your organization's requirement.
How it works
Durability: the Markov model
Each RAID group (a mirror pair, or one RAIDZ parity group) is modeled as a Markov chain. State k means k drives in the group have failed and are not yet rebuilt. Drives fail at rate λ each, and a failed drive is rebuilt at rate μ. A group of n drives that tolerates m failures moves from state k to k+1 at rate (n−k)λ and back at rate μ; reaching state m+1 means data loss.
When rebuilds are much faster than failures (μ ≫ λ), the mean time to data loss (MTTDL) of the chain reduces to a closed form, and the pool, a stripe of G independent groups, loses data if any group does:
lambda = AFR / 8760 failures per drive-hour mu = 1 / MTTR rebuilds per hour MTTDL_group = mu^m / ( n (n-1) ... (n-m) * lambda^(m+1) ) hours P_loss_year ~= G * 8760 / MTTDL_group (pool of G groups) nines = -log10( P_loss_year )
The model assumes independent, exponentially distributed failures, the basis of mean time between failures figures and of the annualized failure rate. Read the exponential distribution article for the underlying math.
Availability: removing single points of failure
Within the storage system, availability comes from making sure no single component stops the pool:
- HA failover groups. In a shared-JBOD pair, two servers are cabled to the same enclosures and a storage pool HA failover group lets the partner import the pool if the owner fails. Client traffic follows the pool on a virtual interface (High-availability VIF Management). Hardware setup is in HA Cluster Setup (JBODs).
- Dual heartbeat rings. Two cluster rings on separate networks (Site Add Cluster Ring) keep a single switch or cable failure from looking like a node failure.
- Enclosure-redundant pools. With several JBODs, pool creation distributes each RAID group's drives round-robin across enclosures where the layout allows it. An enclosure outage then costs each group at most as many drives as it can tolerate, so the pool runs degraded instead of going offline.
- Redundant paths. Dual SAS paths to each JBOD (Multipath IO Configuration) and LACP bonds on the front end (Changing Network Bonding Mode) turn cable and port failures into path events.
Design and sizing
The table applies the model to 84 × 24 TB NL-SAS drives (2,016 TB raw). Assumptions: a conservative field AFR of 1% per drive (λ ≈ 1.14 × 10⁻⁶ per hour) and a 48-hour mean time to restore redundancy (detection, hot-spare activation and resilver of a 24 TB drive under load), so μ/λ = 18,250.
| Layout | Group (n, tolerates m) | Groups × drives | Usable (before ZFS overhead) | Annual data-loss probability | Durability |
|---|---|---|---|---|---|
| RAID1 mirror | 2, 1 | 42 × 2 | 1,008 TB (50%) | 4.6 × 10⁻⁵ | ~4.3 nines |
| RAIDZ2 4d+2p | 6, 2 | 14 × 6 | 1,344 TB (67%) | 5.0 × 10⁻⁸ | ~7.3 nines |
| RAIDZ2 8d+2p | 10, 2 | 8 × 10 (+4 spares) | 1,536 TB (76%) | 1.7 × 10⁻⁷ | ~6.8 nines |
| RAIDZ3 9d+3p | 12, 3 | 7 × 12 | 1,512 TB (75%) | 1.4 × 10⁻¹⁰ | ~9.9 nines |
| RAIDZ3 18d+3p | 21, 3 | 4 × 21 | 1,728 TB (86%) | 9.5 × 10⁻¹⁰ | ~9.0 nines |
How to read it:
- Parity count dominates. Each extra parity drive multiplies MTTDL by roughly μ/λ, which is four orders of magnitude here. Two-way mirrors give the best small-block performance but the lowest durability of the five.
- Width costs less than parity buys. Going from 9d+3p to 18d+3p gains 11 points of usable capacity for about one nine. In practice wider groups also rebuild more slowly, so treat the wide layouts' figures as optimistic.
- Rebuild time matters. Halving MTTR to 24 hours adds about 0.3 nines per tolerated failure: +0.3 for mirrors, +0.6 for RAIDZ2 and +0.9 for RAIDZ3. Hot spares shorten the time to start a rebuild.
- The model is a floor for planning, not a guarantee. It ignores correlated failures (a bad drive batch, an enclosure outage) and latent sector errors found during a rebuild. Regular scrubs reduce the latter, and enclosure-redundant placement reduces the former.
For availability, follow the brief's rule of thumb: stripe the pool across multiple JBODs, configure dual heartbeat rings, and size servers slightly above the workload so they are not running continuously at their limits.
Setting it up
- Plan the layout from the table and your durability target. For example, a 99.9999999% (nine nines) requirement on this hardware points to RAIDZ3.
- Cable each server to every JBOD with dual SAS paths, and create LACP bonds for client networks.
- Create the pool, for example
pool1, under Storage Management → Storage Pools → Create. Choose the RAID Type and Set Size that match the planned layout and select drives from all JBODs so the groups can be spread across enclosures. See Storage Pools.
- Mark hot spares and set a scrub schedule (Storage Pool Scrub Schedule Update).
- For HA, create the site cluster with two heartbeat rings, then create the HA failover group for
pool1and a client VIF, for example 192.0.2.50.
Operating and testing
Durability only holds if failed drives are replaced quickly, so treat a degraded pool as urgent: the model's numbers assume the 48-hour restore window. Keep scrubs running so latent errors surface while redundancy is intact.
Availability has to be tested in your environment, because that is where most of the risk lives. Before production, and after network or firmware changes, check that the HA group shows Normal, then pull a front-end cable, a SAS cable and one heartbeat network in turn, and run a manual failover while a client runs I/O.
Record the results alongside the environment's own dependencies (DNS, NTP, Active Directory, power and cooling) so the whole availability chain is documented.
FAQ
How many nines of availability does QuantaStor have?
OSNexus does not quote nines of availability, because uptime depends on factors outside the storage software: network redundancy, firewalls, DNS, NTP, Active Directory, cabling, power, cooling and firmware maintenance. QuantaStor provides HA failover groups, dual heartbeat rings, enclosure-redundant pools and multipath to remove storage-side single points of failure. The availability you achieve is a property of the whole deployment.
How many nines of durability does it have?
It depends on the layout you choose. For 84 × 24 TB NL-SAS drives at a 1% AFR and a 48-hour restore window, the Markov model gives about 4.3 nines for mirrors, 6.8 to 7.3 nines for RAIDZ2 layouts and 9 to 9.9 nines for RAIDZ3 layouts.
Why do 4d+2p groups score higher than 8d+2p?
Both tolerate two failures, but a 10-drive group has more drives that can fail during a rebuild. The product n(n−1)(n−2) is six times larger, which outweighs having fewer groups in the pool.
Does an HA pair improve durability?
No. Both nodes share the same drives, so durability comes from the RAID layout alone. HA improves availability by letting the partner import the pool when a server fails; use remote replication for a second copy of the data.
What happens if a whole JBOD fails?
If the pool was built across enough enclosures that each RAID group has no more drives in one JBOD than it can tolerate, the pool continues in a degraded state. Replace or restore the enclosure promptly, since the pool has less redundancy until it resilvers.
Part of the QuantaStor Guides series. For reference documentation, see the QuantaStor documentation.