Hardware Compatibility List (HCL): Difference between revisions

From OSNEXUS Online Documentation Site
Jump to navigation Jump to search
← Older edit
VisualWikitext
 
(193 intermediate revisions by one other user not shown)
Line 1: Line 1:
=  QuantaStor Hardware Compatibility List (HCL) =
[[index.php?title=Category:Design guide]]
QuantaStor is Linux based and has kernel driver updates released multiple times per year to ensure broad support for the latest hardware from the major vendors including HPE, Dell/EMC, Cisco, Intel, Seagate, Supermicro, Gigabyte and Lenovo. The hardware listed below has been tested for specific compatibility with QuantaStor to ensure system stability, performance and compatibility. If you have questions about specific hardware compatibility for components not listed below please contact support@osnexus.com for assistance.


QuantaStor is Linux based and has kernel driver updates released multiple times per year to ensure broad support for the latest hardware from the major vendors including HPE, Dell/EMC, Cisco, Intel, Western Digital, Supermicro and Lenovo.  The hardware listed below has been tested for specific compatibility with QuantaStor to ensure system stability, performance and compatibility. If you have questions about specific hardware compatibility for components not listed below please contact support@osnexus.com for assistance.
== Servers for QuantaStor SDS Systems ==
 
== Servers for QuantaStor SDS Appliances ==


QuantaStor supports the broad spectrum of AMD and Intel x86 64bit servers on the market from major vendors.  Special configuration files are used to integrate new servers and storage enclosures and there are configuration files for monitoring media, thermals, power supply and fan health.  Although QuantaStor comes with configuration settings for most major server models in some cases we may need to update the configuration files at the time of deployment if your selected server model is not one we have default settings for.


{| class="wikitable"
{| class="wikitable"
Line 11: Line 11:
! Boot Controller
! Boot Controller
! CPU
! CPU
! Memory (48GB min)
! Memory (32GB min)
|-
|-
| Cisco
| Cisco
| [https://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-s3260-storage-server/index.html UCS S3260 Storage Server], C240, C3260 (M3, M4, M5 Series)
| C220, C240 (M8/M7/M6/M5/M4 Series)
| UCS RAID and HBAs
| Software RAID1 M.2 NVMe
| Dual Xeon Silver, Gold, or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
|-
| DellEMC
| DellEMC
| [https://www.dell.com/en-us/work/shop/povw/poweredge-r740 PowerEdge R740/R740xd], R730/R730xd/R720/R630/R620
| Gen 7 (eg. R770/R77x5/R670/R67x5), Gen 6 (eg. R760/R76x5/R660/R66x5), Gen5 (eg. R750), Gen4 (eg. R740), Gen3 (eg. R730)
| Dell PERC H8xx, H7xx (OEM MegaRAID)
| Software RAID1 M.2 NVMe or Dell BOSS
| Dual Xeon Silver, Gold or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
| Gigabyte
| Intel and AMD servers
| Software RAID1 M.2 NVMe
| AMD EPYC, Intel Scalable Processors
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
|-
| HPE
| HPE
| DL360/DL380 Gen10, Gen9, Gen8, Gen7
| HP DLxxx Servers (Gen11, Gen10, Gen9, Gen8, Gen7)
| HPE P4xx w/ FBWC
| Software RAID1 M.2 NVMe or HPE P4xx w/ FBWC
| Dual Xeon Silver, Gold Scalable Processors or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
|-
| HPE
| HPE
| Apollo 4200/4510 Gen9 and Gen10 Series
| HP Apollo Servers (Gen9, Gen10 and Gen11)
| HPE P4xx w/ FBWC
| Software RAID1 M.2 NVMe or HPE P4xx w/ FBWC
| Dual Xeon Silver, Gold Scalable Processors or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
|-
| Intel
| MiTAC
| [https://www.intel.com/content/www/us/en/products/servers/server-chassis-systems/server-board-s2600wf-systems.html S2600WF Based Server Systems]
| Intel Server Systems (M50FCP/M50CYP)
| Intel OEM Broadcom/LSI
| Software RAID1 M.2 NVMe
| Dual Xeon Silver, Gold Scalable Processors or E5-2620v3/v4 or faster
| Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
|-
| Lenovo
| Lenovo
| [https://www.lenovo.com/us/en/data-center/servers/racks/c/racks ThinkSystem SR650/SR550/SR590 Series], x3650 M5/M4/M4 BD, x3250 M5
| SRxxx V3 & V4 Rack Servers
| ServeRAID M5xxx (OEM MegaRAID)
| Software RAID1 M.2 NVMe or Hardware Boot RAID1
| Dual Xeon Silver, Gold Scalable Processors or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
| Seagate
| [https://www.seagate.com/products/storage/data-storage-systems/application-platforms/exos-ap-2u24/ Seagate 2U24 AP Storage Server], 5U84 AP, 2U12 AP (AMD / Bonneville)
| Internal M.2 NVMe
| AMD EPYC 16 core
| 128GB RAM (Gold) 256GB RAM (Platinum), 512GB RAM (Titanium)
|-
|-
| Supermicro
| Supermicro
| All X11, X10 & X9 based servers
| X14/H14, X13/H13, X12/H12, X11, X10 & X9 based Intel and AMD servers
| SMC AOC (OEM MegaRAID)
| Software RAID1 M.2 NVMe
| Dual Xeon Silver, Gold Scalable Processors or E5-2620v3/v4 or faster
| AMD EPYC, Intel Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
| 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|-
| Western Digital
| Ultrastar Serv24 and Serv24HA (NVMe systems)
| Internal controller
| Dual Xeon Silver, Gold Scalable Processors
| 0.5GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
|}
|}


== Disk Expansion Chassis / JBOD ==  
== Disk Expansion Chassis / JBOD ==  


* QuantaStor supports SAS/SATA & NVME external Expansion Shelves / JBOD devices from Dell, HP, IBM/Lenovo and SuperMicro.
* QuantaStor Scale-Up
* All JBOD must have two or more SAS expansion ports per module to allow for SAS multipathing.  For chaining of expansion shelves three or more SAS ports are required per expander.
** supports the use of SAS/NL-SAS media JBODs from most vendors
** support external NVMe-oF JBOF from Western Digital (Data24 42xx series)
** all JBODs must be dual-expander / dual-controller / dual-IO module (naming conventions vary by vendor)
* QuantaStor Scale-Out
** supports the use of SAS/NL-SAS and SATA JBODs from most vendors
** support external NVMe-oF JBOF from Western Digital (Data24 41xx series)
** supports JBODs that are both single and dual-expander but we recommend dual-IO module / dual-expanders here too


=== Supported Vendors ===
{| class="wikitable"
{| class="wikitable"
! Vendor
! Vendor
Line 73: Line 86:
| Cisco
| Cisco
| All Models
| All Models
|-
| Dell
| [https://www.dell.com/en-us/work/shop/productdetailstxn/storage-md1420 All Models]
|-
|HighPoint
|RocketStor 4243AS
|-
| HPE
| All Models ([https://www.hpe.com/us/en/product-catalog/storage/disk-enclosures/pip.hpe-d3000-disk-enclosures.6923837.html D3000 Series], D2000 Series, [https://www.hpe.com/us/en/product-catalog/storage/disk-enclosures/pip.hpe-d6020-enclosure-with-dual-io-modules.1009000694.html D6020])
|-
|-
| IBM/Lenovo
| IBM/Lenovo
| All Models
| All Models
|-
|-
| Dell
| Seagate Exos E
| [https://www.seagate.com/enterprise-storage/systems/exos/?utm_source=eol&utm_medium=redirect&utm_campaign=modular-enclosures All Models]
|-
| Seagate Corvault
| All Models
| All Models
|-
|-
| HGST
| Seagate Exos X
| Data60 / Data102 / 4U60-G2 / 2U24 SSD Models
|-
| Seagate
| 4U106 / 5U84 / 2U24 SSD Models
|-
| HPE
| All Models
| All Models
|-
|-
| SuperMicro
| SuperMicro
| All Models (HA requires dual expander [E26, E2C])
| All Models (HA requires dual expander JBODs)
|-
| Western Digital
| All Models (including Data60 / Data102 / Data24)
|}
|}


== SAS HBAs & SAS/SATA RAID Controllers ==
== SAS HBA Controllers ==
 
To make Storage Pools highly-available (ZFS based pools) media must be connected to two or more servers via SAS (or FC) connectivity.  We recommend Broadcom HBAs and their OEM equivalents.  HPE is the exception where HPE disk expansion units must be used with HPE HBAs which are now Microsemi based.  Make sure the HBA is flashed with the latest firmware to ensure compatibility with the latest storage media.  For OEM branded HBAs always use the OEM's supplied firmware even if a newer firmware is available from Broadcom. SAS HBAs with only external ports (8e, 16e models) should be configured to ignore Boot ROM from Disk Devices. For Broadcom/LSI Controllers, this option is called 'Boot Support' and should be set to 'OS only' in the MPTSAS BIOS.


{| class="wikitable"
{| class="wikitable"
Line 99: Line 123:
! Model
! Model
! Type
! Type
! Integrated Management
! QS HW Mgmt Module
|-
| Broadcom/LSI/Avago
| 9600 24Gb tri-mode series
| SAS/NVMe HBA
| Yes
|-
|-
| Adaptec
| Broadcom/LSI/Avago
| 5xxx/6xxx/7xxx/8xxx
| 9500 12Gb tri-mode series
| SATA/SAS RAID
| SAS HBA
| Yes
| Yes
|-
|-
Line 116: Line 145:
| Yes
| Yes
|-
|-
| Broadcom/LSI/Avago
| Cisco
| 9200 6Gb series
| UCS 6Gb & 12Gb HBAs
| SAS HBA
| Yes
|-
| DELL/EMC
| SAS HBA 6Gb & 12Gb
| SAS HBA
| Yes
|-
| HPE
| SmartArray H241 12Gb
| SAS HBA
| Yes
|-
| Lenovo/IBM
| ServeRAID M5xxx 12Gb
| SAS HBA
| SAS HBA
| Yes
| Yes
|}
== Boot RAID Controllers ==
QuantaStor supports both software RAID1 for boot and hardware RAID1 for boot using high quality datacenter grade media.  We recommend SSDs in the 240GB to 480GB size range for boot.  If SATADOM SSD devices are used use the 128GB devices and use software RAID1 to mirror them.  Do not use Intel VROC but vendor specific dedicated boot controllers like the Dell BOSS are generally supported.
{| class="wikitable"
! Vendor
! Model
! Type
! QS HW Mgmt Module
|-
|-
| Broadcom LSI/Avago
| Broadcom/LSI/Avago
| MegaRAID (all models)
| MegaRAID (all models)
| SATA/SAS RAID
| SATA/SAS RAID
Line 131: Line 186:
| Yes
| Yes
|-
|-
| Cisco
| DellEMC
| UCS 6Gb & 12Gb HBAs
| SAS HBA
| Yes
|-
| DELL/EMC
| PERC H7xx, H8xx 6Gb & 12Gb models
| PERC H7xx, H8xx 6Gb & 12Gb models
| SATA/SAS RAID
| SATA/SAS RAID
| Yes
| Yes
|-
|-
| DELL/EMC
| DellEMC
| SAS HBA 6Gb & 12Gb
| BOSS
| SAS HBA
| SATA SSD RAID1
| Yes
| Yes
|-
|-
Line 151: Line 201:
| Yes
| Yes
|-
|-
| HPE
| Lenovo/IBM
| SmartArray H241 12Gb HBA
| ServeRAID M5xxx
| SAS HBA
| SATA/SAS RAID
| Yes
| Yes
|-
|-
| Lenovo/IBM
| Microsemi/Adaptec
| ServeRAID M5xxx
| 5xxx/6xxx/7xxx/8xxx
| SATA/SAS
| SATA/SAS RAID
| Yes
| Yes
|}
|}


* External SAS HBA's should be configured to ignore Boot ROM from Disk Devices. For LSI Controllers, this option is called 'Boot Support' and should be set to 'OS only' in the MPTSAS BIOS.
== NVMe RAID Controllers ==
 
QuantaStor supports the use of NVMe RAID controllers for scale-out configurations and single node configurations.  They are not supported for use with scale-up HA (high-availability) configurations.
 
=== Single Node Use Cases ===
* High performance SAN/NAS using software single parity (RAIDZ1) over NVMe hardware RAID5.
* High performance NVMeoF/FC/iSCSI target passthru of logical drives from NVMe hardware RAID5
 
=== Scale-out Use Cases ===
* High performance storage for WAL/MDB
* High performance OSDs with local rebuild capability, variable OSD count for higher performance


== HDD Disk Devices ==
=== Supported Vendors ===


{| class="wikitable"
{| class="wikitable"
! Vendor
! Vendor
! Model
! Model
! Type
! QS HW Mgmt Module
|-
|-
| Hitachi (HGST) UltraStar
| [https://www.graidtech.com/how-it-works/ Graid Technology Inc.]
| 10K SAS, NL-SAS & Enterprise SATA Drives
| [https://www.graidtech.com/product/sr-1000/ SupremeRAID™ SR-1000]
| NVMe RAID
| Yes
|-
|-
| Seagate Enterprise Capacity/Enterprise Performance
| [https://www.graidtech.com/how-it-works/ Graid Technology Inc.]
| 10K SAS, NL-SAS & Enterprise SATA Drives
| [https://www.graidtech.com/product/sr-1010/ SupremeRAID™ SR-1010]
| NVMe RAID
| Yes
|-
|-
| Western Digital RE(RAID Edition)
| [https://pliops.com/ Pliops]
| 10K SAS, NL-SAS & Enterprise SATA Drives
| [https://pliops.com/raidplus/ XDP-RAIDplus]
| NVMe RAID
| No
|-
|-
| Toshiba Enterprise Capacity/Enterprise Performance
| [https://xinnor.io/ Xinnor]
| 10K SAS, NL-SAS & Enterprise SATA Drives
| [https://xinnor.io/what-is-xiraid/ xiRAID]
| NVMe RAID
| Yes
|}
|}


== SAN/NAS SSD Journal Devices ==
== Storage Devices/Media ==
 
=== Scale-up HA Cluster Media (ZFS based Storage Pools) ===
 
Scale-up clusters require dual-ported media so that QuantaStor's IO fencing system can reserve device access to specific nodes for specific pools. These clusters generally consist of two QuantaStor servers with shared access to the storage media in a separate enclosure (JBOD or JBOF).  Scale-up clusters using dual-ported NVMe media require the use of special NVMe based Storage-Bridge-Bay servers from Supermicro or the use of Western Digital Data24 42xx JBOFs.


A mirrored pair of SSDs are required for each storage pool to enable SSD write acceleration.  SSDs can be added or removed from a given storage at any time with zero downtime.  Be sure to select a storage chassis / JBOD with adequate drive slots for both the data drives and the SSD journal devices. Select 400GB models when purchasing write-intensive SSDs and 800GB models when purchasing mixed-used SSDs.
==== SAS/NL-SAS Media ====


{| class="wikitable"
{| class="wikitable"
! Vendor
! Vendor
! Model
! Model
! Type
! Media Type
! Use Case
! Notes
|-
|-
| Western Digital
| Western Digital
| Ultrastar SS300 Write-Intensive & Mixed-use Models
| Ultrastar NL-SAS HDDs
| 12Gb SAS
| Dual-port 6/12/24Gb SAS SSD & NL-SAS HDD
| ZFS ZIL (2000MB/sec)
|-
|-
| Western Digital
| Seagate
| Ultrastar SS200 Mixed Use Models
| Exos NL-SAS HDDs, Nytro & Enterprise Performance SAS SSDs
| 12Gb SAS
| Dual-port 12/24Gb SAS SSD & NL-SAS HDD
| ZFS ZIL (1000MB/sec)
|-
|-
| Western Digital
| Samsung
| Ultrastar SAS SSD HUSMH Series
| Enterprise SAS SSDs
| 12Gb SAS
| Dual-port 12/24Gb SAS SSD
| ZFS ZIL (765MB/sec)
|-
|-
| Western Digital
| KIOXIA
| Ultrastar SAS SSD HUSMM Series
| PM Enterprise Capacity/Enterprise Performance SAS SSD
| 12Gb SAS
| Dual-port 12/24Gb SAS SSD
| ZFS ZIL (765MB/sec)
|-
|-
| Micron
| Cisco, HPE, DellEMC, Lenovo
| S650DC Series MTFDJAK400MBW-2AN1ZAB
| OEM SAS SSD
| 12Gb SAS
| Dual-port 6/12/24Gb SAS SSD & NL-SAS HDD
| ZFS ZIL (850MB/sec)
|-
|-
| Micron
| S655DC Series MTFDJAK400MBW-2AN1ZAB
| 12Gb SAS
| ZFS ZIL (850MB/sec)
|}
|}


== Object Storage SSD Journal Devices ==
==== Enterprise NVMe Media ====
 
We highly recommend the use of NVMe devices vs SATA/SAS SSDs due to the higher throughput performance per device which reduces overall costs.


{| class="wikitable"
{| class="wikitable"
! Vendor
! Vendor
! Model
! Model
! Type
! Media Type
! Use case
! Notes
|-
| HGST
| Ultrastar DC SN200, SN620
| NVMe
| Ceph Journal / WAL
|-
| Intel
| DC All Models
| NVMe
| Ceph Journal / WAL
|-
|-
| Intel
| Samsung
| Optane 100GB / 200GB
| Enterprise NVMe SSDs
| NVMe
| Dual-port NVMe
| Ceph Journal / WAL
| Supermicro SBB or WD Data24 4200
|-
|-
| Micron
| KIOXIA
| 9200 PRO / MAX Series
| [https://americas.kioxia.com/en-us/business/ssd/enterprise-ssd/cm7-v.html CM Enterprise]
| NVMe
| Dual-port NVMe
| Ceph Journal / WAL
| Supermicro SBB or WD Data24 4200
|-
|-
| Techman
| SanDisk
| XC 100 Series
| [https://www.sandisk.com/products/ssd/internal-ssd/sandisk-sn655-nvme-ssd?sku=0TS2458 SN655] NVMe models with dual-port and IO fencing
| NVMe
| Dual-port NVMe
| Ceph Journal / WAL
| Supermicro SBB or WD Data24 4200
|}
|}
==== Checking for NVMe Persistent Reservation Support ====
This command shows how to get the information on what modes of IO fencing are supported.  What we're looking for is '''"[4:4] : 0    Exclusive Access - Registrants Only"''' but as you can see in the output mode below that is not supported on this media (Sandisk SN655)
<pre>
# nvme id-ns -H /dev/nvme6n1 | grep -A9 rescap
rescap  : 0
  [7:7] : 0    Ignore Existing Key - Used as defined in revision 1.2.1 or earlier
  [6:6] : 0    Exclusive Access - All Registrants Not Supported
  [5:5] : 0    Write Exclusive - All Registrants Not Supported
  [4:4] : 0    Exclusive Access - Registrants Only Not Supported
  [3:3] : 0    Write Exclusive - Registrants Only Not Supported
  [2:2] : 0    Exclusive Access Not Supported
  [1:1] : 0    Write Exclusive Not Supported
  [0:0] : 0    Persist Through Power Loss Not Supported
</pre>
==== Journal Media (Metadata-Offload, ZIL & L2ARC) ====
* Metadata-offload (special VDEV) is highly recommended for all HDD based storage pools.  Add 3x flash devices that in sum represents 1% of the HDD raw capacity.  For example a 1000TB pool would require 10TB of raw flash so 3x 3.84TB flash devices is ideal.
* ZIL/SLOG is not recommended in most modern configurations.  It is used to efficiently handle synchronous writes (SYNC_IO) common to databases and VMs.  We recommend using all-flash pools for these workloads.  If you do have a need for ZIL add 2x flash devices such as 2x 3.84TB devices.
* L2ARC is not recommended in most modern configurations as the benefits of L2ARC are largely covered by the Metadata-offload (special VDEV).  If you do elect to add L2ARC then add one or two flash devices, usually 1.92TB are more then sufficent.
=== Scale-out Cluster Media (Ceph Storage Pools) ===
QuantaStor supports a broad spectrum of NVMe, SAS, and SATA media when deploying scale-out clusters.  When selecting flash media we highly recommend the use of NVMe devices vs slower SATA/SAS SSDs due to the higher throughput performance for write logging and metadata operations.  All major media vendors are supported including Seagate, Solidigm, Micron, Western Digital, ScaleFlux, SanDisk, Kioxia, Toshiba and more.  Note that you must use enterprise or datacenter grade media in all configurations. Desktop grade media in unsuitable and will lead to durability, performance and uptime issues.


== Fibre Channel HBAs ==
== Fibre Channel HBAs ==


* QuantaStor SDS only supports Qlogic Fibre Channel HBAs for use with the FC SCSI target feature.
* QuantaStor only supports Qlogic Fibre Channel HBAs for use as FC target ports.  Emulex and other FC cards may be used in initiator mode only. QuantaStor systems with Qlogic FC ports in target mode will work with clients using FC initiators of any brand including Emulex and QLogic models and their OEM derivatives.


{| class="wikitable"
{| class="wikitable"
Line 268: Line 348:
! Model
! Model
! Type
! Type
! Notes
|-
| QLogic (including all OEM models)
| QLE2742
| 32Gb
|
|-
|-
| QLogic (including all OEM models)
| QLogic (including all OEM models)
| QLE26xx
| QLE2692
| 16Gb
| 16Gb
|
|-
| QLogic (including all OEM models)
| QLE267x
| 16Gb
|
|-
|-
| QLogic (including all OEM models)
| QLogic (including all OEM models)
| QLE25xx
| QLE25xx
| 8Gb
| 8Gb
|-
|
| QLogic (including all OEM models)
| QLE24xx
| 4Gb
|}
|}


== Network Interface Cards ==
== Network Interface Cards ==


* QuantaStor supports almost all 1GB and 10/100 network cards on the market
QuantaStor being Linux based supports the broad spectrum of network cards on the market but we generally recommend using nVidia/Mellanox, Intel, and Broadcom in that order for your network cards.  We're generally less concerned about the vendor used on the motherboard for the 1GbE/2.5GbE/10GbE management interface and generally support them all as long as they're one of the listed server vendors noted above that we've done at least one certification test with.
* High speed 10GbE/25GbE/40GbE/50GbE/100GbE NICs that have been tested for compatibility, stability and performance with QuantaStor are as listed.
 
=== Supported Vendors ===


{| class="wikitable"
{| class="wikitable"
Line 294: Line 384:
|-
|-
| Intel
| Intel
| X520, X520, X710, XL710,XXV710 (all 1/10/25/40GbE models)
| X520, X550, X710, XL710,XXV710 (all 1/10/25/40GbE models), E810
| 1/10/25/40GbE
| 1/10/25/40GbE, 100GbE
| SFP28, QSFP, SFP+
| 10GBaseT, SFP28, QSFP, SFP+, QSFP28
|-
|-
| Emulex
| Emulex
Line 303: Line 393:
| QSFP
| QSFP
|-
|-
| Mellanox
| nVidia
| ConnectX-7 EN Series
| 100/200/400GbE
| QSFP56
|-
| nVidia
| ConnectX-6 EN Series
| 25/50/100/200GbE
| SFP28/QSFP28/QSFP56
|-
| nVidia/Mellanox
| ConnectX-5 EN Series
| ConnectX-5 EN Series
| 25/50/100GbE
| 25/50/100GbE
| SFP28/QSFP28
| SFP28/QSFP28
|-
|-
| Mellanox
| nVidia/Mellanox
| ConnectX-4 EN Series
| ConnectX-4 EN Series
| 10/25/40/50/100GbE
| 10/25/40/50/100GbE
| SFP28/QSFP28
| SFP28/QSFP28
|-
|-
| Mellanox
| nVidia/Mellanox
| ConnectX-3 EN/VPI Series
| ConnectX-3 EN/VPI Series
| 10/40/56GbE
| 10/40/56GbE
| SFP+/QSFP+
| SFP+/QSFP+
|-
|-
| Mellanox
| nVidia/Mellanox
| ConnectX-3 Pro EN/VPI Series
| ConnectX-3 Pro EN/VPI Series
| 10GbE/40GbE/56GbE
| 10GbE/40GbE/56GbE
Line 327: Line 427:
| Quad-port 10GbE
| Quad-port 10GbE
| SFP+
| SFP+
|-
| Broadcom
| BCM57508
| 100GbE
| QSFP28
|}
|}


== SRP / Infiniband Adapters ==
== Infiniband Adapters ==
 
QuantaStor supports the use of Infiniband adapters for both scale-up and scale-out cluster configurations.  QuantaStor uses the ipoib (IP-over-IB) linux driver to support TCP/IP over Infiniband and the configuration.  These Infiniband interfaces in IP-over-IB mode will appear in the system with interface names like "ibN" where N is the interface number such as "ib0", "ib1", "ib2", "ib3" and so on. We recommend the use of Nvidia/Mellanox Infiniband NICs for all systems using IPoIB mode and requires installing the OFED drivers.  See (/opt/osnexus/quantastor/bin/mellanox-ofed-install.sh) and OSNexus Support for assistance with installing the latest OFED drivers.
 
* Note: Support for the SRP & iSER has been deprecated but standard iSCSI mode is supported with IP-over-IB


{| class="wikitable"
{| class="wikitable"
Line 336: Line 445:
! Type
! Type
|-
|-
| Mellanox
| Nvidia/Mellanox
| ConnectX Series
| 20Gb (DDR)
|-
| Mellanox
| ConnectX-2 VPI Series
| 40Gb (QDR)
|-
| Mellanox
| ConnectX-3 VPI Series
| ConnectX-3 VPI Series
| 40/56Gb (QDR)
| 40/56GbE (QDR)
|-
|-
| Mellanox
| Nvidia/Mellanox
| ConnectX-4 VPI Series
| ConnectX-4 VPI Series
| 25/40/50/100Gb
| 25/40/50/100GbE
|-
|-
| Mellanox
| Nvidia/Mellanox
| ConnectX-5 VPI Series
| ConnectX-5 VPI Series
| 25/50/100Gb
| 25/50/100GbE
|-
| Nvidia
| ConnectX-6 VPI Series
| 100/200GbE
|-
| Nvidia
| ConnectX-7 VPI Series
| 200/400GbE
|}
|}


Line 364: Line 473:
|-
|-
| Microsoft
| Microsoft
| Windows XP, 7, 8, 10, Windows 2003 Server, 2008 Server, 2008 Server R2, 2012 Server, 2016 Server
| Windows (all versions, 7 and newer)
| Windows iSCSI
| Windows iSCSI Initiator
|-
| Microsoft
| Windows Server (all versions, 2003 and newer)
| Windows iSCSI Initiator
|-
|-
| Apple
| Apple
| Mac OS X v10.5 'Leopard', v10.6 'Snow Leopard', v10.7 'Lion', v10.8 'Mountain Lion', v10.9 'Mavericks', v10.10 'Yosemite', v10.11 'El Capitan', v10.12 'Sierra', macOS 10.12: Sierra, macOS 10.13: High Sierra, macOS 10.14: Mojave
| macOS (all versions, 11 and newer)
| ATTO Xtend SAN initiator (globalSAN is not supported)
| ATTO Xtend SAN initiator (globalSAN is not supported)
|-
|-
| Citrix
| XenServer
| XenServer 5.x (iSCSI, FC, NFS), XenServer 6.x (iSCSI, FC, NFS), XenServer 7.x (iSCSI, FC, NFS), Citrix Ready Certified
| XenServer 7 and newer
| iSCSI SR, NFS SR, StorageLink SR
| iSCSI SR, NFS SR, StorageLink SR
|-
|-
| VMware
| VMware
| VMware ESXi 4.x, 5.x, 6.x (iSCSI, FC, NFS)
| VMware ESXi 6 and newer
| VMware initiator
| VMware initiator
|-
| Proxmox
| Proxmox 8 and newer
| Proxmox/Linux iSCSI initiator
|-
|-
| Linux
| Linux
| RHEL, SUSE, CentOS, Debian, Ubuntu, OEL, etc. (all major distros)
| RHEL, AlmaLinux, RockyLinux, SUSE, CentOS, Debian, Ubuntu, Oracle Linux (OEL), etc. (all major distros)
| open-iscsi
| open-iscsi
|}
|}

Latest revision as of 23:21, 8 June 2026

index.php?title=Category:Design guide QuantaStor is Linux based and has kernel driver updates released multiple times per year to ensure broad support for the latest hardware from the major vendors including HPE, Dell/EMC, Cisco, Intel, Seagate, Supermicro, Gigabyte and Lenovo. The hardware listed below has been tested for specific compatibility with QuantaStor to ensure system stability, performance and compatibility. If you have questions about specific hardware compatibility for components not listed below please contact support@osnexus.com for assistance.

Servers for QuantaStor SDS Systems

QuantaStor supports the broad spectrum of AMD and Intel x86 64bit servers on the market from major vendors. Special configuration files are used to integrate new servers and storage enclosures and there are configuration files for monitoring media, thermals, power supply and fan health. Although QuantaStor comes with configuration settings for most major server models in some cases we may need to update the configuration files at the time of deployment if your selected server model is not one we have default settings for.

Vendor Model Boot Controller CPU Memory (32GB min)
Cisco C220, C240 (M8/M7/M6/M5/M4 Series) Software RAID1 M.2 NVMe AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
DellEMC Gen 7 (eg. R770/R77x5/R670/R67x5), Gen 6 (eg. R760/R76x5/R660/R66x5), Gen5 (eg. R750), Gen4 (eg. R740), Gen3 (eg. R730) Software RAID1 M.2 NVMe or Dell BOSS AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
Gigabyte Intel and AMD servers Software RAID1 M.2 NVMe AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
HPE HP DLxxx Servers (Gen11, Gen10, Gen9, Gen8, Gen7) Software RAID1 M.2 NVMe or HPE P4xx w/ FBWC AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
HPE HP Apollo Servers (Gen9, Gen10 and Gen11) Software RAID1 M.2 NVMe or HPE P4xx w/ FBWC AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
MiTAC Intel Server Systems (M50FCP/M50CYP) Software RAID1 M.2 NVMe Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
Lenovo SRxxx V3 & V4 Rack Servers Software RAID1 M.2 NVMe or Hardware Boot RAID1 AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)
Seagate Seagate 2U24 AP Storage Server, 5U84 AP, 2U12 AP (AMD / Bonneville) Internal M.2 NVMe AMD EPYC 16 core 128GB RAM (Gold) 256GB RAM (Platinum), 512GB RAM (Titanium)
Supermicro X14/H14, X13/H13, X12/H12, X11, X10 & X9 based Intel and AMD servers Software RAID1 M.2 NVMe AMD EPYC, Intel Scalable Processors 0.3GB RAM (Archive) to 1GB RAM per usable TB (OLTP/Virtualization)

Disk Expansion Chassis / JBOD

  • QuantaStor Scale-Up
    • supports the use of SAS/NL-SAS media JBODs from most vendors
    • support external NVMe-oF JBOF from Western Digital (Data24 42xx series)
    • all JBODs must be dual-expander / dual-controller / dual-IO module (naming conventions vary by vendor)
  • QuantaStor Scale-Out
    • supports the use of SAS/NL-SAS and SATA JBODs from most vendors
    • support external NVMe-oF JBOF from Western Digital (Data24 41xx series)
    • supports JBODs that are both single and dual-expander but we recommend dual-IO module / dual-expanders here too

Supported Vendors

Vendor JBOD Model
Cisco All Models
Dell All Models
HighPoint RocketStor 4243AS
HPE All Models (D3000 Series, D2000 Series, D6020)
IBM/Lenovo All Models
Seagate Exos E All Models
Seagate Corvault All Models
Seagate Exos X All Models
SuperMicro All Models (HA requires dual expander JBODs)
Western Digital All Models (including Data60 / Data102 / Data24)

SAS HBA Controllers

To make Storage Pools highly-available (ZFS based pools) media must be connected to two or more servers via SAS (or FC) connectivity. We recommend Broadcom HBAs and their OEM equivalents. HPE is the exception where HPE disk expansion units must be used with HPE HBAs which are now Microsemi based. Make sure the HBA is flashed with the latest firmware to ensure compatibility with the latest storage media. For OEM branded HBAs always use the OEM's supplied firmware even if a newer firmware is available from Broadcom. SAS HBAs with only external ports (8e, 16e models) should be configured to ignore Boot ROM from Disk Devices. For Broadcom/LSI Controllers, this option is called 'Boot Support' and should be set to 'OS only' in the MPTSAS BIOS.

Vendor Model Type QS HW Mgmt Module
Broadcom/LSI/Avago 9600 24Gb tri-mode series SAS/NVMe HBA Yes
Broadcom/LSI/Avago 9500 12Gb tri-mode series SAS HBA Yes
Broadcom/LSI/Avago 9400 12Gb tri-mode series SAS HBA Yes
Broadcom/LSI/Avago 9300 12Gb series SAS HBA Yes
Cisco UCS 6Gb & 12Gb HBAs SAS HBA Yes
DELL/EMC SAS HBA 6Gb & 12Gb SAS HBA Yes
HPE SmartArray H241 12Gb SAS HBA Yes
Lenovo/IBM ServeRAID M5xxx 12Gb SAS HBA Yes

Boot RAID Controllers

QuantaStor supports both software RAID1 for boot and hardware RAID1 for boot using high quality datacenter grade media. We recommend SSDs in the 240GB to 480GB size range for boot. If SATADOM SSD devices are used use the 128GB devices and use software RAID1 to mirror them. Do not use Intel VROC but vendor specific dedicated boot controllers like the Dell BOSS are generally supported.

Vendor Model Type QS HW Mgmt Module
Broadcom/LSI/Avago MegaRAID (all models) SATA/SAS RAID Yes
Cisco UCS RAID SATA/SAS RAID Yes
DellEMC PERC H7xx, H8xx 6Gb & 12Gb models SATA/SAS RAID Yes
DellEMC BOSS SATA SSD RAID1 Yes
HPE SmartArray P4xx/P8xx SATA/SAS RAID Yes
Lenovo/IBM ServeRAID M5xxx SATA/SAS RAID Yes
Microsemi/Adaptec 5xxx/6xxx/7xxx/8xxx SATA/SAS RAID Yes

NVMe RAID Controllers

QuantaStor supports the use of NVMe RAID controllers for scale-out configurations and single node configurations. They are not supported for use with scale-up HA (high-availability) configurations.

Single Node Use Cases

  • High performance SAN/NAS using software single parity (RAIDZ1) over NVMe hardware RAID5.
  • High performance NVMeoF/FC/iSCSI target passthru of logical drives from NVMe hardware RAID5

Scale-out Use Cases

  • High performance storage for WAL/MDB
  • High performance OSDs with local rebuild capability, variable OSD count for higher performance

Supported Vendors

Vendor Model Type QS HW Mgmt Module
Graid Technology Inc. SupremeRAID™ SR-1000 NVMe RAID Yes
Graid Technology Inc. SupremeRAID™ SR-1010 NVMe RAID Yes
Pliops XDP-RAIDplus NVMe RAID No
Xinnor xiRAID NVMe RAID Yes

Storage Devices/Media

Scale-up HA Cluster Media (ZFS based Storage Pools)

Scale-up clusters require dual-ported media so that QuantaStor's IO fencing system can reserve device access to specific nodes for specific pools. These clusters generally consist of two QuantaStor servers with shared access to the storage media in a separate enclosure (JBOD or JBOF). Scale-up clusters using dual-ported NVMe media require the use of special NVMe based Storage-Bridge-Bay servers from Supermicro or the use of Western Digital Data24 42xx JBOFs.

SAS/NL-SAS Media

Vendor Model Media Type Notes
Western Digital Ultrastar NL-SAS HDDs Dual-port 6/12/24Gb SAS SSD & NL-SAS HDD
Seagate Exos NL-SAS HDDs, Nytro & Enterprise Performance SAS SSDs Dual-port 12/24Gb SAS SSD & NL-SAS HDD
Samsung Enterprise SAS SSDs Dual-port 12/24Gb SAS SSD
KIOXIA PM Enterprise Capacity/Enterprise Performance SAS SSD Dual-port 12/24Gb SAS SSD
Cisco, HPE, DellEMC, Lenovo OEM SAS SSD Dual-port 6/12/24Gb SAS SSD & NL-SAS HDD

Enterprise NVMe Media

Vendor Model Media Type Notes
Samsung Enterprise NVMe SSDs Dual-port NVMe Supermicro SBB or WD Data24 4200
KIOXIA CM Enterprise Dual-port NVMe Supermicro SBB or WD Data24 4200
SanDisk SN655 NVMe models with dual-port and IO fencing Dual-port NVMe Supermicro SBB or WD Data24 4200

Checking for NVMe Persistent Reservation Support

This command shows how to get the information on what modes of IO fencing are supported. What we're looking for is "[4:4] : 0 Exclusive Access - Registrants Only" but as you can see in the output mode below that is not supported on this media (Sandisk SN655) 

# nvme id-ns -H /dev/nvme6n1 | grep -A9 rescap
rescap  : 0
  [7:7] : 0     Ignore Existing Key - Used as defined in revision 1.2.1 or earlier
  [6:6] : 0     Exclusive Access - All Registrants Not Supported
  [5:5] : 0     Write Exclusive - All Registrants Not Supported
  [4:4] : 0     Exclusive Access - Registrants Only Not Supported
  [3:3] : 0     Write Exclusive - Registrants Only Not Supported
  [2:2] : 0     Exclusive Access Not Supported
  [1:1] : 0     Write Exclusive Not Supported
  [0:0] : 0     Persist Through Power Loss Not Supported

Journal Media (Metadata-Offload, ZIL & L2ARC)

  • Metadata-offload (special VDEV) is highly recommended for all HDD based storage pools. Add 3x flash devices that in sum represents 1% of the HDD raw capacity. For example a 1000TB pool would require 10TB of raw flash so 3x 3.84TB flash devices is ideal.
  • ZIL/SLOG is not recommended in most modern configurations. It is used to efficiently handle synchronous writes (SYNC_IO) common to databases and VMs. We recommend using all-flash pools for these workloads. If you do have a need for ZIL add 2x flash devices such as 2x 3.84TB devices.
  • L2ARC is not recommended in most modern configurations as the benefits of L2ARC are largely covered by the Metadata-offload (special VDEV). If you do elect to add L2ARC then add one or two flash devices, usually 1.92TB are more then sufficent.

Scale-out Cluster Media (Ceph Storage Pools)

QuantaStor supports a broad spectrum of NVMe, SAS, and SATA media when deploying scale-out clusters. When selecting flash media we highly recommend the use of NVMe devices vs slower SATA/SAS SSDs due to the higher throughput performance for write logging and metadata operations. All major media vendors are supported including Seagate, Solidigm, Micron, Western Digital, ScaleFlux, SanDisk, Kioxia, Toshiba and more. Note that you must use enterprise or datacenter grade media in all configurations. Desktop grade media in unsuitable and will lead to durability, performance and uptime issues.

Fibre Channel HBAs

  • QuantaStor only supports Qlogic Fibre Channel HBAs for use as FC target ports. Emulex and other FC cards may be used in initiator mode only. QuantaStor systems with Qlogic FC ports in target mode will work with clients using FC initiators of any brand including Emulex and QLogic models and their OEM derivatives.
Vendor Model Type Notes
QLogic (including all OEM models) QLE2742 32Gb
QLogic (including all OEM models) QLE2692 16Gb
QLogic (including all OEM models) QLE267x 16Gb
QLogic (including all OEM models) QLE25xx 8Gb

Network Interface Cards

QuantaStor being Linux based supports the broad spectrum of network cards on the market but we generally recommend using nVidia/Mellanox, Intel, and Broadcom in that order for your network cards. We're generally less concerned about the vendor used on the motherboard for the 1GbE/2.5GbE/10GbE management interface and generally support them all as long as they're one of the listed server vendors noted above that we've done at least one certification test with.

Supported Vendors

Vendor Model Type Connector
Intel X520, X550, X710, XL710,XXV710 (all 1/10/25/40GbE models), E810 1/10/25/40GbE, 100GbE 10GBaseT, SFP28, QSFP, SFP+, QSFP28
Emulex OneConnect OCe14xxx Series 10/40GbE QSFP
nVidia ConnectX-7 EN Series 100/200/400GbE QSFP56
nVidia ConnectX-6 EN Series 25/50/100/200GbE SFP28/QSFP28/QSFP56
nVidia/Mellanox ConnectX-5 EN Series 25/50/100GbE SFP28/QSFP28
nVidia/Mellanox ConnectX-4 EN Series 10/25/40/50/100GbE SFP28/QSFP28
nVidia/Mellanox ConnectX-3 EN/VPI Series 10/40/56GbE SFP+/QSFP+
nVidia/Mellanox ConnectX-3 Pro EN/VPI Series 10GbE/40GbE/56GbE SFP+/QSFP+
Interface Masters NIAGARA 32714L (OEM Intel) Quad-port 10GbE SFP+
Broadcom BCM57508 100GbE QSFP28

Infiniband Adapters

QuantaStor supports the use of Infiniband adapters for both scale-up and scale-out cluster configurations. QuantaStor uses the ipoib (IP-over-IB) linux driver to support TCP/IP over Infiniband and the configuration. These Infiniband interfaces in IP-over-IB mode will appear in the system with interface names like "ibN" where N is the interface number such as "ib0", "ib1", "ib2", "ib3" and so on. We recommend the use of Nvidia/Mellanox Infiniband NICs for all systems using IPoIB mode and requires installing the OFED drivers. See (/opt/osnexus/quantastor/bin/mellanox-ofed-install.sh) and OSNexus Support for assistance with installing the latest OFED drivers.

  • Note: Support for the SRP & iSER has been deprecated but standard iSCSI mode is supported with IP-over-IB
Vendor Model Type
Nvidia/Mellanox ConnectX-3 VPI Series 40/56GbE (QDR)
Nvidia/Mellanox ConnectX-4 VPI Series 25/40/50/100GbE
Nvidia/Mellanox ConnectX-5 VPI Series 25/50/100GbE
Nvidia ConnectX-6 VPI Series 100/200GbE
Nvidia ConnectX-7 VPI Series 200/400GbE

iSCSI Initiators / Client-side

Vendor Operating System iSCSI Initiator
Microsoft Windows (all versions, 7 and newer) Windows iSCSI Initiator
Microsoft Windows Server (all versions, 2003 and newer) Windows iSCSI Initiator
Apple macOS (all versions, 11 and newer) ATTO Xtend SAN initiator (globalSAN is not supported)
XenServer XenServer 7 and newer iSCSI SR, NFS SR, StorageLink SR
VMware VMware ESXi 6 and newer VMware initiator
Proxmox Proxmox 8 and newer Proxmox/Linux iSCSI initiator
Linux RHEL, AlmaLinux, RockyLinux, SUSE, CentOS, Debian, Ubuntu, Oracle Linux (OEL), etc. (all major distros) open-iscsi
Retrieved from "https://wiki.osnexus.com/index.php?title=Hardware_Compatibility_List_(HCL)&oldid=27159"