Storage Pools - Revision history

Qadmin: /* Multiple Pool Failovers */

2024-03-22T06:25:48Z

Multiple Pool Failovers

← Older revision		Revision as of 06:25, 22 March 2024
Line 176:		Line 176:
	If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode. These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu. This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.		If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode. These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu. This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.

	=== Multiple Pool Failovers ===		==== Multiple Pool Failovers ====
	Storage Pools configured for HA have an associated High-availability Group with a set of HA failover policies. The default policy is to do a failover if all data ports go offline. Systems should have ethernet ports connected to two separate switches that a switch outage does not bring all ports offline. Similarly with systems with FC ports a FC switch should always be used in-between the QuantaStor system and the client machines else the HA Group policy should be reconfigured to not failover when all FC ports are down as a direct connection to a FC initiator that is restarting will confuse the system into thinking all ports are down and that a failover is needed. QuantaStor will use the heart-beat network ports to communicate with the secondary node to ensure that connectivity is available before doing a failover so if both nodes lose access there is not an unnecessary pool failover.		Storage Pools configured for HA have an associated High-availability Group with a set of HA failover policies. The default policy is to do a failover if all data ports go offline. Systems should have ethernet ports connected to two separate switches that a switch outage does not bring all ports offline. Similarly with systems with FC ports a FC switch should always be used in-between the QuantaStor system and the client machines else the HA Group policy should be reconfigured to not failover when all FC ports are down as a direct connection to a FC initiator that is restarting will confuse the system into thinking all ports are down and that a failover is needed. QuantaStor will use the heart-beat network ports to communicate with the secondary node to ensure that connectivity is available before doing a failover so if both nodes lose access there is not an unnecessary pool failover.

Qadmin: /* Troubleshooting */

2024-03-22T06:25:23Z

Troubleshooting

← Older revision		Revision as of 06:25, 22 March 2024
Line 175:		Line 175:
	==== due to 'foreign media' ====		==== due to 'foreign media' ====
	If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode. These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu. This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.		If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode. These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu. This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.

			=== Multiple Pool Failovers ===
			Storage Pools configured for HA have an associated High-availability Group with a set of HA failover policies. The default policy is to do a failover if all data ports go offline. Systems should have ethernet ports connected to two separate switches that a switch outage does not bring all ports offline. Similarly with systems with FC ports a FC switch should always be used in-between the QuantaStor system and the client machines else the HA Group policy should be reconfigured to not failover when all FC ports are down as a direct connection to a FC initiator that is restarting will confuse the system into thinking all ports are down and that a failover is needed. QuantaStor will use the heart-beat network ports to communicate with the secondary node to ensure that connectivity is available before doing a failover so if both nodes lose access there is not an unnecessary pool failover.

Qadmin: /* Pool Performance Inconsistent */

2024-03-22T06:18:23Z

Pool Performance Inconsistent

← Older revision		Revision as of 06:18, 22 March 2024
Line 158:		Line 158:

	If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read. If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices. We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.		If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read. If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices. We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.

			==== Small File Performance Issues ====
			If you're having performance issues with small files in a HDD based pool we recommend adding 3x SSDs to the pool as meta-data offload with a 32K small-file offload size. Best performance for IOPS, small files, databases, VMs, and other workloads with effectively random and small block IO access patterns is to use small parity length layout (RAIDZ2 / 4d+2p) or mirroring (RAID10). The other important factor is having many VDEVs, at least 3 and ideally 4 or more. If you don't have enough VDEVs expand you system to increase the number of VDEVs which are referred to as Storage Pool Device Groups in QuantaStor.

	==== Failed Pool ====		==== Failed Pool ====

Qadmin: /* Troubleshooting */

2024-03-22T06:14:12Z

Troubleshooting

← Older revision		Revision as of 06:14, 22 March 2024
Line 158:		Line 158:

	If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read. If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices. We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.		If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read. If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices. We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.

			==== Failed Pool ====

			==== due to media connectivity ====
			If a pool is in the Failed state that indicates that more devices in the pool have failed within a given VDEV/stripe than can be repaired and the pool has stopped. Often this problem is related to device connectivity and not failed media. For example, if an extern JBOD is powered off this will remove access to many devices and if the pool is not striped across many JBODs the pool will stop in a Failed state. To address this check any external JBODs and make sure they're cabled up properly and powered on.

			==== due to HA failover ====
			Another reason one could see a Failed state is in a HA configuration where the other node has taken over the pool. The System seeing the pool in the Failed state is reporting that because it no longer has write access to the pool. In an HA cluster configuration the node that had the pool at the time it transitioned to the Failed state will need to be rebooted. This is because any write I/O in-flight is effectively stuck in the disk IO stack and this cannot be cleared without a reboot. QuantaStor has many guards in place to prevent a Failed pool from being re-imported on a system until it is rebooted.

			==== due to too many failed devices ====
			If you see multiple devices in a single VDEV that are failed beyond what the pool can repair (for example 3x devices in a VDEV of 4d+2p) then the pool may not be recoverable without assistance from a drive repair service. In such conditions we recommend powering everything off and trying to restart the system (cold boot including cold boot of any JBODs) a few times to see if one or more of the failed devices can recover.

			==== due to 'foreign media' ====
			If your system is using media from a hardware RAID card (not recommended) you may have the media passed through to QuantaStor using several single RAID0 devices as many early hardware RAID card models did not support a true JBOD passthrough mode. These cards can get into a state where the media is not recognized as owned by the RAID card which can happen if the RAID card is replace. You can address this in the QuantaStor web management interface via the hardware Controllers & Enclosures section. Locate the hardware RAID card for the system then right-click and choose 'Import Foreign Units..' from the pop-up menu. This will import any/all hardware RAID units discovered by the controller and then you'll be able to start the storage pool.

Qadmin: /* Hot-Spare Management Policies */

2024-03-22T05:59:26Z

Hot-Spare Management Policies

← Older revision		Revision as of 05:59, 22 March 2024
Line 145:		Line 145:

	If the policy is set to one that includes 'exact match', the Storage Pool will first attempt to replace the failed data drive with a disk that is of the same model and capacity before trying other options. The ''Manual Hot-spare Management Only'' mode will disable QuantaStor's hot-spare management system for the pool. This is useful if there are manual/custom reconfiguration steps being run by an administrator via a root SSH session.		If the policy is set to one that includes 'exact match', the Storage Pool will first attempt to replace the failed data drive with a disk that is of the same model and capacity before trying other options. The ''Manual Hot-spare Management Only'' mode will disable QuantaStor's hot-spare management system for the pool. This is useful if there are manual/custom reconfiguration steps being run by an administrator via a root SSH session.

			===Troubleshooting===

			==== Degraded Pool ====
			When a pool becomes '''degraded''' this state indicates more media devices has failed or become disconnected from the system. If the pool has an assigned hot-spare after a minute it will try to repair the pool using the spare. If the system has no hot-spares add a device that is the same size (or larger) and type as the failed device and mark it as a hot-spare in the Physical Disks section. QuantaStor will automatically inject the spare device into the pool to start the resilvering/rebuild process to bring the pool back to full health.

			==== Rebuild Taking Long Time ====
			The amount of time it takes to heal a pool varies based on the speed and type of the media (HDD, SSD, NVMe) how much load is on the system and how many vdevs you have in the pool. All storage pools should be comprised of at least two VDEVs so that the pool can better manage workloads during drive rebuilds. If the pool is HDD based we highly recommend adding 3x SSDs/NVMe for meta-data offload. Rebuilds are very metadata intensive so having the metadata in an all-flash vdev ("special" type vdev) greatly speeds up pool rebuilds and pool scrubs.

			==== Pool Performance Inconsistent ====
			Inconsistent performance can be an indicator of media that is failing. QuantaStor looks for predictive drive failure indicators and reports on that and also marks bad and failing media with a 'Warning' state. To check the media first navigate to the Physical Disks section and check to see that all devices are in the Normal state. If any are in the Warning or Error state click on them, then review the State Detail in the Properties section to get information about what is wrong with the device. If predictive drive failures are reporting then the media should be replaced. When predictive drive failures are detected QuantaStor generates alerts so it is also important to make sure you have email or other alert notification options configured in the Alert Manager section.

			If all the media shows as Normal we recommend running a 'Read Performance Test' and this is also available in the Physical Disks section. The read performance test will read from all the selected devices using a basic 'dd' read. If you see large inconsistencies in the performance between one or two devices and the rest that can be an indicator that there's a problem with those devices. We recommend opening a support ticket, sending logs, and getting assistance with reviewing the media health.

Qadmin: /* Storage Pool Layout Types (ZFS based) */

2024-03-20T06:05:26Z

Storage Pool Layout Types (ZFS based)

← Older revision		Revision as of 06:05, 20 March 2024
Line 64:		Line 64:
	* Striping '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.		* Striping '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.
	* Mirroring '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.		* Mirroring '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.
	* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via what's called a parity calculation where one of the drives contains an XOR calculation of the bits on the other drives. For example, if you have 4 disk drives and you create a RAID5 storage pool, 3 of the disks will store data, and the last disk will contain parity information. This parity information on the 4th drive can be used to recover from any data disk failure. In the event that the parity drive fails, it can be replaced and reconstructed using the data disks. RAID5 (and RAID6) are especially well suited for audio/video streaming, archival, and other applications which do a heavy sequential write I/O operations (such as reading/writing large files) and are not as well suited for database applications which do heavy amounts of small random write I/O operations or with large file-systems containing lots of small files with a heavy write load.
	* Double-Parity '''RAID6/RAIDZ2''' improves upon RAID5 in that it can handle two drive failures but it requires that you have two disk drives dedicated to parity information. For example, if you have a RAID6 storage pool comprised of 5 disks then 3 disks will contain data, and 2 disks will contain parity information. In this example, if the disks are all 1TB disks then you will have 3TB of usable disk space for the creation of volumes. So there's some sacrifice of usable storage space to gain the additional fault tolerance. If you have the disks, we always recommend using RAID6 over RAID5. This is because all hard drives eventually fail and when one fails in a RAID5 storage pool your data is left vulnerable until a spare disk is utilized to recover your storage pool back to a fault tolerant status. With RAID6 your storage pool is still fault tolerant after the first drive failure. (Note: Fault-tolerant storage pools (RAID1,5,6,10) that have suffered a single disk drive failure are called '''degraded''' because they're still operational but they require a spare disk to recover back to a fully fault-tolerant status.)
	* Striped-Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.		* Striped-Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.
	* Double-parity '''RAID60/RAIDZ2''' ~~combines the benefits of RAID6 with some of the benefits of RAID10~~. It is ~~a good compromise when you need better IOPS performance than RAID6 will deliver 67%~~ and ~~higher useable storage than RAID10 delivers (50% of raw)~~.		* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via a single parity block per data stripe. It's fast but not fault-tolerant enough for production workloads and should be avoided.
	* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive applications as it can sustain three simultaneous disk failures per device group (VDEV).		* Double-parity '''RAID60/RAIDZ2''' provides two parity blocks per data stripe and provides robust throughput and IOPS performance. It is our recommended layout for most workloads and is the default in QuantaStor.
			* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive and backup applications as it can sustain three simultaneous disk failures per device group (VDEV).
			* Distributed-RAID '''DRAID''' This layout is a variation of the Double-parity and Triple-parity layouts that provides much faster rebuilds by distributing the data and parity information over a much larger group of devices. We recommend this on systems with multiple large 60, 84, and 90, and 106 bay JBODs used for backup and archive use cases.

	It can be useful to create more than one storage pool so that you have low cost fault-tolerant storage available in RAIDZ2/60 for backup and archive and higher IOPS storage in RAID10/mirrored layout for virtual machines, containers and databases.		It can be useful to create more than one storage pool so that you have low cost fault-tolerant storage available in RAIDZ2/60 for backup and archive and higher IOPS storage in RAID10/mirrored layout for virtual machines, containers and databases.

Qadmin: /* Storage Pool Layout Types (ZFS based) */

2024-03-20T05:59:39Z

Storage Pool Layout Types (ZFS based)

← Older revision		Revision as of 05:59, 20 March 2024
Line 62:		Line 62:
	QuantaStor supports all industry standard RAID levels (RAID1/10/5(Z1)/50/6(Z2)/60), some additional advanced RAID levels (RAID1 triple copy, RAIDZ3 triple-parity), and simple striping with RAID0. Over time all disk media degrades and as such we recommend marking at least one device as a hot-spare disk so that the system can automatically heal itself when a bad device needs replacing. One can assign hot-spare disks as universal spares for use with any storage pools as well as pinning of hot-spares to specific storage pools. Finally, RAID0 is not fault-tolerant at all but it is your only choice if you have only one disk and it can be useful in some scenarios where fault-tolerance is not required. Here's a breakdown of the various RAID types and their pros & cons.		QuantaStor supports all industry standard RAID levels (RAID1/10/5(Z1)/50/6(Z2)/60), some additional advanced RAID levels (RAID1 triple copy, RAIDZ3 triple-parity), and simple striping with RAID0. Over time all disk media degrades and as such we recommend marking at least one device as a hot-spare disk so that the system can automatically heal itself when a bad device needs replacing. One can assign hot-spare disks as universal spares for use with any storage pools as well as pinning of hot-spares to specific storage pools. Finally, RAID0 is not fault-tolerant at all but it is your only choice if you have only one disk and it can be useful in some scenarios where fault-tolerance is not required. Here's a breakdown of the various RAID types and their pros & cons.

	* '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.		* Striping '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.
	* '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.		* Mirroring '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.
	* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via what's called a parity calculation where one of the drives contains an XOR calculation of the bits on the other drives. For example, if you have 4 disk drives and you create a RAID5 storage pool, 3 of the disks will store data, and the last disk will contain parity information. This parity information on the 4th drive can be used to recover from any data disk failure. In the event that the parity drive fails, it can be replaced and reconstructed using the data disks. RAID5 (and RAID6) are especially well suited for audio/video streaming, archival, and other applications which do a heavy sequential write I/O operations (such as reading/writing large files) and are not as well suited for database applications which do heavy amounts of small random write I/O operations or with large file-systems containing lots of small files with a heavy write load.		* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via what's called a parity calculation where one of the drives contains an XOR calculation of the bits on the other drives. For example, if you have 4 disk drives and you create a RAID5 storage pool, 3 of the disks will store data, and the last disk will contain parity information. This parity information on the 4th drive can be used to recover from any data disk failure. In the event that the parity drive fails, it can be replaced and reconstructed using the data disks. RAID5 (and RAID6) are especially well suited for audio/video streaming, archival, and other applications which do a heavy sequential write I/O operations (such as reading/writing large files) and are not as well suited for database applications which do heavy amounts of small random write I/O operations or with large file-systems containing lots of small files with a heavy write load.
	* Double-Parity '''RAID6/RAIDZ2''' improves upon RAID5 in that it can handle two drive failures but it requires that you have two disk drives dedicated to parity information. For example, if you have a RAID6 storage pool comprised of 5 disks then 3 disks will contain data, and 2 disks will contain parity information. In this example, if the disks are all 1TB disks then you will have 3TB of usable disk space for the creation of volumes. So there's some sacrifice of usable storage space to gain the additional fault tolerance. If you have the disks, we always recommend using RAID6 over RAID5. This is because all hard drives eventually fail and when one fails in a RAID5 storage pool your data is left vulnerable until a spare disk is utilized to recover your storage pool back to a fault tolerant status. With RAID6 your storage pool is still fault tolerant after the first drive failure. (Note: Fault-tolerant storage pools (RAID1,5,6,10) that have suffered a single disk drive failure are called '''degraded''' because they're still operational but they require a spare disk to recover back to a fully fault-tolerant status.)		* Double-Parity '''RAID6/RAIDZ2''' improves upon RAID5 in that it can handle two drive failures but it requires that you have two disk drives dedicated to parity information. For example, if you have a RAID6 storage pool comprised of 5 disks then 3 disks will contain data, and 2 disks will contain parity information. In this example, if the disks are all 1TB disks then you will have 3TB of usable disk space for the creation of volumes. So there's some sacrifice of usable storage space to gain the additional fault tolerance. If you have the disks, we always recommend using RAID6 over RAID5. This is because all hard drives eventually fail and when one fails in a RAID5 storage pool your data is left vulnerable until a spare disk is utilized to recover your storage pool back to a fault tolerant status. With RAID6 your storage pool is still fault tolerant after the first drive failure. (Note: Fault-tolerant storage pools (RAID1,5,6,10) that have suffered a single disk drive failure are called '''degraded''' because they're still operational but they require a spare disk to recover back to a fully fault-tolerant status.)
	* Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.		* Striped-Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.
	* Double-parity '''RAID60/RAIDZ2''' combines the benefits of RAID6 with some of the benefits of RAID10. It is a good compromise when you need better IOPS performance than RAID6 will deliver 67% and higher useable storage than RAID10 delivers (50% of raw).		* Double-parity '''RAID60/RAIDZ2''' combines the benefits of RAID6 with some of the benefits of RAID10. It is a good compromise when you need better IOPS performance than RAID6 will deliver 67% and higher useable storage than RAID10 delivers (50% of raw).
	* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive applications as it can sustain three simultaneous disk failures per device group (VDEV).		* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive applications as it can sustain three simultaneous disk failures per device group (VDEV).

Qadmin: /* Storage Pool Layout Types (ZFS based) */

2024-03-20T05:59:07Z

Storage Pool Layout Types (ZFS based)

← Older revision		Revision as of 05:59, 20 March 2024
Line 64:		Line 64:
	* '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.		* '''RAID0''' layout is also called 'striping' and it writes data across all the disk drives in the storage pool in a round robin fashion. This has the effect of greatly boosting performance. The drawback of RAID0 is that it is not fault tolerant, meaning that if a single disk in the storage pool fails then all of your data in the storage pool is lost. As such RAID0 is not recommended except in special cases where the potential for data loss is non-issue.
	* '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.		* '''RAID1/mirroring''' is also called 'mirroring' because it achieves fault tolerance by writing the same data to two disk drives so that you always have two copies of the data. If one drive fails, the other has a complete copy and the storage pool continues to run. RAID1 and it's variant RAID10 are ideal for databases and other applications which do a lot of small write I/O operations.
	* '''RAID5/RAIDZ1''' achieves fault tolerance via what's called a parity calculation where one of the drives contains an XOR calculation of the bits on the other drives. For example, if you have 4 disk drives and you create a RAID5 storage pool, 3 of the disks will store data, and the last disk will contain parity information. This parity information on the 4th drive can be used to recover from any data disk failure. In the event that the parity drive fails, it can be replaced and reconstructed using the data disks. RAID5 (and RAID6) are especially well suited for audio/video streaming, archival, and other applications which do a heavy sequential write I/O operations (such as reading/writing large files) and are not as well suited for database applications which do heavy amounts of small random write I/O operations or with large file-systems containing lots of small files with a heavy write load.		* Single-Parity '''RAID5/RAIDZ1''' achieves fault tolerance via what's called a parity calculation where one of the drives contains an XOR calculation of the bits on the other drives. For example, if you have 4 disk drives and you create a RAID5 storage pool, 3 of the disks will store data, and the last disk will contain parity information. This parity information on the 4th drive can be used to recover from any data disk failure. In the event that the parity drive fails, it can be replaced and reconstructed using the data disks. RAID5 (and RAID6) are especially well suited for audio/video streaming, archival, and other applications which do a heavy sequential write I/O operations (such as reading/writing large files) and are not as well suited for database applications which do heavy amounts of small random write I/O operations or with large file-systems containing lots of small files with a heavy write load.
	* '''RAID6/RAIDZ2''' improves upon RAID5 in that it can handle two drive failures but it requires that you have two disk drives dedicated to parity information. For example, if you have a RAID6 storage pool comprised of 5 disks then 3 disks will contain data, and 2 disks will contain parity information. In this example, if the disks are all 1TB disks then you will have 3TB of usable disk space for the creation of volumes. So there's some sacrifice of usable storage space to gain the additional fault tolerance. If you have the disks, we always recommend using RAID6 over RAID5. This is because all hard drives eventually fail and when one fails in a RAID5 storage pool your data is left vulnerable until a spare disk is utilized to recover your storage pool back to a fault tolerant status. With RAID6 your storage pool is still fault tolerant after the first drive failure. (Note: Fault-tolerant storage pools (RAID1,5,6,10) that have suffered a single disk drive failure are called '''degraded''' because they're still operational but they require a spare disk to recover back to a fully fault-tolerant status.)		* Double-Parity '''RAID6/RAIDZ2''' improves upon RAID5 in that it can handle two drive failures but it requires that you have two disk drives dedicated to parity information. For example, if you have a RAID6 storage pool comprised of 5 disks then 3 disks will contain data, and 2 disks will contain parity information. In this example, if the disks are all 1TB disks then you will have 3TB of usable disk space for the creation of volumes. So there's some sacrifice of usable storage space to gain the additional fault tolerance. If you have the disks, we always recommend using RAID6 over RAID5. This is because all hard drives eventually fail and when one fails in a RAID5 storage pool your data is left vulnerable until a spare disk is utilized to recover your storage pool back to a fault tolerant status. With RAID6 your storage pool is still fault tolerant after the first drive failure. (Note: Fault-tolerant storage pools (RAID1,5,6,10) that have suffered a single disk drive failure are called '''degraded''' because they're still operational but they require a spare disk to recover back to a fully fault-tolerant status.)
	* '''RAID10~~/mirroring~~''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.		* Mirroring '''RAID10''' is similar to RAID1 in that it utilizes mirroring, but RAID10 also does striping over the mirrors. This gives you the fault tolerance of RAID1 combined with the performance of RAID10. The drawback is that half the disks are used for fault-tolerance so if you have 8 1TB disks utilized to make a RAID10 storage pool, you will have 4TB of usable space for creation of volumes. RAID10 will perform very well with both small random IO operations as well as sequential operations and it is highly fault tolerant as multiple disks can fail as long as they're not from the same mirror-pairing. If you have the disks and you have a mission critical application we '''highly''' recommend that you choose the RAID10 layout for your storage pool.
	* '''RAID60/RAIDZ2''' combines the benefits of RAID6 with some of the benefits of RAID10. It is a good compromise when you need better IOPS performance than RAID6 will deliver and ~~more~~ useable storage than RAID10 delivers (50% of raw).		* Double-parity '''RAID60/RAIDZ2''' combines the benefits of RAID6 with some of the benefits of RAID10. It is a good compromise when you need better IOPS performance than RAID6 will deliver 67% and higher useable storage than RAID10 delivers (50% of raw).
			* Triple-parity '''RAIDZ3''' Is a triple-parity layout and is ideal for long term archive applications as it can sustain three simultaneous disk failures per device group (VDEV).

	It can be useful to create more than one storage pool so that you have low cost fault-tolerant storage available in RAIDZ2/60 for backup and archive and higher IOPS storage in RAID10/mirrored layout for virtual machines, containers and databases.		It can be useful to create more than one storage pool so that you have low cost fault-tolerant storage available in RAIDZ2/60 for backup and archive and higher IOPS storage in RAID10/mirrored layout for virtual machines, containers and databases.

Qadmin: /* RAM Read Cache Configuration (ARC) */

2024-03-20T05:56:18Z

RAM Read Cache Configuration (ARC)

← Older revision		Revision as of 05:56, 20 March 2024
Line 52:		Line 52:

	==== RAM Read Cache Configuration (ARC) ====		==== RAM Read Cache Configuration (ARC) ====
	ZFS based storage pools use what are called "ARC" as a in-memory read cache rather than the Linux filesystem buffer cache to boost disk read performance. Having a good amount of RAM in your system is critical to deliver solid performance. It is very common with disk systems for blocks to be read multiple times. The frequently accessed "hot data" is cached into RAM where it can serve read requests orders of magnitude faster than reading it from spinning disk. Since the cache takes on some of the load it also reduces the load on the disks and this too leads to additional boosts in read and write performance. It is recommended to have a minimum of 32GB to 64GB of RAM in small systems, 96GB to 128GB of RAM for medium sized systems and 256GB or more in large systems.		ZFS based storage pools use what are called "ARC" as a in-memory read cache rather than the Linux filesystem buffer cache to boost disk read performance. Having a good amount of RAM in your system is critical to deliver solid performance. It is very common with disk systems for blocks to be read multiple times. The frequently accessed "hot data" is cached into RAM where it can serve read requests orders of magnitude faster than reading it from spinning disk. Since the cache takes on some of the load it also reduces the load on the disks and this too leads to additional boosts in read and write performance. It is recommended to have a minimum of 32GB to 64GB of RAM in small systems, 96GB to 128GB of RAM for medium sized systems and 256GB or more in large systems. Our [https://www.osnexus.com/zfs-designer design tools] can help with solution design recommendations here.

	=== Storage Pool Layout Types (ZFS based) ===		=== Storage Pool Layout Types (ZFS based) ===

Qadmin: /* Accelerating Write Performance with SSD Write Log (ZFS SLOG/ZIL) */

2024-03-20T05:55:14Z

Accelerating Write Performance with SSD Write Log (ZFS SLOG/ZIL)

← Older revision		Revision as of 05:55, 20 March 2024
Line 41:		Line 41:

	[[File:Add Cache Device Web 6.jpg\|1024px]]		[[File:Add Cache Device Web 6.jpg\|1024px]]

			==== Accelerating All Performance with SSD Meta-data & Small File Offload (ZFS special VDEVs) ====
			Scale-up Storage Pools using HDDs should always include 3x SSDs for metadata offload and small file offload. This feature of the filesystem is so powerful and impactful to improving performance for scrubs, replication, HA failover, small-file performance and more that we do not design any HDD based pools without it. All-flash storage pools do not need it and should not use it unless you're looking to optimize the use of QLC media. QLC media is not ideal for small block IO so using TLC media to offload the filesystem metadata and small files is a good idea. That would be the only use case for using metadata-offload SSDs with an all-flash storage pool.

	==== Accelerating Write Performance with SSD Write Log (ZFS SLOG/ZIL) ====		==== Accelerating Write Performance with SSD Write Log (ZFS SLOG/ZIL) ====