Difference between revisions of "Hardware Compatibility List (HCL)"

Revision as of 21:57, 14 May 2014

System Architecture Guide & Performance Considerations

The following sections provide some general recommendations for selecting the right hardware and configuration settings, so that your QuantaStor appliance delivers great performance for your workload. Note that these are general guidelines, and not all workloads fit in a single solution category. As such, if you need assistance on selecting the right hardware and configuration strategy for your QuantaStor storage appliances, please email us at support@osnexus.com for advice and assistance.

By Use Case

Server Virtualization

Server Virtualization / VM workloads are fairly sequential when you have just a few virtual machines, but don't be fooled. As you add more VMs, the I/O patterns to the storage appliance will become more and more random in nature. As such, you must design a configuration that is tuned to maximize IOPS. For this reason, for virtualization we always recommend that you configure your storage pool to use the RAID10 layout. We also recommend that you use a hardware RAID controller like the LSI MegaRAID to make it easy to detect and replace bad hard drives, even if you are using ZFS pool storage types. Within QuantaStor, you can direct the hardware RAID controller to make one large RAID10 unit, which you can then use to create a storage pool using the ZFS type with software RAID0.

This provides you with a RAID10 storage pool that both leverages the hardware RAID controller for hot-spare management, and leverages the capabilities of ZFS for easy expansion of the pool. When you expand the pool later, you can create an additional an RAID10 unit, then grow the pool using that new fault-tolerant logical device from the RAID controller. Alternatively you can expand the hardware RAID10 unit, then expand the storage pool afterwards to use the new additional space.

Another important enhancement is to add extra RAM to your system which will work as a read cache (ARC) and further boost performance. A good rule of thumb is 24GB for the base system plus 1GB-2GB for each additional VM, but more RAM is always better. If you have the budget to put 64GB, 128GB or even 256GB of RAM in your server head unit you will see performance advantages. Also use 10GbE or 8Gb FC for systems with many VMs or with VMs with high load applications like databases and Microsoft Exchange.

Desktop Virtualization (VDI)

All the recommendations from the above Server Virtualization section applies to VDI, plus you will want to move to SSD drives due to the high IOPS demands of desktops. Here again it is a good idea to use a ZFS based storage pool, so that you can create template desktop configurations, then clone or snapshot these templates to produce virtual desktops for end-users. More RAM is also recommended; 128GB is generally good for 50-100 desktops.

Disk Archive / Backup

Disk archive and backup applications generally write large amounts of data in a very sequential fashion. As such it works most efficiently with RAID6 formatting. Although you could chose RAID5, we don't recommend it - the performance of RAID6 is nearly identical and it allows the RAID unit to continue working after two disk failures. Make sure, therefore, to leave capacity in the chassis for 1-2 hot-spares, and note, these do not count towards your licensed capacity - get two if you have room.

The performance with an LSI 9271/9286 is about 1.6GB/sec sequential with the XFS storage pool type and about 1GB/sec sequential with the ZFS storage pool type. It takes about 16-20 drives to max out the throughput of a single LSI RAID controller, so we recommend that you get a second controller if you have 40+ disk drives in your QuantaStor archive appliance. We also don't recommend going over 20 drives in a RAID6 configuration due to the increase in rebuild times, and generally speaking 16 drives or less is best. If you have a 45-drive chassis, then making 3x RAID6 units of 14 drives each or 4x RAID6 units of 11 drives each is best.

Larger capacity drives like 3TB, 4TB, and 5TB drives will deliver better performance due to larger density platters. That said, 16x 2TB drives will be much faster than 8x 4TB drives due to the larger number of spindles and stripe size. You can combine multiple RAID6 units with the ZFS storage pool type using ZFS RAID0. This will produce a RAID60 storage pool, but it must be expanded using a full RAID6 unit. So if you combine 4x RAID6 units with 8 drives each, note that you'll need to add an 8 drive RAID6 unit when you expand the pool later. RAID60 is good to use if you have multiple simultaneous backup streams. It is also good to just create separate pools for each backup stream if you can, to limit spindle contention. If you have more than 5x disks in your archive system, use 10GbE and/or 8Gb FC or your network will be the bottleneck.

Media Post-Production Editing & Playback

For Media applications 10GbE is critical, 1GbE NICs are just not fast enough for most modern playback and editing environments. You will also want to have large stripe sets that can keep up with ingest and playback. Choose a chassis with room for 20+ disks. Performance will increase linearly as you add more disks so you want as many disks as possible, up to ~20 per RAID unit in RAID6. Add a good amount of RAM to be used as a read cache and boost performance. This is assuming you have largely sequential I/O patterns. If you have multiple workstations all writing to the storage pool at the same time, then you will want to consider using RAID10.

Research and Life Sciences

The correct RAID layout depends on access patterns and file size. For millions of small files, and systems with many active workstations concurrently writing data to a given storage pool, it is best to use RAID10. Alternatively, RAID6 or RAID60 with an SSD write cache (ZIL) may work well, if the number of concurrent writes is low. The best approach is to configure two pools, one using RAID10, one in RAID60, and see which one works best. Each LSI RAID controller will max out at 1.6 to 1.8GB/sec so an optimal configuration will have 1-2 10GbE ports per RAID controller and one RAID controller per 20x-40x 7200 RPM disks. With ZFS the throughput to the RAID controller will be less, at about 1GB to 1.3GB/sec with dual 8Gb FC.

Large systems with 100+ disks should use 4x RAID controllers and should consider breaking up the load among 2x or more QuantaStor appliances. QuantaStor's grid management technology allows you to combine multiple appliances together. That said, the storage pools will be tied to specific appliances, so it is not a single namespace by default. There is an advantage to this, as heavy I/O load on Pool A will not impact the performance of an application running on Pool B, even if both pools are in the same appliance, so long as there is adequate network bandwidth and back-end RAID controller bandwidth. Also, note that you can create scale-out single namespace Gluster volumes with QuantaStor's integrated Gluster management. You can also use storage pools simultaneously for Gluster, NFS/CIFS, FC/iSCSI, and Hadoop.

Recommended Hardware Guide

QuantaStor is based on Linux, specifically Ubuntu Linux 12.04 with QuantaStor v3 so the driver support is very broad. That said, the following equipment is generally recommended as it has been tested and used reliably in many QuantaStor customer deployments. If you have questions about hardware compatibility please to contact support@osnexus.com for assistance.