QuantaStor Installation Guide

From OSNEXUS Wiki
Jump to: navigation, search

Contents

Minimum Appliance Requirements

  • Intel Xeon or AMD x64 bit server (or virtual server)
  • 16GB RAM for bare-metal/physical appliances -or- 4GB RAM for virtual storage appliances (VSAs)
  • 2x disks for system/boot drives (SATA/SAS or virtual)
    • Boot drives should be hardware mirrored and should be at least 100GB in size
  • One or more disks per appliance for storage pools.
    • Block devices used to create storage pools can be of any type (SSD/NVMe/PCI SSD/SATA/SAS/FC/iSCSI/AoE)
  • Download QuantaStor USB/ISO Installation Media for installation
    • The QuantaStor ISO image is a hybrid ISO that allows for installation from DVD media or USB media like thumb-drives. More detail on creating a USB bootable flash drive for installing QuantaStor is available here.

Standard Installation

BIOS Configuration Checks

Boot Order

Before installing QuantaStor from the ISO, two changes must be made to the BIOS. First the USB or CD/DVD device with the QuantaStor ISO media must be selected as the primary boot device so that the install media loads at the initial install time. Second, the system disk, (the disk the system will boot to after installation has completed) should be the selected hard disk to boot from and assigned to the second position in the BIOS boot order. After installation is complete you will need to remove the ISO installation media.

Power Save Settings

Make sure that your storage appliance server is set to the Performance settings. You do not want the drives to go into power save mode as this can cause stability problems. QuantaStor is not designed to hibernate or sleep like a desktop operating system.

IPMI/iDRAC Settings

Be sure to apply an IP address to the server's IPMI port so that you can remotely access the console. This is especially important for triage of connectivity and network issues.

Starting the Installation from ISO/USB media

Preparing the ISO/USB media

For bare-metal installation we recommend using a 8GB or larger USB stick. Once you've downloaded the ISO file from the OSNEXUS web site you can put it onto your USB device using software like usbwriter. Another option is to burn a DVD and use a portable USB DVD device to install from. Detailed instructions on how to put QuantaStor on to USB media is provided here for both Windows and Linux.

Insert and Boot from the QuantaStor USB Install Media

After connecting your QuantaStor Install USB stick to the server you will see the QuantaStor screen appear and it will automatically select 'English' as the default language and begin the installation process.

Selecting a hostname

The first question you will be asked as part of the installation process is what do you want to name the host. By default the name 'quantastor' is given, but you can give the system any valid host name. If you're setting up multiple appliances you should give them each unique names. You can also change the name later through the web management interface using the Storage System Modify dialog.

Selecting a network port

The second step in the console level installation process is the selection of which port to use for network communication during the install process. If the new QuantaStor system has a single network interface then it will automatically attempt to configure the using DHCP; if DHCP is not available then standard details will need to be provided (Static IP Address, Subnetmask, and gateway if applicable). If multiple ports are detected a menu will be provided allowing you to choose which interface to use for the management port. The installation process will use the network to try to get more recent versions of packages than those that are on the CD.

Selecting the device to install the QuantaStor OS

Finally, the configuration wizard will ask which partitioning scheme to use. We recommend 'Guided installation, use entire disk'. If you are planning to mirror your boot drive (highly recommended) then you'll want to use the Hardware RAID controller's configuration BIOS to create a RAID1 logical drive first. The next screen will ask you which disk you want to use as the system boot disk, at that point you'll select your RAID1 mirrored system/boot drive.

WARNING, this disk will be reformatted as part of the installation process so be 
careful to choose a disk that does not have data on it you need.

Bare-metal Installation is Complete

That's all the key questions you will be asked as part of the installation process. After the installation completes the system will reboot and will start-up the QuantaStor system. At this point you can login using the 'qadmin' user account and 'qadmin' password so that you can get the IP address of the system. To get the IP address just type 'ifconfig' at the command prompt after you have logged in. You'll see an entry for 'eth0' with an IP address assigned to it. Enter that IP address into your web browser to connect to your QuantaStor storage system via your favorite web browser so that you can start to configure the rest of the system.

A video of the installation process is available on the main web site in the Videos section.

Virtual Storage Appliance - VM Template Creation

If you're installing QuantaStor from ISO into a virtual machine you may also want to make that virtual machine (virtual storage appliance / VSA) into a template. With a template you can easily setup larger QuantaStor grids and try out multi-node features like HA, DR, scale-out NAS, etc. There are a few commands you'll need to run in order to prepare the VSA to be used as a template, they are:

sudo -i
qs-util resetids
rm /etc/udev/rules.d/70-persistent-net.rules
shutdown -P now

That will reset the storage system ID for the appliance and clear the network MAC address to port name mapping so that it can be reinitialized for each snapshot/clone you make of the VSA template. Note that when you clone the VSA be sure to check the box that says 'Reset all MAC addresses' as you need unique MAC addresses for the virtual network interfaces for each VM instance. Also note that for basic test configurations you'll want at least 1GB of RAM and 2GB of RAM if you're using scale-out NAS. You'll also want to assign the VM template a virtual disk to be used as pool storage and a 10GB or larger disk to be used for the boot/system disk. Last, you'll need a unique license key block for each VM instance so you'll need to fill out the Try Now form multiple times to get the keys you'll need, one per appliance.

Software RAID1 for Boot/System Disk Configuration

Software RAID for the boot device is not recommended. We recommend using two boot drives in hardware RAID1 as that is by far the easiest configuration to setup, maintain, and monitor through the QuantaStor web management interface. That said, it is possible to setup QuantaStor with mirrored boot drives using the mdraid software RAID driver using the general Ubuntu Linux documentation available online. If you do so you'll need to make sure that QuantaStor doesn't recognize the MDRAID swap and boot devices as storage pools '0' and '1', and this is done by creating a file called /etc/qs_poolblacklist.conf.

/etc/qs_poolblacklist.conf

The contents of the file should look something like this when you're done editing it:

UUID=28d9d8b6-9fcc-4a66-9993-efc90c88f121
UUID=c2eec0d4-24da-4366-93e4-919f02c92198

To get the UUID of the storage pools you can get the information from the command 'qs pool-list' or you can get the information from 'qs-showlog' and look for the warning messages around the storage pools that it could not start. After you've edit the file you'll need to restart the QuantaStor management service like so: service quantastor restart

Install Process for QuantaStor Virtual Storage Appliances (VSA)

Installing to VMware Workstation 9 (Virtual Storage Appliance)

There's really just one key step to successful installation on VMware Workstation 9 and that's to *not* use the Easy Installer. The Easy Installer will completely bypass the QuantaStor custom installation steps and will result in installing just plain Ubuntu Server. To ensure that the QuantaStor custom installation steps are run you must not use 'Easy Installer'. To bypass 'Easy Installer' simply choose 'I will install the operating system later.' and the virtual machine will be created with a blank hard disk. After you have created the VM instance you can then 'Edit Virtual Machine Settings' to point it to your QuantaStor CDROM or ISO image and begin the install. Here are a series of screenshots to illustrate the installation process:

Step 1.
Step 2.


Step 3. This is a key step. If you choose one of the first two it will try to use 'Easy Install' mode which will bypass the QuantaStor installation components. As such you must choose 'I will install the operating system later.'.
Step 4. Choose 64bit Ubuntu Linux from the list of available operating systems. (n.b. QuantaStor builds on top of the Ubuntu 64bit server Linux distribution.)


Step 5.
Step 6.


Step 7. 1GB of RAM is sufficient for doing basic things with a QuantaStor virtual storage appliance but for real production use you should allocate 2GB, 4GB or more.
Step 8. The installation will use DHCP to get an IP address and will connect to the net to determine your Time Zone. You'll also want to be able to connect to this VM via iSCSI from the host workstation so Bridged Networking is necessary.


Step 9. The boot device need only be 8GB in size but I just kept with the VMware default of 20GB. For production use, 20GB for the boot/system drive is best.
Step 10. SCSI is always best for the drive interface as it provides additional information and a unique ID by which QuantaStor can identify the device.


Step 11. You can use a single file or multiple files, I choose single file for simplicity.
Step 12.


Step 13.
Step 14.


Step 15.
Step 16.







Step 17.
Step 18.


Step 19. This disk drive is for creating a storage pool so we create a device here so later we can create 'pool0' within QuantaStor.
Step 20.


Step 21.
Step 22.


Step 23.
Step 24.



Note that the default console user is 'qadmin' and the default password is 'qadmin'. The console account is completely separate from the QuantaStor web management interface account which is 'admin' with the initial password of just 'password'.

Installing to VMware ESXi (Virtual Storage Appliance)

Using VMware with ESXi requires some specific settings to be chosen as part of the VM creation process. In this section we'll go step by step through each screen the VM creation wizard in the vSphere Client so the overall process is a snap.


Step 1. This first screen shows the available operating system selections, since QuantaStor is based on Ubuntu Server (64bit) we choose that from the drop down list of Linux OS types.
Step 2. Next we're asked to make a boot/system disk for the new virtual appliance. We suggest 6GB or more for your virtual machine, but more than 32GB is unnecessary unless you plan to install custom applications on your virtual storage appliance.



Step 3. Next you'll want to check that box that says 'Edit the virtual machine settings before completion.' as we have a number of things to edit before you can boot the system to start the installation. Once you check the box, press the 'Continue' button and you'll then be able to edit the virtual machine settings.
Step 4. At this point you'll be presented with the 'Virtual Machine Properties' dialog. In here there are a few things that you need to make sure of. First, make sure that you change the SCSI controller from LSI Parallel to LSI Logic SAS.










Step 5. Next, select the network adapter and make sure 'Connect at power on' is checked. This will allow the installer to access the Ubuntu package servers during the installation and will also configure your network via DHCP. You'll want to change this to a static address later but it is very convenient to have the system use DHCP for the initial installation phases.
Step 6. Next you'll need to select the ISO file for where you're keeping your osn_quantastor_vX.Y.Z.iso ISO file. This must also be 'Connected at power on' so that it can boot from the ISO to start the installation process.




The default amount of memory given to the VM is only 512MB which is insufficient. QuantaStor has an internal web server which uses a fair bit of memory and additional memory is used as a buffer cache to accelerate read performance. If you have it, set this to 2GB or more, else set it to at least 1GB of RAM.


This is what it should look like when you're all done. Go ahead and click finish and you can start your VM to start the installation process.



Installing to Oracle VirtualBox (Virtual Storage Appliance)

Getting a QuantaStor storage system setup as a virtual appliance is one of the easiest ways to get started with QuantaStor to test out the features and get a feel for the product. To install QuantaStor as a VirtualBox virtual appliance you'll first need to download the virtual appliance from our downloads page, and you'll need to install the latest version of VirtualBox from their website http://www.virtualbox.org.

Once you have the quantastor_vsa_ovf.zip file downloaded you'll need to unzip the file. In there you will find a file with a .ovf extension and some associated files, that's what you'll be importing as a virtual appliance. Now you'll need to startup VirtualBox and select 'Import Appliance..' from the 'File' menu. Next you'll select the .ovf file that you unzipped from the quantastor_vsa_ovf.zip in the previous step. That's all there is to it. Note that by default QuantaStor is setup to use Bridged Networking so that your hosts can access the exported iSCSI targets exposed by the VM.

Adding disks to your QuantaStor Virtual Storage Appliance

Step 1. Before starting your QuantaStor VSA VM you'll want to first add some storage to it so that you have some storage with which to create storage pools. Bring up the settings panel for your VM so that you can add more storage.


Step 2. Our testing has found that virtual storage disks hosted on a virtual SAS Controller is much more reliable than the default SATA controller. The first step will be to add this SAS controller as shown in this screen capture.


Step 3. Once the SAS controller is available, a new disk can be added to it using the button highlighted below. We recommend that you attach at least a 40GB disk to the virtual appliance so that you have enough disk space to create a storage pool that's large enough to provision storage volumes that are 8GB in size to install Windows, Linux, and/or applications onto.



Configuring your QuantaStor VSA VirtualBox appliance

Now that you have one or more disks attached to your imported appliance, all you need to do now is to start it up and configure it as per the general Getting Started Guide.

Installing to Citrix XenServer v6.x (Virtual Storage Appliance)

Installing to Microsoft Hyper-V (Virtual Storage Appliance)

Running QuantaStor as a virtual storage appliance (VSA) can be done by installed from ISO, but there's a few configuration changes that you'll need to make after you create the virtual machine.

One change is to add a Legacy Network Adapter as the installation process will not be able to access the network using the standard adapter. To do this navigate to the VM's settings. In the 'Add Hardware' section, add a 'Legacy Network Adapter'. Now configure this to your choosing. Then select the network adapter created from the create VM wizard, and remove it.

LegacyAdapterSetting.png

The next change is during the installation of QuantaStor. When the iso is first being loaded hit the 'f6' key when on the language selection screen (you may have to hit it before you see the screen as Hyper-V sometimes doesnt load the screen in time before the language is selected). This will bring up an options screen. Hit 'f6' again to bring up 'Other Options', and then hit escape to close it. This step is to bring up the boot options string that we will now modify. In the boot options string that is listed remove 'nodmraid' and 'acpi=off'. You will also want to add the field 'vga=788' to fix some graphical issues that occur sometimes. Now hit enter to start the installation.

The options on the left should be changed to look like the options on the right.

OldOptions.png NewOptions.png

You can now install QuantaStor onto your virtual machine just as you would any other operating system. Note that you should allocate at least 1GB of RAM to your QuantaStor VM and you should allocate at least a 30GB + the number of GB in RAM virtual hard drive for use as your boot/system disk. Example: A system with 32GB in ram should have atleast 62GB in hard drive space.


Here are some of the problems you may encounter if you do not make these changes. As you can see the graphics might not be displayed properly, and the disks may not be discovered.

BadVideo.png NoDisk.png

Installing to an Amazon EC2 instance

The following steps summarize the process for installing QuantaStor in the Amazon EC2 cloud.

Selecting an Amazon Machine Image

We currently do not have an Amazon Machine Image for QuantaStor so the installation procedure requires upgrading a stock Ubuntu Server 10.10 image to use as a starting point. You'll find these in the Community AMI images, the one we used was AMI: ebs/ubuntu-images/ubuntu-maverick-10.10-amd64-server-20101225 (ami-cef405a7)

EC2 Firewall Configuration

Be sure to open up the TCP port 3260 to your QuantaStor system so you can access your iSCSI devices from other machines in the cloud. Next you'll need to open up the TCP port 22 on your QuantaStor system so you can login to the SSH to the console. You'll also need access to port 22 for remote replication. Finally, open up port 80 and/or 443 so that you can access the QuantaStor web management interface, and if you're going to be replicating between a local QuantaStor instance and your Amazon Cloud instance you'll need to open TCP port 5151. In summary, open the TCP ports 80, 443, 5151, & 3260.

EC2 PEM Files / SSH Login

When you create and download your .pem key file, remember to use PuttyGen to convert the .pem key into a .ppk private key that you can load into Pageant, after that you'll be able to login to the system as 'ubuntu'. If you're not familiar with PuTTY, it is a free SSH client for Windows that allows you to login to the QuantaStor console remotely over a secure connection. You'll need to login to your Ubuntu Server Amazon Machine Image using SSH in order to upgrade/convert it to a QuantaStor system. PuTTY is available here [1]:

Upgrading the Amazon Machine Image

To install QuantaStor onto your Amazon machine image after you have it created you'll need to follow a few steps in our Upgrade Guide here: [2]

Installing the QuantaStor Kernel

After all the packages are installed you'll need to switch the kernel over to our custom kernel by modifying the /boot/grub/menu.lst file. In there, you'll want to remove the entry for the older non-quantastor kernel. Final note, sometimes you'll see a longer delay when creating the Storage Pool in the Amazon cloud that you would normally see in a local system. That's

Troubleshooting the installation

Troubleshooting Hardware RAID Boot Configuration Issues

Installing to an LSI MegaRAID, Adaptec, or LSI 3ware controller

It is highly recommended that QuantaStor be installed to a mirrored (RAID1) pair of system disks so that no downtime is experienced in the event of a disk failure of the boot/system disk. The underlying Linux operating system on which QuantaStor builds has support for software RAID which could accomplish this but the recovery scenarios would be somewhat complex. As such the most efficient route is to use a hardware RAID mirror using the same RAID controller that you're using for your storage pools. Note that currently you must set aside two full disks for QuantaStor's system disks in order to make a RAID1 mirror to install too. QuantaStor has very limited storage needs so two small capacity 250GB drives will suffice for the boot/system drive (any type will work HDD/SSD and SAS/SATA). For deployment as a Virtual Storage Appliance the QuantaStor OS needs only a modest 30GB plus the amount of GB in RAM of disk space, though 50-100GB or more is recommended for larger configs.

Creating the RAID1 Mirrored System Disk

The first step will be to use the RAID controller's BIOS based configuration system to create a RAID1 mirror using two disks. With the 3ware controllers you can enter the BIOS by pressing Alt-3 during boot, and with the MegaRAID you can enter the WebBIOS by pressing Ctrl-H during boot. Please refer to the vendors documentation for complete details on how to create a RAID1 mirror using a pair of disks. For MegaRAID this will involve creating a Disk Group 0 with both disks and then a single Span with a single Virtual Drive.

Loading a Custom Built Driver During Install

With newly released hardware you may reach a phase of the installation where it is asking for you to load an external RAID driver. We recommend getting assistance from support@osnexus.com so that we can review your hardware and firmware levels and to ensure that QuantaStor contains the latest version of the driver for your hardware.

Details of how to load a custom driver at Install time

At the phase where it asks to load a custom driver for your controller press 'ESC' and then choose 'Open Command Window' at the prompt. Next you'll need to mount and navigate to where your driver is located. For example:

cd /usbmedia/drivers/quantastor4/
insmod driver.ko

At this point the driver is loaded an you should be able to now detect your hardware RAID mirror disk and install QuantaStor to it. To do this you'll first exit the command shell.

exit

And now you'll just need to choose 'Detect Disks' from the menu and the installation process will continue.

Problem: It won't boot to the Installation CD

  • This is typically a BIOS configuration problem. You'll need to check the BIOS boot order to make sure that your CD or DVD drive is the first drive in the BIOS boot ordering. After that be sure to save your new BIOS configuration before rebooting. We have seen cases where the boot order is ignored and you will need to make the CD-ROM or USB CD-ROM the only boot device. In such cases you'll have to reconfigure the BIOS to boot from hard drive after the installation is complete.
  • This could also be caused by a bad CD. Try burning a new CD from the QuantaStor ISO file and check the surface of the disk to make sure there are no scratches.

Problem: Installation works fine but after I reboot the screen is black

  • One of the first things to check is your PCIe Boot Option ROM configuration. Most PCI cards from FC cards to NICs or RAID controllers all can be used to boot an OS. As such if you have multiple cards the motherboard can get confused strange things happen. To address this, look at your motherboard manual and disable the Option ROM on all of the PCIe ports except for the one containing the RAID controller that you're booting your QuantaStor system with.
  • This can also be caused by the 'boot' flag missing on your designated RAID unit for the QuantaStor OS. The installation can complete successfully even when this is not set correctly so you'll only see this problem after the installation completes and it doesn't boot. Be sure to verify that the boot flag is set on the proper device in the RAID controller BIOS and that that device is selected as the first boot device in your motherboard BIOS.

Problem: It asks me to insert a CD-ROM driver

  • You can skip that step and just continue on selecting the defaults ('none') and it will typically find the CD-ROM. This condition can occur when the system has multiple CD-ROM drives (or a virtual CD-ROM emulated by the BIOS). In some cases you may only have one CD-ROM drive but the BIOS is emulating a second drive hence the confusion to the installer.

Problem: It won't boot up after the installation completes

  • Check your BIOS boot order to make sure that the hard disk you installed the system to is now the first item in the BIOS boot order.
  • If that doesn't work check in the BIOS for the part that says 'Plug n' Play OS Support', and make sure that is set to 'No' or 'Disabled', then reboot.
  • If that doesn't work it may be due to an on-board SATA RAID contoller. Some motherboards have on-board SATA RAID (ASUS, others), sometimes referred to as 'fake RAID' and some of these can cause problems while others work better when that SATA RAID firmware is turned on. Go into your BIOS and look under the 'IDE Configuration' page. In there you will typically have three choices, 'AHCI', 'IDE' and 'RAID'. Generally speaking, QuantaStor works best when you have it set to 'IDE'. That said, we've seen some systems that work better when set to 'RAID' mode, but be sure to not create any RAID sets with that hardware BIOS. If you have that turned on but have no RAID sets created through BIOS then the devices will just pass-thru to QuantaStor. There is a related problem with on-board RAID controllers that can occur when trying to create storage pools that you will want to check here.

Problem: Install fails part way through

  • Try selecting 'F6' immediately when on the language selection screen appears. This will bring up an options screen. Hit 'F6' again to bring up 'Other Options', and then hit escape to close it. This step is to bring up the boot options string that we will now modify. In the boot options string that is listed remove 'nodmraid' and 'acpi=off'. You will also want to add the field 'vga=788' to fix some graphical issues that occur sometimes. Now hit enter to start the installation.

Problem: Install fails with error 'Select and Install Software'

  • Prior to 1.5 you needed to make sure that the system you are installing QuantaStor on has access to the internet. There are parts of the installation process with v1.4 where it will try to update the package repository using the Ubuntu servers and if you have a firewall or web filter in place blocking the Ubuntu repository sites you will have installation problems that shows up as an error in the 'Select and Install Software' stage.
  • Issues have also been seen with USB CD/DVD drives where the USB controller will go offline partway through the install. If you're seeing the install fail intermittently at different stages this is probably the issue, that or you have a bad CD.
    • If it's a bad CD, you can burn a new one or check the MD5 hash to make sure it matches the one posted in our Change Log.
    • If it's the USB controller then we recommend physically unplugging the power from your server for 10 seconds so that the motherboard fully powers down, then plug the power back in and reinstall. Powering off the motherboard seems to clear the on-board USB controller state and subsequently has resolved this installation problem on the HP server gear that we were able to reproduce it on.

Problem: My system disk doesn't show up in the list of disks to install the QuantaStor system onto.

  • The most common problem here is partitioning. If the disk(s) you want to use for creating a storage pool have LVM partitions on them then QuantaStor will automatically filter them out to prevent you from inadvertently stomping on it. The way to fix this is to use fdisk at the console to delete the partitions off the device. We're looking at adding a 'Scrub Disk' command to remove the partitions off of such disks so that they can be prepared for use without having to deal with fdisk.
  • This could also be a cabling problem. Some motherboards have multiple SATA/SAS controllers on-board. Try connecting your SATA/SAS boot drive to another SATA/SAS port and/or separate controller.

If all else fails, try installing plain Ubuntu Server 10.10 on the system and then use the upgrade guide to install the QuantaStor packages. This is a great way to triage problems since if Ubuntu doesn't install and run then there's a problem that's not specific to the QuantaStor installer. That typically points to a BIOS configuration or a cabling problem.