Cloudera(tm) Hadoop Integration Guide
To get Cloudera Hadoop running within your QuantaStor system the steps are basically the same as they would be with a standard Ubuntu Precise server. Essentially you'll need to login to your system using SSH or use the console then run these commands.
Step 1 - Add the Cloudera package server
Run this command to add the GPG key for Cloudera's packages.
sudo -i curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
Next you'll need to tell QuantaStor where the Cloudera package servers are located by creating a sources file called /etc/apt/sources.list.d/cloudera.list with the following contents:
deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib
You can create the above file using nano or vi or you can create this file with the necessary content using a couple of echo commands:
echo "deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib" > /etc/apt/sources.list.d echo "deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib" >> /etc/apt/sources.list.d
Now we can update the local system's package repository database like so:
apt-get update
Finally, we're ready to install Hadoop and you can find those instructions here:
Generally speaking you'll run your name nodes and job trackers on separate servers from QuantaStor where QuantaStor nodes are running as just data nodes like so:
sudo apt-get install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode
That's the basics of getting Hadoop running with QuantaStor. We are looking at doing some deeper integration with Hadoop this year and would appreciate your feedback on what would help you most in terms of integration features. So if you have some ideas, please write Steve at steve (at) osnexus.com.
