Configuring hive with hadoop

1. Infinity
dwivedishashwat@gmail.com
Shashwat Shriparv
Hive with Hadoop configuration document

2. Requirements
• Java 1.6
• Hadoop 0.20.x.
Installing Hive from a Stable Release
Start by downloading the most recent stable release of Hive from one of the Apache download
mirrors :
http://hive.apache.org/releases.html
We can directly download from site or issue command through terminsl to download as follows
:
wgethttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gz
After downloading complete issue following command :
tar -xzf hive-0.8.1.tar.gz
open hive-0.8.1/conf/hive-env.sh
# Set HADOOP_HOME to point to a specific hadoop install directory

3. export HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205
export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf
# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf
# Folder containing extra ibraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-
0.20.205/lib:/home/shashwat/Hadoop/hbase-0.90.4/lib:/home/shashwat/Hadoop/hive-
0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar

4. Then open hive-site.xml, in this file you will find a property called “hive.aux.jars.path” as
follows :
<property>
<name>hive.aux.jars.path</name>
<value>file:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-
0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-
0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar</value>
</property>
Version compatibility with hadoop and hive :
Hive
Version
Hadoop
Version
Configuration Parameters
0.4 0.18 --hadoop-version 0.18
0.5 0.20
--hadoop-version 0.20 --hive-versions 0.5 --
ami-version 1.0
0.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7

5. Hive
Version
Hadoop
Version
Configuration Parameters
--ami-version 1.0
0.7 0.20
--hadoop-version 0.20 --hive-versions 0.7 --
ami-version 1.0
0.7.1 0.20
--hadoop-version 0.20 --hive-versions 0.7.1 -
-ami-version 1.0
0.7.1 0.20.205
--hadoop-version 0.20 --hive-versions 0.7.1 -
-ami-version 2.0
0.7.1.1 0.20.205
--hadoop-version 0.20.205 --hive-versions
0.7.1.1 --ami-version 2.0
0.7.1.2 0.20.205
--hadoop-version 0.20.205 --hive-versions
0.7.1.2 --ami-version 2.0
0.7.1.3 0.20.205
--hadoop-version 0.20.205 --hive-versions
0.7.1.3 --ami-version 2.0
0.7.1.4 0.20.205
--hadoop-version 0.20.205 --hive-versions
0.7.1.4 --ami-version 2.0
After configuration complete issue these commmands from terminal :
bin/hive - -service hiveserver : it will start the thrift server :
then open new terminal and issue commmand
bin/hive : this will give you hive terminal where you can execute commands like
create table
show tables etc.
More configuration options
Hive Configuration Variables used to interact with Hadoop
Variable Name Description Default Value
hadoop.bin.path
The location of hadoop script
which is used to submit jobs to
hadoop when submitting through a
separate jvm.
$HADOOP_HOME/bin/hadoop
hadoop.config.dir
The location of the configuration
directory of the hadoop installation
$HADOOP_HOME/conf