hadoop集群配置

1、准备工作
三台机器
report1.ka.hadoop.corp.koubei.net(master)
report2.ka.hadoop.corp.koubei.net(slave)
report3.ka.hadoop.corp.koubei.net(slave)
http://www.apache.org/dist//hadoop/core/hadoop-0.18.3/
http://www.apache.org/dist//hadoop/hive/hive-0.5.0/
2、SSH配置
Hadoop需要namenode到datanode的无密码SSH登录
# cd /root/.ssh/
# ssh-keygen -t rsa
# touch authorized_keys
# cat id_rsa.pub >>authorized_keys
将report1 ./.ssh/id_rsa.pub内容追加 report2、report3  authorized_keys文件末尾
3、jdk安装配置
由于redhat中自带安装了jdk1.4的，所以在安装jdk1.6前先把jdk1.4卸载
#rpm -qa | grep gcj
#rpm -e --nodeps java-1.4.2-gcj-compat-1.4.2.0-40jpp.115
# ./jdk-linux.bin
.bash_profile
export JAVA_HOME=/opt/jdk1.6.0_21
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
# source .bash_profile
4、hadoop安装配置
#cd /opt
#tar -xzvf hadoop-0.18.3.tar.gz
.bash_profile
export HADOOP_INSTALL=/opt/hadoop-0.18.3
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin
hadoop-env.sh
export JAVA_HOME=/opt/jdk1.6.0_21
hadoop-site.xml
<configuration>
  <property>
        <name>fs.default.name</name>
        <value>hdfs://report1.ka.hadoop.corp.koubei.net</value>
  </property>
  <property>
        <name>dfs.replication</name>
        <value>2</value>
  </property>
  <property>
        <name>mapred.job.tracker</name>
        <value>report1.ka.hadoop.corp.koubei.net:8021</value>
  </property>
</configuration>
report1.ka.hadoop.corp.koubei.net配置namenode及datanode
masters
report1.ka.hadoop.corp.koubei.net
slaves
report2.ka.hadoop.corp.koubei.net
report3.ka.hadoop.corp.koubei.net
# hadoop namenode -format
#hadoop fs -ls /tmp
# hadoop fs -put test.txt hdfs://report1.ka.hadoop.corp.koubei.net/tmp/test.txt
5、hive安装配置
# tar -zxvf hive-0.5.0-dev.tar.gz
.bash_profile
export HIVE_HOME=/opt/hive-0.5.0-dev
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_INSTALL/bin:$HIVE_HOME/bin
#source .bash_profile