搭建Hadoop1.2.1集群

环境：
三台CentOS6.5虚拟机，机器名分别为hadoop1、hadoop2、hadoop3，通过桥接连网，互相可以ping通，均安装好JDK1.7，并关闭iptables防火墙
[root@hadoop1 ~]$ service iptables stop
[root@hadoop1 ~]# chkconfig iptables off

一、配置hosts文件
[root@hadoop1 ~]# vi /etc/hosts
127.0.0.1   localhost
::1         localhost
192.168.0.104    hadoop1
192.168.0.103    hadoop2
192.168.0.105    hadoop3
hadoop2、hadoop3上也做类似设置

二、给每个虚拟机建立hadoop运行账号
[root @localhost ~]# useradd -s /bin/sh -d /home/grid -m grid
[root @localhost ~]# passwd grid

三、配置ssh免密码连入（本机到本机的免密码也需要配置）
生成密钥：
[grid@hadoop1 ~]$ ssh-keygen -t rsa
直接三次回车生成公钥和私钥，hadoop2、hadoop3同样如此
[grid@hadoop1 ~]$ chmod 700 ~/.ssh

分发密钥：
[grid@hadoop1 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop2:.ssh/id_rsa.pub1
[grid@hadoop1 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop3:.ssh/id_rsa.pub1
[grid@hadoop2 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop1:.ssh/id_rsa.pub2
[grid@hadoop2 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop3:.ssh/id_rsa.pub2
[grid@hadoop3 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop1:.ssh/id_rsa.pub3
[grid@hadoop3 ~]$ scp ~/.ssh/id_rsa.pub grid@hadoop2:.ssh/id_rsa.pub3

将公钥写入到authorized_keys文件中：
[grid@hadoop1 ~]$ touch ~/.ssh/authorized_keys;chmod 644 ~/.ssh/authorized_keys
[grid@hadoop1 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys;cat ~/.ssh/id_rsa.pub2 >> ~/.ssh/authorized_keys;cat ~/.ssh/id_rsa.pub3 >> ~/.ssh/authorized_keys

四、在一台虚拟机中下载并解压hadoop安装包
[grid@hadoop1 ~]$ wget http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz
[grid@hadoop1 ~]$ tar -zxf hadoop-1.2.1-bin.tar.gz

五、配置namenode，修改site文件
修改core-site.xml，指定NameNode主服务器的地址、临时文件目录：
[grid@hadoop1 ~]$ vi hadoop-1.2.1/conf/core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/grid/hadoop-1.2.1/tmp</value>
</property>
</configuration>

创建临时文件目录
[grid@hadoop1 ~]$ mkdir -p /home/grid/hadoop-1.2.1/tmp

修改hdfs-site.xml文件，指定复制因子：
[grid@hadoop1 ~]$ vi hadoop-1.2.1/conf/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>

修改mapred-site.xml文件，指定JobTracker主服务器地址及端口
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>hadoop1:9001</value>
</property>
</configuration>

六、配置hadoop-env.sh
[grid@hadoop1 ~]$ vi hadoop-1.2.1/conf/hadoop-env.sh
修改JAVA_HOME路径

七、配置masters和slaves文件（masters中设置的是secondary namenode节点）
[grid@hadoop1 ~]$ vi hadoop-1.2.1/conf/masters
hadoop1
[grid@hadoop1 ~]$ vi hadoop-1.2.1/conf/slaves
hadoop2
hadoop3

八、向各节点复制hadoop
[grid@hadoop1 ~]$ scp -r ~/hadoop-1.2.1 grid@hadoop2:~

九、在masters节点上格式化namenode
[grid@hadoop1 ~]$ hadoop-1.2.1/bin/hadoop namenode -format

十、启动hadoop
[grid@hadoop1 ~]$ hadoop-1.2.1/bin/start-all.sh

十一、用jps检验各后台进程是否成功启动
[grid@hadoop1 ~]$ /usr/java/jdk1.7.0_72/bin/jps
2485 JobTracker
2637 Jps
2410 SecondaryNameNode
2259 NameNode
[grid@hadoop2 ~]$ /usr/java/jdk1.7.0_72/bin/jps
2128 DataNode
2281 Jps
2191 TaskTracker
[grid@hadoop3 ~]$ /usr/java/jdk1.7.0_72/bin/jps
2185 TaskTracker
2293 Jps
2123 DataNode