环境:redhat 4 三台、jdk 6u21、hadoop-0.20.2
(1)用root登陆,建立hadoop用户
useradd hadoop
passwd hadoop
(2)分别在四台机器上运行二进制文件:./jdk-6u21-linux-x64-rpm.bin
(3)编辑四台机器的/etc/profile,增加如下内容
export JAVA_HOME=/usr/jdk1.6.0_21
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=
$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
(4)以root登录,在master和 slave1上分别修改/etc/hosts,添加如下内容:
192.168. 0.100 master
192.168.0.101 slaveA
192.168.0.102 slaveB
(5)Hadoop的安装,将下载好的hadoop-0.20.2.tar.gz文件上传到master
tar xvfz hadoop-0.20.2.tar.gz
配置/etc/profile
export HADOOP_HOME=/home/hadoop/hadoop-0.20.2
export PATH=$PATH:$HADOOP_HOME/bin
(6)修改Hadoop的配置文件
masters文件里添加如下内容:
192.168.0.100
slaves文件添加如下内容:
192.168.0.101
192.168.0.102
hadoop-env.sh中添加如下语句:
export JAVA_HOME=/usr/jdk1.6.0_21
core-site.xml添加如下语句在<configuration></configuration>之间:
<property>
<name>fs.default.name</name>
<value>hdfs:// 192.168.0.100:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
</property>
mapred-site.xml添加如下语句在<configuration></configuration>之间:
<property>
<name>mapred.job.tracker</name>
<value>192.168.0.100:9001</value>
</property>
hdfs-site.xml添加如下语句在<configuration></configuration>之间,
replication 默认为3,如果不修改,datanode 少于三台就会报错
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
(7)配置好的hadoop发送到salves上:
scp -r /home/hadoop/hadoop-0.20.2 slaveA:/home/hadoop
(8)设置ssh密钥每个机器上都要如此
su - hadoop
ssh-keygen -t rsa
hadoop生成其密钥对,询问其保存路径时直接回车采用默认路径,当提示要为生成的密钥输入passphrase的时 候,直接回车,也就是将其设定为空密码。生成的密钥对id_rsa,id_rsa.pub,默认存储在/root/.ssh目录下
chmod 755 .ssh #赋予755权限
cd .ssh
#ls –l
id_rsa id_rsa.pub
master上执行:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys#把公用密匙添加到authorized_keys 文件中(此文件最后一定要赋予644权限)
ssh slaveA cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh slaveB cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys slaveA:~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys slaveB:~/.ssh/authorized_keys
(9)在master上连接所有slave节点,输入如下命令,上有几个slave节点就要输入几次,注意每次输入后,要返回master节点,重新输入命令:
ssh slave1 #需要选择yes/no 选择yes即可
此时 ls .ssh 结果如下:
authorized_keys id_rsa id_rsa.pub known_hosts
ssh slaveA 不需要输入密码就算配置成功
(10)./hadoop namenode -format #格式化文件系统
./start-all.sh #启动namenode上的hadoop
确保9001和9000端口是通的。
jps 查看是否启动正常!
查看50070,54310,50030端口是否启动 或者远程访问如下链接
#1080
#50030
#50060
netstat -tnl
http://master:50070访问
http://master:50030