Hadoop集群搭建
Hadoop集群搭建
一、Linux系统配置
1、在所有的机器上都建立相同的目录，也可以就建立相同的用户，以该用户的home路径来做hadoop的安装路径。例如在所有的机器上都建立了名为fan的用户：/home/fan。
etc/hosts文件末尾添加IP-主机映射关系
2、rysnc
vi/etc/rysnc.conf
rsync--daemon
客户端：
rsync -avzP mjiang@venus::hadoop hadoop-1.0.0/etc/hadoop/
2、ssh 免密码登录
http://blog.csdn.net/j3smile/article/details/7259963
附录1：
#!/bin/bash
filename=/home/mjiang/.ssh/authorized_keys
slaves=(mars jupiter saturn)
for i in ${slaves[@]}
do
echo $i
scp $filename mjiang@$i:$filename
done
3、安装hadoop，借助rysnc
tar zxvf hadoop-1.0.0-bin.tar.gz
4、安装jdk，借助rysnc
su tar zxvf jdk-7u1-linux-i586.tar.gz -C /usr/share/
5、配置文件修改
http://hadoop.apache.org/common/docs/r0.19.2/cn/cluster_setup.html
vi hadoop-1.0.0/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/share/jdk1.7.0_01
chmod+x hadoop-1.0.0/sbin/*
6、开始
./sbin/stop-all.sh
stoppingjobtracker
mars: stoppingtasktracker
jupiter:stopping tasktracker
saturn:stopping tasktracker
no namenodeto stop
（如果出现了这个那么就会出现
mjiang@venus~/hadoop-1.0.0 $ hadoop fs -ls /
12/02/1516:31:32 INFO ipc.Client: Retrying connect to server:venus/211.83.105.12:9000. Already tried 0 time(s).）
mars: stoppingdatanode
jupiter:stopping datanode
saturn:stopping datanode
venus:stopping secondarynamenode
hadoopnamenode -format
12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name has been successfully formatted.
./sbin/start-dfs.sh
..12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name has been successfully formatted....
./sbin/start-mapred.sh
在客户端：
mjiang@mars~ $ hadoop fs -mkdir /user/mjiang
mkdir: cannotcreate directory /user/mjiang: File exists
－－－是同步的