1:下载好jdk1.8的安装包,并解压到到一个目录(本人命名为 /opt/module/)
2:切换root用户,在 /ect/profile.d/ 下面创建一个 xxx.sh文件 (本人设定为 my_env.ssh),该文件可以用于存放自己配置的环境变量
3:切记,最后一定要执行 source /etc/profile 命令
说明:主机如下
hadoop112 192.168.142.112
hadoop113 192.168.142.113
hadoop114 192.168.142.114
1.1:xsync脚本分发文件,在用户家目录下面创建一个bin目录,这里我们存放脚本文件,vim xsync 然后粘贴下面代码块,只需改动主机名称即可。切记创建完脚本后执行 chmod 777 xsync(xsync为脚本名字),增加执行权限
#!/bin/bash
#1 获取输入参数个数,如果没有参数,直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi
#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname
#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir
#4 获取当前用户名称
user=`whoami`
#5 循环 host为主机编号,主机名为hadoop112,hadoop113,hadoop114
for((host=112; host<115; host++)); do
echo ------------------- hadoop$host --------------
rsync -rvl $pdir/$fname $user@hadoop$host:$pdir
done
1:执行命令 ssh-keygen -t rsa 后面一直回车即可,执行完在 /home/atguigu 目录下会生成 .ssh 文件 ,其中id_rsa为私钥,id_rsa.pub为公钥
2:执行命令 ssh-copy-id hadoop112 ssh-copy-id hadoop113 ssh-copy-ihadoop114 然后依次输入密码
3:注意,仅对执行命令的用户进行免密
hadoop102 |
hadoop103 |
hadoop104 |
|
HDFS |
NameNode DataNode |
DataNode |
SecondaryNameNode DataNode |
YARN |
NodeManager |
ResourceManager NodeManager |
NodeManager |
1: vim core-site.xml
fs.defaultFS
hdfs://hadoop112:9000
hadoop.tmp.dir
/opt/module/hadoop-2.7.2/data/tmp
2: vim hadoop-env.sh yarn-env.sh mapper-env.sh,修改如下内容
export JAVA_HOME=/opt/module/jdk1.8.0_144
3: vim hdfs-site.xml
dfs.replication
3
dfs.namenode.secondary.http-address
hadoop114:50090
4: vim yarn-stie.xml
yarn.nodemanager.aux-services
mapreduce_shuffle
yarn.resourcemanager.hostname
hadoop113
5 : 复制mapper-site.xml cp mapred-site.xml.template mapred-site.xml 然后添加如下内容
mapreduce.framework.name
yarn
6:vim slaves 设置集群是那几台主机
hadoop112
hadoop113
hadoop114
7: 分发hadoop配置 xsync /opt/module/hadoop
8: 初始化主节点 bin/hdfs namenode -format
9:集群启动脚本 vim /home/atguigu/bin/myhadoop.sh
#!/bin/bash
if [ $# -lt 1 ]
then
echo "No Args Input..."
exit ;
fi
case $1 in
"start")
echo " =================== 启动 hadoop 集群 ==================="
echo " --------------- 启动 yarn ---------------"
ssh hadoop113 "/opt/module/hadoop-2.7.2/sbin/start-yarn.sh"
echo " --------------- 启动 historyserver ---------------"
ssh hadoop112 "/opt/module/hadoop-2.7.2/bin/mapred --daemon start historyserver"
;;
"stop")
echo " =================== 关闭 hadoop 集群 ==================="
echo " --------------- 关闭 yarn ---------------"
ssh hadoop113 "/opt/module/hadoop-2.7.2/sbin/stop-yarn.sh"
echo " --------------- 关闭 hdfs ---------------"
ssh hadoop112 "/opt/module/hadoop-2.7.2/sbin/stop-dfs.sh"
;;
*)
echo "Input Args Error..."
;;
esac
在浏览器输入 192.168.142.112:50070显示如下