大数据学习之Hadoop+Zookeeper

安装环境

• 共五个节点
  namenode
  namenode2
  datanode1
  datanode2
  datanode3

• 关闭防火墙，selinux，增加hadoop账户，免密登录，

一、Zookeeper

zookeeper 最少需要3个节点，一个leader，两个follower，

zookeeper安装

• 安装包解压到：/usr/local/zookeeper
• 环境变量：vim /etc/profile

export JAVA_HOME=/usr/local/jdk1.8
export CLASSPATH=.:${JAVA_HOME}/jre/lib/rt.jar:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar
export PATH=$PATH:${JAVA_HOME}/bin

export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$ZOOKEEPER_HOME/bin:$PATH

• 修改配置：

cp /usr/local/zookeeper/conf/zoo_sample.cfg{,.bak}
vim /usr/local/zookeeper/conf/zoo.cfg

dataDir=/usr/local/zookeeper/zkdata
dataLogDir="/usr/local/zookeeper/zklog"

server.1=hadoop03:2888:3888
server.2=hadoop04:2888:3888
server.3=hadoop05:2888:3888

echo 1 /usr/local/zookeeper/zkdata/myid

• 三个节点重复以上操作后启动
  启动三个节点：zkServer.sh start
  查看节点状态：zkServer.sh status

一个leader 两个flollower即正常。jps命令会显示QuorumPeerMain进程。

• zookeeper批量启动脚本

#! /bin/bash
hosts=(hadoop03 hadoop04 hadoop05)
if [ $1 == "start" ]; then
  for i in ${hosts[@]}
    do
      ssh hadoop@$i "source /etc/profile;zkServer.sh start" &
    done

elif [ $1 == "stop" ]; then
  for i in ${hosts[@]}
    do
      ssh hadoop@$i "source /etc/profile;zkServer.sh stop" &
    done

elif [ $1 == "restart" ]; then
  for i in ${hosts[@]}
    do
      ssh hadoop@$i "source /etc/profile;zkServer.sh restart" &
    done

elif [ $1 == "status" ]; then
  for i in ${hosts[@]}
    do
      ssh hadoop@$i "source /etc/profile;zkServer.sh status" &
    done

else
  echo "参数错误"

fi

• systemctl 管理zookeeper
  vim /etc/systemd/system/zookeeper.service

[Unit]
Description=Zookeeper
Requires=network.target
After=network.target

[Service]
User=root
Group=root
Type=forking
Environment=JAVA_HOME=/usr/local/src/jdk1.8.0_241/
WorkingDirectory=/usr/local/src/zookeeper
ExecStart=/usr/local/src/zookeeper/bin/zkServer.sh start /usr/local/src/zookeeper/conf/zoo.cfg
ExecStop=/usr/local/src/zookeeper/bin/zkServer.sh stop /usr/local/src/zookeeper/conf/zoo.cfg
ExecReload=/usr/local/src/zookeeper/bin/zkServer.sh restart /usr/local/src/zookeeper/conf/zoo.cfg

[Install]
WantedBy=multi-user.target

systemctl daemon-reload