01.ZooKeeper入门

一、准备工作

1. 安装虚拟机软件Virtual Box
2. 在Virtual Box准备三台（或以上）虚拟机node0， node1，node2 ，操作系统为CentOS 6.9，IP设置为192.168.3.180，192.168.3.181，192.168.3.182(参考Nginx环境搭建　的准备工作部分)
3. 每台虚拟机安装jdk1.8(参考CentOS 安装JDK)
4. 下载 ZooKeeper

二、安装与配置

1、将zookeeper文件上传到3个节点（在真实机器操作，我使用系统的是Ubuntu）

user@linux:~$ scp zookeeper-3.4.14.tar.gz  [email protected]:/root/test
user@linux:~$ scp zookeeper-3.4.14.tar.gz  [email protected]:/root/test
user@linux:~$ scp zookeeper-3.4.14.tar.gz  [email protected]:/root/test

2、在node0中操作，解压缩文件并放到指定的目录

[root@localhost java]# cd ~/test
[root@localhost test]# tar -zxvf zookeeper-3.4.14.tar.gz 
[root@localhost test]# mv zookeeper-3.4.14 /usr/local/java/

3、配置zookeeper

[root@localhost test]# cd /usr/local/java/zookeeper-3.4.14/conf/
[root@localhost conf]# cp zoo_sample.cfg zoo.cfg 
[root@localhost conf]# ls
configuration.xsl  log4j.properties  zoo.cfg  zoo_sample.cfg
[root@localhost conf]# vim zoo.cfg 

zoo.cfg配置如下

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper
clientPort=2181
#maxClientCnxns=60
...
...

• tickTime： 指定了ZooKeeper的基本时间单位（以毫秒为单位）；
• initLimit： 指定了启动zookeeper时，zookeeper实例中的随从实例同步到领导实例的初始化连接时间限制，超出时间限制则连接失败（以tickTime为时间单位）；
• syncLimit： 指定了zookeeper正常运行时，主从节点之间同步数据的时间限制，若超过这个时间限制，那么随从实例将会被丢弃；
• dataDir： zookeeper存放数据的目录；
• clientPort： 用于连接客户端的端口。

在文件的最后加上集群的服务器信息

server.1=192.168.3.180:2888:3888
server.2=192.168.3.181:2888:3888
server.3=192.168.3.182:2888:3888

4、创建zookeeper数据的存储目录，也就是上面配置的dataDir目录

[root@localhost bin]# mkdir /tmp/zookeeper

5、在zookeeper数据的存储目录中创建文件myid

[root@localhost bin]# cd /tmp/zookeeper
[root@localhost bin]# touch myid

6、在node1和node2中重复执行步骤2到5

7、编辑node*中的myid文件

node0 中执行：

[root@localhost bin]# echo -n "1">/tmp/zookeeper/myid 

node1中执行

[root@localhost bin]# echo -n "2">/tmp/zookeeper/myid 

node2中执行

[root@localhost bin]# echo -n "3">/tmp/zookeeper/myid 

上面的myid设置主要是设置zoo.cfg中server.xx中的xx，例如node0是集群服务器的server.1，那么node1中的myid设置为1。

8、3个节点都启动zookeeper

[root@localhost bin]# cd /usr/local/java/zookeeper-3.4.14/bin/
[root@localhost bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

9、查看zookeeper运行状态

[root@localhost bin]# jps
15601 QuorumPeerMain
15666 Jps

存在QuorumPeerMain，zookeeper已成功启动

[root@localhost bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: follower

看到zookeeper的状态为Mode: follower或者Mode: leader，集群服务已经成功。

10、关闭zookeeper服务

[root@localhost bin]# ./zkServer.sh stop

11、重启zookeeper服务

[root@localhost bin]# ./zkServer.sh restart

三、Ｗeb界面

四、异常解决

问题1. zookeeper启动后，jps命令看不到进程QuorumPeerMain

查看日志：

[root@localhost bin]# cat /usr/local/java/zookeeper-3.4.14/bin/zookeeper.out 
2019-05-19 19:46:55,928 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
2019-05-19 19:46:55,934 [myid:] - ERROR [main:QuorumPeerConfig@217] - 192.168.3.180 does not have the form host:port or host:port:port  or host:port:port:type
2019-05-19 19:46:55,935 [myid:] - ERROR [main:QuorumPeerMain@92] - Unexpected exception, exiting abnormally
java.lang.ArrayIndexOutOfBoundsException: 1
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:223)
    at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)

说明是server.xx配置出了问题，配置格式应为host:port或host:port:port或host:port:port:type，配置为如下即可。

server.1=192.168.3.180:2888:3888
server.2=192.168.3.181:2888:3888
server.3=192.168.3.182:2888:3888

问题2. 所有节点成功启动后，查看ZooKeeper的运行状态，报以下错误信息

[root@localhost bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

解决方法:
先查看日志

[root@localhost bin]#  cat /usr/local/java/zookeeper-3.4.14/bin/zookeeper.out
java.net.NoRouteToHostException: 没有到主机的路由 (Host unreachable)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:558)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
    at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:958)
2019-05-19 17:41:17,553 [myid:2] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:QuorumPeer$QuorumServer@185] - Resolved hostname: 192.168.3.182 to address: /192.168.3.182
2019-05-19 17:41:17,553 [myid:2] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@847] - Notification time out: 60000

日志中有一个异常信息没有到主机的路由，也就是服务器之间通讯有问题。

• 尝试ping一下，相互能ping通，那服务器节点的IP设置没有问题，如果不能通，先把IP设置好。
• 查看防火墙是否开启

[root@localhost bin]# service iptables status
表格：filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
2    ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22 
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:80 
6    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination         
1    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination    

• 尝试关闭防火墙，然后重启zookeeper

[root@localhost bin]# service iptables stop
iptables：将链设置为政策 ACCEPT：filter                    [确定]
iptables：清除防火墙规则：                                 [确定]
iptables：正在卸载模块：                                   [确定]
[root@localhost bin]# ./zkServer.sh restart
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/java/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: follower

看到zookeeper的状态为Mode: follower或者Mode: leader，那么问题就已经解决，多说一句，这里解决问题是把防火墙关了，生产环境不能这么干，应该是配置防火墙信息，一般由运维来负责。