zookeeper集群安装小结

Zookeeper是一种在分布式系统中被广泛用来作为：分布式状态管理、分布式协调管理、分布式配置管理、和分布式锁服务的集群。kafka增加和减少服务器都会在Zookeeper节点上触发相应的事件kafka系统会捕获这些事件，进行新一轮的负载均衡，客户端也会捕获这些事件来进行新一轮的处理

安装过程可以参考
http://www.cnblogs.com/wrong5566/p/6056788.html
https://www.cnblogs.com/linuxprobe/p/5851699.html

要点：

在三台服务器上配置zoo.cfg文件（来自zoo_sample.cfg文件的复制为zoo.cfg）

[root@zookeeper conf]# pwd
/opt/zookeeper-3.4.12/conf
[root@zookeeper conf]# vi zoo.cfg 
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/opt/zookeeper-3.4.12/data
dataLogDir=/opt/zookeeper-3.4.12/logs
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1= 192.168.197.129:2888:3888
server.2= 192.168.197.134:2888:3888
server.3= 192.168.197.136:2888:3888
~
~
~
~
~
~
~
~
~
~
"zoo.cfg" 34L, 1139C

需要定义该目录：
dataDir=/opt/zookeeper-3.4.12/data
dataLogDir=/opt/zookeeper-3.4.12/logs
其次：
新建myid文件：

在三个linux机器上刚刚创建的data目录下新建一个myid文件并打开，vi myid命令。

在192.168.197.129机器上输入1，保存退出；在192.168.197.134机器上输入2，保存退出；在192.168.197.136机器上输入3，保存退出。这里的1,2,3是与server1,2,3相对应的。
启动：
需要先依次启动3台服务器上的zookeeper服务，
最后再查看状态有：leader和follower，证明启动成功
常用命令
[root@zk1 bin]# zkServer.sh start来启动。
　　　　zkServer.sh restart　　(重启)
　　　　zkServer.sh status　　(查看状态)
　　　　zkServer.sh stop　　(关闭)
　　　　zkServer.sh start-foreground　　(以打印日志方式启动)`
129上：

[root@xiehao1 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@xiehao1 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: follower

134上：

[root@zookeeper bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@zookeeper bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: leader

136上：

[root@zookeeper bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@zookeeper bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: follower

**

zookeeper集群连接：

**

• Zookeeper集群搭建完毕之后，可以通过客户端脚本连接到zookeeper集群上面，对客户端来说，zookeeper集群是一个整体，连接到zookeeper集群实际上感觉在独享整个集群的服务。
• 在129上进行测试，连接到134上的zookeeper服务

[root@xiehao1 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Mode: follower
[root@xiehao1 bin]# ./zkCli.sh -server 192.168.197.134:2181
Connecting to 192.168.197.134:2181
2019-04-28 23:47:50,484 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.12-e5259e437540f349646870ea94dc2658c4e44b3b, built on 03/27/2018 03:55 GMT
2019-04-28 23:47:50,497 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=<NA>
2019-04-28 23:47:50,497 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_161
2019-04-28 23:47:50,504 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2019-04-28 23:47:50,504 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/opt/jdk1.8.0_161/jre
2019-04-28 23:47:50,505 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/opt/zookeeper-3.4.12/bin/../build/classes:/opt/zookeeper-3.4.12/bin/../build/lib/*.jar:/opt/zookeeper-3.4.12/bin/../lib/slf4j-log4j12-1.7.25.jar:/opt/zookeeper-3.4.12/bin/../lib/slf4j-api-1.7.25.jar:/opt/zookeeper-3.4.12/bin/../lib/netty-3.10.6.Final.jar:/opt/zookeeper-3.4.12/bin/../lib/log4j-1.2.17.jar:/opt/zookeeper-3.4.12/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.12/bin/../lib/audience-annotations-0.5.0.jar:/opt/zookeeper-3.4.12/bin/../zookeeper-3.4.12.jar:/opt/zookeeper-3.4.12/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.12/bin/../conf:.:/opt/jdk1.8.0_161/lib.tools.jar
2019-04-28 23:47:50,505 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-04-28 23:47:50,505 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-04-28 23:47:50,505 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-28 23:47:50,506 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2019-04-28 23:47:50,506 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2019-04-28 23:47:50,506 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-642.el6.x86_64
2019-04-28 23:47:50,506 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2019-04-28 23:47:50,506 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2019-04-28 23:47:50,507 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/opt/zookeeper-3.4.12/bin
2019-04-28 23:47:50,513 [myid:] - INFO  [main:ZooKeeper@441] - Initiating client connection, connectString=192.168.197.134:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@4b85612c
Welcome to ZooKeeper!
2019-04-28 23:47:50,610 [myid:] - INFO  [main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@1028] - Opening socket connection to server 192.168.197.134/192.168.197.134:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-28 23:47:50,866 [myid:] - INFO  [main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@878] - Socket connection established to 192.168.197.134/192.168.197.134:2181, initiating session
2019-04-28 23:47:51,027 [myid:] - INFO  [main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server 192.168.197.134/192.168.197.134:2181, sessionid = 0x2000035a8500000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: 192.168.197.134:2181(CONNECTED) 0] 
[zk: 192.168.197.134:2181(CONNECTED) 0] ls
[zk: 192.168.197.134:2181(CONNECTED) 1] ls /
[zookeeper]
[zk: 192.168.197.134:2181(CONNECTED) 2] quit
Quitting...
2019-04-28 23:48:41,983 [myid:] - INFO  [main:ZooKeeper@687] - Session: 0x2000035a8500000 closed
2019-04-28 23:48:41,990 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@521] - EventThread shut down for session: 0x2000035a8500000
[root@xiehao1 bin]# 

通过上图可以看出整个zookeeper集群已经搭建并测试完成。

#!/bin/bash 
 
#snapshot file dir 
dataDir=/opt/zookeeper/zkdata/version-2
#tran log dir 
dataLogDir=/opt/zookeeper/zkdatalog/version-2

#Leave 66 files 
count=66 
count=$[$count+1] 
ls -t $dataLogDir/log.* | tail -n +$count | xargs rm -f 
ls -t $dataDir/snapshot.* | tail -n +$count | xargs rm -f 

#以上这个脚本定义了删除对应两个目录中的文件，保留最新的66个文件，可以将他写到crontab中，设置为每天凌晨2点执行一次就可以了。


#zk log dir   del the zookeeper log
#logDir=
#ls -t $logDir/zookeeper.log.* | tail -n +$count | xargs rm -f

（图片博客链接https://www.cnblogs.com/luotianshuai/p/5206662.html）

错误记录：
当对三台机器同时 zkServer.sh start启动时，发现都启动成功了，但是以zkServer.sh status命令来查看启动状态时，发现总是在报错，如下图：

[root@xiehao1 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

查看报错日志（默认输出日志 bin/zookeeper.out）：

[root@xiehao1 zookeeper-3.4.12]# cd bin/
[root@xiehao1 bin]# ll
total 40
-rwxrwxr--. 1 root root  232 Mar 26  2018 README.txt
-rwxrwxr--. 1 root root 1937 Mar 26  2018 zkCleanup.sh
-rwxrwxr--. 1 root root 1056 Mar 26  2018 zkCli.cmd
-rwxrwxr--. 1 root root 1534 Mar 26  2018 zkCli.sh
-rwxrwxr--. 1 root root 1759 Mar 26  2018 zkEnv.cmd
-rwxrwxr--. 1 root root 2696 Mar 26  2018 zkEnv.sh
-rwxrwxr--. 1 root root 1089 Mar 26  2018 zkServer.cmd
-rwxrwxr--. 1 root root 6773 Mar 26  2018 zkServer.sh
-rw-r--r--. 1 root root 2090 Apr 28 20:09 zookeeper.out
[root@xiehao1 bin]# cat zookeeper.out 
2019-04-28 20:09:36,746 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg
2019-04-28 20:09:36,784 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.136 to address: /192.168.197.136
2019-04-28 20:09:36,784 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.134 to address: /192.168.197.134
2019-04-28 20:09:36,786 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.129 to address: /192.168.197.129
2019-04-28 20:09:36,786 [myid:] - INFO  [main:QuorumPeerConfig@398] - Defaulting to majority quorums
2019-04-28 20:09:36,794 [myid:1] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2019-04-28 20:09:36,794 [myid:1] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2019-04-28 20:09:36,794 [myid:1] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2019-04-28 20:09:36,819 [myid:1] - INFO  [main:QuorumPeerMain@130] - Starting quorum peer
2019-04-28 20:09:36,841 [myid:1] - INFO  [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2019-04-28 20:09:36,851 [myid:1] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
2019-04-28 20:09:36,893 [myid:1] - ERROR [main:QuorumPeerMain@92] - Unexpected exception, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$SnapDirContentCheckException: Snapshot directory has log files. Check if dataLogDir and dataDir configuration is correct.
        at org.apache.zookeeper.server.persistence.FileTxnSnapLog.checkSnapDir(FileTxnSnapLog.java:135)
        at org.apache.zookeeper.server.persistence.FileTxnSnapLog.<init>(FileTxnSnapLog.java:108)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:141)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)

网上说的常见的解决办法：
（1）：zoo.cfg配置文件中，server.x:2888:3888配置出现错误；
（2）：myid文件内容和server.x不对应，或者myid不在data目录下；
（3）：系统防火墙是否在启动。
经测试，没有以上问题，主要问题是：
（参考https://www.cnblogs.com/yinghao/p/6135095.html）
查看conf/zoo.cfg配置文件

dataDir 路径下删除 version-2文件夹

重新启动，解决问题！