Ubuntu16.04安装配置使用Zookeeper集群

集群系统环境

Zookeeper集群可以使用单节点或者多节点进行配置,但节点的数量必须要求为奇数个,这是因为需要符合zookeeper的选举机制。这里使用仍使用三个节点进行配置。前提:各个主机中/etc/hostname/etc/hosts文件中已经配置好对应主机名称和IP信息。

  • heron01: IP 
  • heron02: IP
  • heron03: IP

说明:该文章为Heron高可用集群配置的一部分内容,因此集群环境沿用集群配置中的内容。

下载安装Zookeeper

1. 这里使用了手动下载:下载地址 。这里选择了3.10.4的版本(2017-10-04)

2. 解压安装文件至指定目录,我的这里为:/home/yitian/zookeeper

yitian@ubuntu:~/zookeeper$ tar -zxvf zookeeper-3.4.10.tar.gz

配置Zookeeper集群 

准备工作:关闭Ubuntu的防火墙:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/conf$ sudo ufw disable
Firewall stopped and disabled on system startup

1. 修改配置文件:在/zookeeper-3.10.4/conf/目录下,复制zoo_sample.cfg文件为zoo.cfg,修改后如下:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/conf$ ll
total 24
...
-rw-rw-r--  1 yitian yitian  535 Mar 23  2017 configuration.xsl
-rw-rw-r--  1 yitian yitian 2161 Mar 23  2017 log4j.properties
-rw-rw-r--  1 yitian yitian 1025 Feb 16 01:40 zoo.cfg
-rw-rw-r--  1 yitian yitian  922 Mar 23  2017 zoo_sample.cfg

2. 编辑zoo.cfg文件,(1)修改和添加zookeeper中的dataDir和dataLogDir目录(zoo.cfg文件中可以找到默认的配置项):

# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
# zookeeper的DATA文件路径,在zookeeper集群环境下,需要在该路径下配置不同的myid文件
dataDir=/home/yitian/zookeeper/data 
dataLogDir=/home/yitian/zookeeper/logs

(2)其他配置项保持默认,在配置文件末尾添加需要配置的集群信息;

server.1=heron01:2888:3888 # 这里使用三台主机
server.2=heron02:2888:3888
server.3=heron03:2888:3888

3. 在集群中各个主机的DataDir目录下,配置各自的myid文件。在之前设置的dataDir(/home/yitian/zookeeper/data # zookeeper)中新建myid文件, 写入一个数字, 该数字表示这是第几号server. 该数字必须和zoo.cfg文件中的server.X中的X一一对应.并且同一集群内每个节点的数字是不相同的。

heron01,heron02,heron03中myid文件内容分别如下:

yitian@heron01:~/zookeeper/data$ cat myid
1
yitian@heron02:~/zookeeper/data$ cat myid
2
yitian@heron03:~/zookeeper/data$ cat myid
3

至此,zookeeper集群配置完毕。

启动并查看状态

对集群中的每个节点配置完成如上步骤后,依次启动各个节点的zookeeper(如下命令):

启动heron01:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
 yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... already running as process 3467.
yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader

启动heron02:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/conf$ ../bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/conf$ ../bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/conf$ ../bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

启动heron03:

yitian@ubuntu:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
yitian@ubuntu:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
yitian@ubuntu:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

可以看到,此时heron01为leader,heron02和heron03为follower。这里zookeeper的选举机制和原理可参考zookeeper的文档这里不再赘述。至此,zookeeper集群安装配置完成。

常见问题和解决方法

1. 在启动heron02中zookeeper之后,3888端口已经监听,但仍启动失败,情况如下:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ netstat -tunlp
(Not all processes could be identified, non-owned process info
  will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      -             
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -             
tcp6       0      0 :::35789                :::*                    LISTEN      3467/java      
tcp6       0      0 192.168.201.129:3888    :::*                    LISTEN      3467/java     
tcp6       0      0 :::22                   :::*                    LISTEN      -             
tcp6       0      0 :::2181                 :::*                    LISTEN      3467/java     
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           -             
udp        0      0 0.0.0.0:44346           0.0.0.0:*                           -             
udp        0      0 0.0.0.0:59810           0.0.0.0:*                           -             
udp        0      0 127.0.1.1:53            0.0.0.0:*                           -             
udp        0      0 0.0.0.0:68              0.0.0.0:*                           -             
udp        0      0 0.0.0.0:631             0.0.0.0:*                           -             
udp6       0      0 :::5353                 :::*                                -             
udp6       0      0 :::58146                :::*

而且,启动zookeeper时出现如下情况:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

问题解决:之前没有关闭防火墙,现在关闭防火墙重启zookeeper命令,防火墙关闭命令见上面,重启后如下:

yitian@ubuntu:~/zookeeper/zookeeper-3.4.10/bin$ ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... already running as process 3467. # 启动成功

2. 启动heron01主机中zookeeper时,总是mode:standalone模式,防护墙已关闭,但出现情况如下:

yitian@ubuntu:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
yitian@ubuntu:~$ ./zookeeper/zookeeper-3.4.10/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/yitian/zookeeper/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: standalone

查看jps,无QuorumPeerMain进程。且查看日志记录(/home/yitian/zookeeper/zookeeper-3.4.10/bin/zookeeper.out)有如下错误:

java.net.BindException: Address already in use
     at sun.nio.ch.Net.bind0(Native Method)
     at sun.nio.ch.Net.bind(Net.java:433)
     at sun.nio.ch.Net.bind(Net.java:425)
     at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
     at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
     at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
     at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
     at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:130)
     at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
     at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)

问题原因2181端口被其他进程占用(停止zkServer.sh没有用)。(1)首先查看端口占用情况:(切换至root用户,可以查看进程PID):

root@ubuntu:/home/yitian/zookeeper/zookeeper-3.4.10/conf# netstat -nltp | grep 2181
tcp6       0      0 :::2181                 :::*                    LISTEN      1111/java  

(2)杀死该zookeeper进程,然后重新启动,即可解决该问题:

root@ubuntu:/home/yitian/zookeeper/zookeeper-3.4.10/conf# kill -9 1111
root@ubuntu:/home/yitian/zookeeper/zookeeper-3.4.10/conf# netstat -nltp | grep 2181

参考资料

  • Ubuntu16.04安装配置ZooKeeper集群

你可能感兴趣的:(Zookeeper,zookeeper,cluster,Ubuntu,Heron)