zookeeper简介、zookeeper部署以及原理介绍

zookeeper简介、部署、原理介绍

zookeeper简介概述

Zookeeper是一个开源的分布式的,为分布式应用提供协调服务的Apache项目

Zookeeper从设计模式角度来理解:是一个基于观察者模式设计的分布式服务管理框架,它负责存储和管理大家都关心的数据,然后接受观察者的注册,一旦这些数据的状态发生变化,Zookeeper就将负责通知已经在Zookeeper上注册的那些观察者做出相应的反应。

zookeeper特点

1)Zookeeper:一个领导者(Leader),多个跟随者(Follower)组成的集群。

2)集群中只要有半数以上节点存活,Zookeeper集群就能正常服务。

3)全局数据一致:每个Server保存一份相同的数据副本,Client无论连接到哪个Server,数据都是一致的。

4)更新请求顺序进行,来自同一个Client的更新请求按其发送顺序依次执行。

5)数据更新原子性,一次数据更新要么成功,要么失败。

6)实时性,在一定时间范围内,Client能读到最新数据。

zookeeper数据结构

ZooKeeper数据模型的结构与Unix文件系统很类似,整体上可以看作是一棵树,每个节点称做一个ZNode。每一个ZNode默认能够存储1MB的数据,每个ZNode都可以通过其路径唯一标识。

zookeeper简介、zookeeper部署以及原理介绍_第1张图片

zookeeper应用场景

提供的服务包括:统一命名服务、统一配置管理、统一集群管理、服务器节点动态上下线、软负载均衡等。

统一命名服务

在分布式环境下,经常需要对应用/服务进行统一命名,便于识别。
例如:IP不容易记住,而域名容易记住。
zookeeper简介、zookeeper部署以及原理介绍_第2张图片

统一配置管理

1)分布式环境下,配置文件同步非常常见。

​ (1)一般要求一个集群中,所有节点的配置信息是一致的,比如 Kafka 集群。

​ (2)对配置文件修改后,希望能够快速同步到各个节点上。

2)配置管理可交由ZooKeeper实现。

​ (1)可将配置信息写入ZooKeeper上的一个Znode。

​ (2)各个客户端服务器监听这个Znode。

​ (3)一旦Znode中的数据被修改,ZooKeeper将通知各个客户端服务器。

zookeeper简介、zookeeper部署以及原理介绍_第3张图片

统一集群管理

1)分布式环境中,实时掌握每个节点的状态是必要的。
(1)可根据节点实时状态做出一些调整。

2)ZooKeeper可以实现实时监控节点状态变化
(1)可将节点信息写入ZooKeeper上的一个ZNode。
(2)监听这个ZNode可获取它的实时状态变化。

zookeeper简介、zookeeper部署以及原理介绍_第4张图片

服务器动态上下线

客户端能实时洞察到服务器上下线的变化

zookeeper简介、zookeeper部署以及原理介绍_第5张图片

软负载均衡

在Zookeeper中记录每台服务器的访问数,让访问数最少的服务器去处理最新的客户端请求

zookeeper简介、zookeeper部署以及原理介绍_第6张图片

zookeeper安装部署

zookeeper下载

附件为百度网盘友情共享连接,version : apache-zookeeper-3.5.7-bin.tar.gz

链接:https://pan.baidu.com/s/1txp55b0XVXPBbI2SK-NU5Q
提取码:jyan

官方网站: https://zookeeper.apache.org/

分布式安装部署(单点部署相当于一个节点部署方式雷同)

1)安装前准备

​ (1)安装Jdk (提前已有)

​ (2)拷贝Zookeeper安装包到Linux系统下

​ (3)解压到指定目录,并把目录名改成zookeeper-3.5.7

在ops01 , ops02 , ops03 三个节点上部署Zookeeper

11.8.37.50 ops01
11.8.36.63 ops02
11.8.36.76 ops03

部署路径/opt/module/zookeeper-3.5.7

wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >cat /etc/hosts
127.0.0.1 ydt-cisp-ops01
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
# zookeeper
11.8.37.50 ops01
11.8.36.63 ops02
11.8.36.76 ops03
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >ssh ops01 java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >ssh ops02 java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >ssh ops03 java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >
wangting@ops01:/home/wangting >tar -zxvf zookeeper-3.5.7.tar.gz -C /opt/module/

2) 配置服务器编号

(1)在/opt/module/zookeeper-3.5.7/这个目录下创建zkData
[wangting@ops01 /opt/module/zookeeper-3.5.7]$ mkdir -p zkData
(2)在/opt/module/zookeeper-3.5.7/zkData目录下创建一个myid的文件
[wangting@ops01 /opt/module/zookeeper-3.5.7]$ cd zkData
[wangting@ops01 /opt/module/zookeeper-3.5.7/zkData]$ touch myid
(3)编辑myid文件
[wangting@ops01 /opt/module/zookeeper-3.5.7/zkData]$ echo 1 > myid

【注意】: 3台机器都需要操作!并且3台机器的myid文件中的内容不一样

​ ops02 对应 echo 2 > myid

​ ops03 对应 echo 3 > myid

​ 最终要保证3台服务器的myid不一样,分别为1 , 2 , 3

​ myid的用途会对应在配置文件中

3)配置zoo.cfg文件

(1)重命名/opt/module/zookeeper-3.5.7/conf这个目录下的zoo_sample.cfg为zoo.cfg
[wangting@ops01 /opt/module/zookeeper-3.5.7/conf]$ mv zoo_sample.cfg zoo.cfg
(2)打开zoo.cfg文件
[atguigu@hadoop102 conf]$ vim zoo.cfg

(修改数据存储路径配置)

dataDir=/opt/module/zookeeper-3.5.7/zkData

(增加如下配置)

#######################cluster##########################
server.1=ops01:2888:3888
server.2=ops02:2888:3888
server.3=ops03:2888:3888

wangting@ops01:/home/wangting >cd /opt/module/zookeeper-3.5.7/conf/
wangting@ops01:/opt/module/zookeeper-3.5.7/conf >cat zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/opt/module/zookeeper-3.5.7/zkData
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
#######################cluster##########################
server.1=ops01:2888:3888
server.2=ops02:2888:3888
server.3=ops03:2888:3888

【注意】:

1 . server.1 / server.2 / server.3 中的 1 2 3 对应的就是zkData目录中myid中的值
2. server后面的ops01 / ops02 / ops03 没采用id是因为配置了/etc/hosts的解析
3. dataDir=/opt/module/zookeeper-3.5.7/zkData 路径可以自定义,建议3台服务器尽量配置一样的路径,便于管理

配置cluster信息解读:

server.A=B:C:D [ server.1=ops01:2888:3888 ( A=1 , B=ops01 , C=2888, D=3888)
A 是一个数字,表示这个是第几号服务器;
集群模式下配置一个文件myid,这个文件在配置的data目录下,这个文件里面数值就是A的值,Zookeeper启动时读取此文件,拿到里面的数据与zoo.cfg里面的配置信息比较从而判断到底是哪个server。
B 是这个服务器的地址;
C 是这个服务器Follower与集群中的Leader服务器交换信息的端口;
D 是万一集群中的Leader服务器挂了,需要一个端口来重新进行选举,选出一个新的Leader,而这个端口就是用来执行选举时服务器相互通信的端口。

4)配置环境变量

  1. 在/etc/profile文件中追加如下内容
#zookeeper
export ZOOKEEPER_HOME=/opt/module/zookeeper-3.5.7
export PATH=$PATH:$ZOOKEEPER_HOME/bin
  1. 引用

    source /etc/profile

【注意】 3台服务器都需要配置

5)启动集群并查看状态

wangting@ops01:/home/wangting > zkServer.sh start
####################################################
wangting@ops02:/home/wangting > zkServer.sh start
####################################################
wangting@ops03:/home/wangting > zkServer.sh start
####################################################
wangting@ops01:/home/wangting >zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower
####################################################
wangting@ops02:/home/wangting >zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: leader
####################################################
wangting@ops03:/home/wangting >zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.5.7/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost.
Mode: follower

客户端命令行操作

1)启动客户端

wangting@ops01:/home/wangting >zkCli.sh 
Connecting to localhost:2181
2021-03-25 15:42:20,076 [myid:] - INFO  [main:Environment@109] - Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT
2021-03-25 15:42:20,080 [myid:] - INFO  [main:Environment@109] - Client environment:host.name=ops01
2021-03-25 15:42:20,080 [myid:] - INFO  [main:Environment@109] - Client environment:java.version=1.8.0_131
2021-03-25 15:42:20,083 [myid:] - INFO  [main:Environment@109] - Client environment:java.vendor=Oracle Corporation
2021-03-25 15:42:20,083 [myid:] - INFO  [main:Environment@109] - Client environment:java.home=/usr/jdk1.8.0_131/jre
2021-03-25 15:42:20,083 [myid:] - INFO  [main:Environment@109] - Client environment:java.class.path=/opt/module/zookeeper-3.5.7/bin/../zookeeper-server/target/classes:/opt/module/zookeeper-3.5.7/bin/../build/classes:/opt/module/zookeeper-3.5.7/bin/../zookeeper-server/target/lib/*.jar:/opt/module/zookeeper-3.5.7/bin/../build/lib/*.jar:/opt/module/zookeeper-3.5.7/bin/../lib/zookeeper-jute-3.5.7.jar:/opt/module/zookeeper-3.5.7/bin/../lib/zookeeper-3.5.7.jar:/opt/module/zookeeper-3.5.7/bin/../lib/slf4j-log4j12-1.7.25.jar:/opt/module/zookeeper-3.5.7/bin/../lib/slf4j-api-1.7.25.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-transport-native-unix-common-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-transport-native-epoll-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-transport-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-resolver-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-handler-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-common-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-codec-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/netty-buffer-4.1.45.Final.jar:/opt/module/zookeeper-3.5.7/bin/../lib/log4j-1.2.17.jar:/opt/module/zookeeper-3.5.7/bin/../lib/json-simple-1.1.1.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jline-2.11.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-util-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-servlet-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-server-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-security-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-io-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jetty-http-9.4.24.v20191120.jar:/opt/module/zookeeper-3.5.7/bin/../lib/javax.servlet-api-3.1.0.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jackson-databind-2.9.10.2.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jackson-core-2.9.10.jar:/opt/module/zookeeper-3.5.7/bin/../lib/jackson-annotations-2.9.10.jar:/opt/module/zookeeper-3.5.7/bin/../lib/commons-cli-1.2.jar:/opt/module/zookeeper-3.5.7/bin/../lib/audience-annotations-0.5.0.jar:/opt/module/zookeeper-3.5.7/bin/../zookeeper-*.jar:/opt/module/zookeeper-3.5.7/bin/../zookeeper-server/src/main/resources/lib/*.jar:/opt/module/zookeeper-3.5.7/bin/../conf::/usr/jdk1.8.0_131/lib/dt.jar:/usr/jdk1.8.0_131/lib/tools.jar
2021-03-25 15:42:20,083 [myid:] - INFO  [main:Environment@109] - Client environment:java.library.path=:/opt/dmdbms/bin:/opt/dmdbms/bin:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:java.io.tmpdir=/tmp
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:java.compiler=<NA>
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:os.name=Linux
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:os.arch=amd64
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:os.version=3.10.0-693.el7.x86_64
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:user.name=wangting
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:user.home=/home/wangting
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:user.dir=/home/wangting
2021-03-25 15:42:20,084 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.free=235MB
2021-03-25 15:42:20,086 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.max=241MB
2021-03-25 15:42:20,087 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.total=241MB
2021-03-25 15:42:20,090 [myid:] - INFO  [main:ZooKeeper@868] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@726f3b58
2021-03-25 15:42:20,097 [myid:] - INFO  [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-03-25 15:42:20,105 [myid:] - INFO  [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes
2021-03-25 15:42:20,114 [myid:] - INFO  [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled=
Welcome to ZooKeeper!
2021-03-25 15:42:20,123 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2021-03-25 15:42:20,197 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /0:0:0:0:0:0:0:1:56234, server: localhost/0:0:0:0:0:0:0:1:2181
2021-03-25 15:42:20,209 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x10099168a1b0001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

2)显示所有操作命令

[zk: localhost:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
	addauth scheme auth
	close 
	config [-c] [-w] [-s]
	connect host:port
	create [-s] [-e] [-c] [-t ttl] path [data] [acl]
	delete [-v version] path
	deleteall path
	delquota [-n|-b] path
	get [-s] [-w] path
	getAcl [-s] path
	history 
	listquota path
	ls [-s] [-w] [-R] path
	ls2 path [watch]
	printwatches on|off
	quit 
	reconfig [-s] [-v version] [[-file path] | [-members serverID=host:port1:port2;port3[,...]*]] | [-add serverId=host:port1:port2;port3[,...]]* [-remove serverId[,...]*]
	redo cmdno
	removewatches path [-c|-d|-a] [-l]
	rmr path
	set [-s] [-v version] path data
	setAcl [-s] [-v version] [-R] path acl
	setquota -n|-b val path
	stat [-w] path
	sync path
Command not found: Command not found help

3)查看当前znode中所包含的内容

[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 2] ls -w /
[zookeeper]
[zk: localhost:2181(CONNECTED) 3] ls -s /
[zookeeper]cZxid = 0x0
ctime = Thu Jan 01 08:00:00 CST 1970
mZxid = 0x0
mtime = Thu Jan 01 08:00:00 CST 1970
pZxid = 0x0
cversion = -1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 1

4)查看当前节点详细数据

[zk: localhost:2181(CONNECTED) 4] ls2 /
'ls2' has been deprecated. Please use 'ls [-s] path' instead.
[zookeeper]
cZxid = 0x0
ctime = Thu Jan 01 08:00:00 CST 1970
mZxid = 0x0
mtime = Thu Jan 01 08:00:00 CST 1970
pZxid = 0x0
cversion = -1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 1

5)分别创建2个普通节点

[zk: localhost:2181(CONNECTED) 5] create /wangting 666

WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/
Created /wangting
[zk: localhost:2181(CONNECTED) 6] create /wangting/20210325 "this_is_value"
Created /wangting/20210325

【注意】: 最后节点后面跟上的内容为节点值(znode对应的value)

6)获得节点的值

[zk: localhost:2181(CONNECTED) 7] get /wangting
666
[zk: localhost:2181(CONNECTED) 8] get /wangting/20210325
this_is_value

7)创建短暂节点

[zk: localhost:2181(CONNECTED) 10] 
[zk: localhost:2181(CONNECTED) 10] create -e /wangting/temp "e_quit_will_not_found"
Created /wangting/temp
[zk: localhost:2181(CONNECTED) 11] ls /wangting
[20210325, temp]
[zk: localhost:2181(CONNECTED) 12] quit

WATCHER::

WatchedEvent state:Closed type:None path:null
2021-03-25 15:48:20,228 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x10099168a1b0001 closed
2021-03-25 15:48:20,228 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x10099168a1b0001
wangting@ops01:/home/wangting >zkCli.sh 
Connecting to localhost:2181
Welcome to ZooKeeper!
2021-03-25 15:48:27,957 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x10099168a1b0002, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /wangting
[20210325]						# 这里的temp没有了

【注意】:create带上-e参数时,只是当前的会话生效,这里的会话指的是当次的sessionid;例如,操作1中启动客户端,可以在启动的输出控制台中找到【sessionid = 0x10099168a1b0001】,也就是说创建短暂节点在sessonid为这个值时有效,当我们quit并且再次创建客户端时,连接sessionid变了,【sessionid = 0x10099168a1b0002】,也就意味着之前的create -e 创建的节点均已失效

8)创建带序号的节点

[zk: localhost:2181(CONNECTED) 1] 
[zk: localhost:2181(CONNECTED) 1] create /wangting/huahua "y"
Created /wangting/huahua
[zk: localhost:2181(CONNECTED) 2] create -s /wangting/sanhua "z"
Created /wangting/sanhua0000000003
[zk: localhost:2181(CONNECTED) 3] create -s /wangting/sanhuali "a"
Created /wangting/sanhuali0000000004
[zk: localhost:2181(CONNECTED) 4] create -s /wangting/sanhualili "b"
Created /wangting/sanhualili0000000005
[zk: localhost:2181(CONNECTED) 5] create -s /wangting/sanhualilili "c"
Created /wangting/sanhualilili0000000006

9)修改节点数据值

[zk: localhost:2181(CONNECTED) 6] 
[zk: localhost:2181(CONNECTED) 6] get /wangting/huahua
y
[zk: localhost:2181(CONNECTED) 7] set /wangting/huahua "not_y"
[zk: localhost:2181(CONNECTED) 8] get /wangting/huahua
not_y

10)其它

参数后面加watch 监听

delete 删除节点 (有子节点删不了)

rmr 递归删除节点 (连带子节点一起删除)

stat 查看节点状态

Zookeeper内部原理

节点类型

持久(Persistent):客户端和服务器端断开连接后,创建的节点不删除

创建znode时设置顺序标识,znode名称后会附加一个值,顺序号是一个单调递增的计数器,由父节点维护

​ (1)持久化目录节点 客户端与Zookeeper断开连接后,该节点依旧存在

​ (2)持久化顺序编号目录节点 ,客户端与Zookeeper断开连接后,该节点依旧存在,只是Zookeeper给该节点名称进行顺序编号

【注意】:在分布式系统中,顺序号可以被用于为所有的事件进行全局排序,这样客户端可以通过顺序号推断事件的顺序

短暂(Ephemeral):客户端和服务器端断开连接后,创建的节点自己删除

​ (3)临时目录节点,客户端与Zookeeper断开连接后,该节点被删除

​ (4)临时顺序编号目录节点 ,客户端与Zookeeper断开连接后,该节点被删除,只是Zookeeper给该节点名称进行顺序编号。

zookeeper简介、zookeeper部署以及原理介绍_第7张图片

Stat结构体

[zk: localhost:2181(CONNECTED) 9] stat /wangting/huahua
cZxid = 0x100000009
ctime = Thu Mar 25 15:50:28 CST 2021
mZxid = 0x10000000e
mtime = Thu Mar 25 15:52:53 CST 2021
pZxid = 0x100000009
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

(1)czxid-创建节点的事务zxid

每次修改ZooKeeper状态都会收到一个zxid形式的时间戳,也就是ZooKeeper事务ID。

事务ID是ZooKeeper中所有修改总的次序。每个修改都有唯一的zxid,如果zxid1小于zxid2,那么zxid1在zxid2之前发生。

(2)ctime - znode被创建的毫秒数(从1970年开始)

(3)mzxid - znode最后更新的事务zxid

(4)mtime - znode最后修改的毫秒数(从1970年开始)

(5)pZxid-znode最后更新的子节点zxid

(6)cversion - znode子节点变化号,znode子节点修改次数

(7)dataversion - znode数据变化号

(8)aclVersion - znode访问控制列表的变化号

(9)ephemeralOwner- 如果是临时节点,这个是znode拥有者的session id。如果不是临时节点则是0。

(10)dataLength- znode的数据长度

(11)numChildren - znode子节点数量

监听器原理

1、监听原理详解:

​ 1)首先要有一个main()线程

​ 2)在main线程中创建Zookeeper客户端,这时就会创建两个线程,一个负责网络连接通信(connet),一个负责监听listener

​ 3)通过connect线程将注册的监听事件发送给Zookeeper。

​ 4)在Zookeeper的注册监听器列表中将注册的监听事件添加到列表中。

​ 5)Zookeeper监听到有数据或路径变化,就会将这个消息发送给listener线程。

​ 6)listener线程内部调用了process()方法。

2、常见的监听

​ 1)监听节点数据的变化 get path [watch]

​ 2)监听子节点增减的变化 ls path [watch]

zookeeper简介、zookeeper部署以及原理介绍_第8张图片

选举机制

(1)半数机制:集群中半数以上机器存活,集群可用。所以Zookeeper适合安装奇数台服务器,偶数集群不可取。

(2)Zookeeper虽然在配置文件中并没有指定Master和Slave。但是,Zookeeper工作时,是有一个节点为Leader,其他则为Follower,Leader是通过内部的选举机制临时产生的。

(3)以一个简单的例子来说明整个选举的过程。

【举例】:

​ 假设有五台服务器组成的Zookeeper集群,它们的id从1 - 5,同时它们都是最新启动的,也就是没有历史数据,在存放数据量这一点上,都是一样的。假设这些服务器依序启动,来看看会发生什么。
zookeeper简介、zookeeper部署以及原理介绍_第9张图片

(1)server1启动,发起一次选举。server1投自己一票。此时server1票数一票,不够半数以上(5票中需满足3票),选举无法完成,server1状态保持为LOOKING;

(2)server2启动,再发起一次选举。server1和2分别投自己一票并交换选票信息:此时server1发现server2的ID (myid) 比自己目前投票推举的(server1)大,更改选票为推举server2。此时server1票数0票,server2票数2票,没有半数以上结果,选举无法完成,server1,2状态保持LOOKING

(3)server3启动,发起一次选举。此时server1和2都会更改选票为server3。此次投票结果:server1为0票,server2为0票,server3为3票。此时server3的票数已经超过半数,server3当选Leader。server1,2更改状态为FOLLOWING,server3更改状态为LEADING;

(4)server4启动,发起一次选举。此时server1,2,3已经不是LOOKING状态,不会更改选票信息。交换选票信息结果:server3为3票,server4为1票。此时server4服从多数,更改选票信息为server3,并更改server4状态为FOLLOWING;

(5)server5启动,发起一次选举。此时server1,2,3,4已经不是LOOKING状态,不会更改选票信息.交换选票信息结果:server3为4票,server5为1票。此时server5服从多数,更改选票信息为server3,并更改server5状态为FOLLOWING;

数据写入流程

zookeeper简介、zookeeper部署以及原理介绍_第10张图片

1.Client向Zookeeper的Server1上写数据,发送一个写请求

2.如果Server1不是Leader,那么Server1会把接收到的请求进一步转发给Leader,因为每个Zookeeper的Server里面有一个是Leader

3.Leader会将写请求广播给各个Server,各个Server写成功后会通知Leader

4.当Leader收到半数以上Server数据写成功了,就说明数据写成功了.如果这里有五个节点的话,只要有三个节点数据写成功了.那么就认为数据写成功了

5.写成功以后,Leader就会告诉Server1数据写成功了

6.Server1会进一步通知Client数据写成功了,这时就认为整个写操作成功

你可能感兴趣的:(运维,大数据,zookeeper,大数据,运维,zookeeper部署,zookeeper简介)