zookeeper安装及基本命令使用

zookeeper是什么

ZooKeeper是一个集中服务,用于维护配置信息、命名、提供分布式同步和提供组服务。
所有这些类型的服务都以某种形式被分布式应用程序使用。每次实现它们时,都要进行大量的工作来修复不可避免的bug和竞争条件。
由于实现这类服务的困难,应用程序最初通常会忽略它们,这使得它们在发生变化时变得脆弱,并且难以管理。即使操作正确,在部署应用程序时,这些服务的不同实现也会导致管理复杂性。

zookeeper常见使用场景

  • 维护配置信息
  • 分布式锁服务
  • 集群管理
  • 生成分布式唯一ID

zookeeper的特性

  • 高性能
    存储方式类似Redis保存至内存中档。
  • 高可用
    一般以集群的方式对外提供服务。
  • 严格按照顺序访问
    对于来自客户端的每个更新请求,ZooKeeper都会分配一个全局的递增编号。这个编号反映了所有事物操作的先后顺序。

安装

zookeeper运行依赖以jdk。

安装jdk

[root@10 jdk]# ll
总用量 181296
-rw-r--r--. 1 root root 185646832 6月  15 00:07 jdk-8u181-linux-x64.tar.gz
[root@10 jdk]# mkdir -p /usr/local/java
[root@10 jdk]# tar -zxvf jdk-8u181-linux-x64.tar.gz -C /usr/local/java

配置jdk环境变量

[root@10 jdk]# vi /etc/profile
#在文件的最后追加
JAVA_HOME=/usr/local/java/jdk1.8.0_181
CLASSPATH=.:%JAVA_HOME%/lib/dt.jar:%JAVA_HOME%/lib/tools.jar
PATH=.:$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

验证jdk安装是否成功

#让我的文件生效,如果不执行该语句请重新打开终端
[root@10 jdk]# source /etc/profile
[root@10 jdk]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
[root@10 jdk]# javac -version
javac 1.8.0_181

安装zookeeper

到官方网站下载安装包:
https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

下载zookeeper

[root@10 zookeeper]# wget https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
#然后讲文件解压
[root@10 zookeeper]# tar -zxvf zookeeper-3.4.14.tar.gz
[root@10 zookeeper]# ll
drwxr-xr-x. 14 2002 2002     4096 3月   7 2019 zookeeper-3.4.14
-rw-r--r--.  1 root root 37676320 4月   1 2019 zookeeper-3.4.14.tar.gz

修改配置文件

#进入配置文件目录
[root@10 zookeeper]# cd zookeeper-3.4.14/conf/
[root@10 conf]# pwd
/home/zookeeper/zookeeper-3.4.14/conf
[root@10 conf]# ll
总用量 16
-rw-rw-r--. 1 2002 2002  535 3月   7 2019 configuration.xsl
-rw-rw-r--. 1 2002 2002 2161 3月   7 2019 log4j.properties
-rw-rw-r--. 1 2002 2002  922 3月   7 2019 zoo_sample.cfg
#拷贝一份配置文件模板
[root@10 conf]# cp zoo_sample.cfg zoo.cfg

zoo_sample.cfg 是zookeeper的一个配置文件模板,zookeeper启动默认加载的配置文件名称是zoo.cfg。这也是我们拷贝一个zoo.cfg文件的原因。

配置文件参数介绍

  • tickTime :ZooKeeper使用的基本时间单位(毫秒)。它是用来做心跳和最小会话超时将是滴答时间的两倍。

  • dataDir : 存储内存中数据库快照以及数据库更新的事务日志(除非另有指定)的位置。

  • clientPort :侦听客户端连接的端口

基本命令使用

更详细的文档,可以参考官方的文档:
https://zookeeper.apache.org/doc/r3.6.1/zookeeperStarted.html

启动zookeeper

#进入zookeeper安装目录的bin目录下
[root@10 bin]# pwd
/home/zookeeper/zookeeper-3.4.14/bin
#启动
[root@10 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
#对应的 还有 停止 重启等等
[root@10 bin]# ./zkServer.sh --help
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.14/bin/../conf/zoo.cfg
Usage: ./zkServer.sh {
     start|start-foreground|stop|restart|status|upgrade|print-cmd}
#查看zookeeper状态
[root@10 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: standalone

使用客户端连接zookeeper

[root@10 bin]# ll
总用量 52
-rwxr-xr-x. 1 2002 2002  232 3月   7 2019 README.txt
-rwxr-xr-x. 1 2002 2002 1937 3月   7 2019 zkCleanup.sh
-rwxr-xr-x. 1 2002 2002 1056 3月   7 2019 zkCli.cmd
-rwxr-xr-x. 1 2002 2002 1534 3月   7 2019 zkCli.sh
-rwxr-xr-x. 1 2002 2002 1759 3月   7 2019 zkEnv.cmd
-rwxr-xr-x. 1 2002 2002 2919 3月   7 2019 zkEnv.sh
-rwxr-xr-x. 1 2002 2002 1089 3月   7 2019 zkServer.cmd
-rwxr-xr-x. 1 2002 2002 6773 3月   7 2019 zkServer.sh
-rwxr-xr-x. 1 2002 2002  996 3月   7 2019 zkTxnLogToolkit.cmd
-rwxr-xr-x. 1 2002 2002 1385 3月   7 2019 zkTxnLogToolkit.sh
-rw-r--r--. 1 root root 5806 8月  14 12:07 zookeeper.out
# 这里默认去连接的是localhost:2181 如果想连接其他集群可以指定主机跟端口号 ./zkCli.sh -server 192.168.1.188:2181
[root@10 bin]# ./zkCli.sh 
Connecting to localhost:2181
2020-08-14 12:08:46,251 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
...
WATCHER::

WatchedEvent state:SyncConnected type:None path:null
#这里表示连接成功
[zk: localhost:2181(CONNECTED) 0] 

zookeeper的数据存储结构跟linux系统类似

例如:/ 表示跟目录

所有zookeeper的数据的操作也是通过层级目录关系操作的

查看某节点或路径下是否还有节点-- ls /ls2

 返回当前节点下子节点
 ls path(路径) 必须从跟目录"/"开始
 返回当前节点下子节点跟当前节点是属性
 ls2 path (路径) 必须从跟目录"/"开始
#查看跟目录下数据情况
#ls
[zk: localhost:2181(CONNECTED) 4] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 18] ls /zookeeper/config
[]
#ls2
[zk: localhost:2181(CONNECTED) 6] ls2 / 
[temp, zookeeper, order0000000005, hadoop, netty]
cZxid = 0x0
ctime = Thu Jan 01 08:00:00 CST 1970
mZxid = 0x0
mtime = Thu Jan 01 08:00:00 CST 1970
pZxid = 0x39
cversion = 21
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 5

查询节点中的数据—get

get path(路径) 必须从跟目录"/"开始
[zk: localhost:2181(CONNECTED) 27] get /netty
123456
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0xd
mtime = Fri Aug 14 17:16:17 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0

节点属性解释:

状态属性 说明
cZxid 数据节点创建的事务ID (16进制)
ctime 数据节点的创建时间
mZxid 数据节点最后一次修改的事务ID (16进制)
mtime 数据节点最后一次修改时间
pZxid 数据节点的子节点最后一次被修改时事务ID (16进制)
cversion 子节点的更新次数(创建一个子节点也算一次修改)
dataVersion 节点数据的更新次数
aclVersion 节点ACL的更改次数 (权限)
ephemeralOwner 如果节点是临时节点,则表示创建该节点的SessionID;如果节点是持久节点则该属性值为0 (16进制)
dataLength 当前节点保存的数据长度
numChildren 当前节点下有多少个子节点

新增节点并在该节点下保存一个数据–create

create [-s] [-e] path data acl  
-s 有序节点加上该参数以后会在path前边自动拼接一堆有序数字
-e 临时节点,该节点数据生命周期是当前client断开数据删除(或主动删除)
path(路径) 必须从跟目录"/"开始
data 该节点的值 必须有
acl 权限控制
[zk: localhost:2181(CONNECTED) 23] create /netty "123456"
Created /netty
[zk: localhost:2181(CONNECTED) 24] ls /
[zookeeper, netty]

create 如果后边不跟参数创建的节点是持久节点跟当前会话无关。

create [-s] [-e] path data  

其中 s参数表示有序节点 e参数表示临时节点 path表示路径 data表示数据

创建持久有序节点

观察发现 持久化有序节点会给我们的节点名称后边追加一堆有序的数字。
应用场景:分布式环境中唯一ID的可以用它来做

[zk: localhost:2181(CONNECTED) 31] create -s /sort "1"
Created /sort0000000004
[zk: localhost:2181(CONNECTED) 32] create -s /order "1"
Created /order0000000005
[zk: localhost:2181(CONNECTED) 33] get /sort0000000004
1
cZxid = 0xf
ctime = Fri Aug 14 17:28:59 CST 2020
mZxid = 0xf
mtime = Fri Aug 14 17:28:59 CST 2020
pZxid = 0xf
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 1
numChildren = 0
临时节点创建

特点:创建的节点跟数据,在当前会话终端后 或者说当前客户跟zookeeper断开后数据节点会自动删除。

[zk: localhost:2181(CONNECTED) 12] create -e /temp "1"   
Created /temp
[zk: localhost:2181(CONNECTED) 13] ls /
[temp, sort0000000006, zookeeper, sort0000000004, netty, order0000000005]
[zk: localhost:2181(CONNECTED) 14] 

#关闭我们的client重新连接
[zk: localhost:2181(CONNECTED) 14] quit 
Quitting...
2020-08-14 17:36:59,188 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x100008f9ef50001 closed
2020-08-14 17:36:59,190 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x100008f9ef50001
[root@10 bin]# ./zkCli.sh 
Connecting to localhost:2181
....
#这里发现我们创建的临时节点已经被删除了
[zk: localhost:2181(CONNECTED) 0] ls /
[sort0000000006, zookeeper, sort0000000004, netty, order0000000005]
临时有序节点创建

同时使用 -s -e参数 就可以创建临时有序节点,当客户端跟zookeeper连接短了以后,数据就会消失。

[zk: localhost:2181(CONNECTED) 1] create -s -e /temp "temp"
Created /temp0000000008
[zk: localhost:2181(CONNECTED) 2] create -s -e /temp2 "temp2"
Created /temp20000000009
[zk: localhost:2181(CONNECTED) 3] ls /
[zookeeper, order0000000005, sort0000000006, sort0000000004, temp20000000009, netty, temp0000000008]
[zk: localhost:2181(CONNECTED) 4] 
[zk: localhost:2181(CONNECTED) 4] quit
Quitting...
2020-08-14 17:43:13,239 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x100008f9ef50002 closed
2020-08-14 17:43:13,241 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x100008f9ef50002
[root@10 bin]# ./zkCli.sh 
Connecting to localhost:2181
## 查看刚才我们创建的临时有序节点都被删除了
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, order0000000005, sort0000000006, sort0000000004, netty]

修改节点数据–set

set path data [version]
path(路径) 必须从跟目录"/"开始
data 值
version 就是dataVersion 如果指定的version 不等于dataVersion修改会失败(一般用来处理并发时候的锁机制)
#查看根目录下有那些节点
[zk: localhost:2181(CONNECTED) 7] ls /
[zookeeper, order0000000005, sort0000000006, sort0000000004, netty]
#查看netty节点下的数据
[zk: localhost:2181(CONNECTED) 8] get /netty
123456
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0xd
mtime = Fri Aug 14 17:16:17 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0
#修改netty节点的数据 发现dataVersion 变成了1 dataLength 变成了8
[zk: localhost:2181(CONNECTED) 9] set /netty "12345678"
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0x1d
mtime = Fri Aug 14 17:49:57 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 0
#查看修改以后的数据 发现已经变了说明修改成功
[zk: localhost:2181(CONNECTED) 10] get /netty
12345678
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0x1d
mtime = Fri Aug 14 17:49:57 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 0

首次创建该节点的时候 dataVersion =0然后每次修改它会自动给增加一。
我们还可以通过dataVersion 这个版本号来当乐观锁来判断是否可以修改。

set path data [version]
后边的那个version参数就是当前数据的那个dataVersion 。
如果加上version参数这个值必须等于dataVersion  否则无法修改成功。

例如:

#再次修改 让版本号加1
[zk: localhost:2181(CONNECTED) 14] set /netty "1234"
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0x1e
mtime = Fri Aug 14 17:58:42 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 0
#然后指定版本号为 1 然后再去修改 发现无法修改了
[zk: localhost:2181(CONNECTED) 15] set /netty "12345" 1
version No is not valid : /netty
#将版本号 改为跟数据节点中的版本号一致,那么可以修改成功。
[zk: localhost:2181(CONNECTED) 16] set /netty "12345" 2
cZxid = 0xd
ctime = Fri Aug 14 17:16:17 CST 2020
mZxid = 0x20
mtime = Fri Aug 14 17:58:59 CST 2020
pZxid = 0xd
cversion = 0
dataVersion = 3
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

节点删除 delete 和rmr

 一、 delete path [version]  #删除某一个节点 也可以带上版本控制
跟set 命令差不多。
 二、 rmr path 如果要删除的某一个节点下有子节点,删除该节点需要这个命令
[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper, order0000000005, sort0000000006, sort0000000004, netty]
#直接删除
[zk: localhost:2181(CONNECTED) 2] delete /sort0000000004 
[zk: localhost:2181(CONNECTED) 3] ls /
[zookeeper, order0000000005, sort0000000006, netty]
# 查看一下节点sort0000000006的数据跟版本号
[zk: localhost:2181(CONNECTED) 4] get /sort0000000006
1
cZxid = 0x14
ctime = Fri Aug 14 17:33:45 CST 2020
mZxid = 0x14
mtime = Fri Aug 14 17:33:45 CST 2020
pZxid = 0x14
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 1
numChildren = 0
# 这里输入不正确的版本号去删除,发现删除失败
[zk: localhost:2181(CONNECTED) 5] delete /sort0000000006 1
version No is not valid : /sort0000000006
#输入正确的版本号去删除,发现删除成功
[zk: localhost:2181(CONNECTED) 6] delete /sort0000000006 0
[zk: localhost:2181(CONNECTED) 7] ls /
[zookeeper, order0000000005, netty]

如果一个节点下边还有子节点,想要一次删除该节点跟子节点
需要使用rmr命令

rmr path(节点路径,必须从“/”开始)
# 在hadoop节点下再创建一个node1节点
[zk: localhost:2181(CONNECTED) 2] create /hadoop "1234"      
Created /hadoop
[zk: localhost:2181(CONNECTED) 3] create /hadoop/node1 "node1"
Created /hadoop/node1
#查看数据是否正常
[zk: localhost:2181(CONNECTED) 4] get /hadoop
1234
[zk: localhost:2181(CONNECTED) 0] get /hadoop/node1
node1
#测试delete 删除 --- 无法删除
[zk: localhost:2181(CONNECTED) 1] delete /hadoop
Node not empty: /hadoop
# 使用rmr命令 删除
[zk: localhost:2181(CONNECTED) 2] rmr /hadoop
[zk: localhost:2181(CONNECTED) 3] ls /
[zookeeper, order0000000005, netty]

查看当前节点属性 --stat

 stat path 只返回当前节点属性不包括当前节点下的值
[zk: localhost:2181(CONNECTED) 5] stat /hadoop
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x35
mtime = Fri Aug 14 20:57:07 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 2

配额限制 quota

个人感觉这个命令有点鸡肋,设置完以后操作不会有提示该成功还是成功,只是在日志里边有答应超额的提示
n参数表示最大可以创建几个子节点(创建到n的时候就是超数量)
b参数表示该节点最大可以存储数据的大小

  • setquota -n|-b val path #设置
  • delquota [-n|-b] path #删除
  • listquota path # 查看某一个节点上的限制
[zk: 192.168.56.101:2181(CONNECTED) 18] setquota -n 2  /node4 
#set
[zk: 192.168.56.101:2181(CONNECTED) 18] setquota -b 2  /node5
#list
[zk: 192.168.56.101:2181(CONNECTED) 28] listquota /node5
absolute path is /zookeeper/quota/node5/zookeeper_limits
Output quota for /node5 count=-1,bytes=2
Output stat for /node5 count=1,bytes=5
#del 这里需要注意的是,删除了以后它的absolute path不会被删除
#再次设置会报错,所以需要手动删除
[zk: 192.168.56.101:2181(CONNECTED) 30] delquota -n 2 /node5
[zk: 192.168.56.101:2181(CONNECTED) 33] rmr /zookeeper/quota/node5

history 与 redo

histroy 产看历史执行的命令 redo 加上历史执行命令的id可以重新执行这个命令

[zk: 192.168.56.101:2181(CONNECTED) 36] history
26 - create /node5 node5
27 - setquota -b 2 /node5
28 - listquota /node5
29 - setquota -n 2 /node5
30 - delquota -n 2 /node5
31 - listquota /node5
32 - setquota -n 2 /node5
33 - rmr /zookeeper/quota/node5
34 - help
35 - redo
36 - history
[zk: 192.168.56.101:2181(CONNECTED) 37] redo 28
absolute path is /zookeeper/quota/node5/zookeeper_limits
quota for /node5 does not exist.

sync

集群模式下,通知zookeeper同步一下机群中的数据。

监听器[watch]

使用[watch] 注册的监听器能够在节点内容发生变化时候,向客户端发出通知。需要注意的是触发是一次性的,触发以后就失效了

get path [watch]
#向hadoop节点注册监听器
[zk: localhost:2181(CONNECTED) 14] get /hadoop watch  
12345
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x3c
mtime = Fri Aug 14 21:48:00 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 4
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 2
#去修改这个节点的属性 发现立刻有消息通知
[zk: localhost:2181(CONNECTED) 12] set /hadoop "1234"

WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/hadoop
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x3b
mtime = Fri Aug 14 21:47:35 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 3
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
# 再次去修改发现没有通知了,说明了时间通知是一次性的
[zk: localhost:2181(CONNECTED) 13] set /hadoop "12345"
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x3c
mtime = Fri Aug 14 21:48:00 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 4
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 2
stat path [watch]
[zk: localhost:2181(CONNECTED) 5] stat /hadoop watch
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x3f
mtime = Fri Aug 14 21:58:46 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 7
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 1
numChildren = 2
[zk: localhost:2181(CONNECTED) 6] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/hadoop

#新的shell中去修改
[zk: localhost:2181(CONNECTED) 18] set /hadoop "12"
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x40
mtime = Fri Aug 14 21:58:58 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 8
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 2
numChildren = 2
ls/ls2 path [watch]

ls/ls2注册的监听器 监听的该节点下子点的变化状态

#监听hadoop节点的子节点
[zk: localhost:2181(CONNECTED) 19] ls /hadoop watch
[node2, node1]

#新shell中操作
#修改hadoop节点 发现监听没有起作用
[zk: localhost:2181(CONNECTED) 6] set /hadoop "123"
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x41
mtime = Fri Aug 14 22:01:36 CST 2020
pZxid = 0x36
cversion = 2
dataVersion = 9
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 2
#在该节点下创建一个新节点
[zk: localhost:2181(CONNECTED) 7] create /hadoop/node3 "123"
Created /hadoop/node3

#发现收到了通知
[zk: localhost:2181(CONNECTED) 20] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/hadoop

测试ls2

#使用ls2监听
[zk: localhost:2181(CONNECTED) 20] ls2 /hadoop watch
[node2, node3, node1]
cZxid = 0x2f
ctime = Fri Aug 14 20:50:13 CST 2020
mZxid = 0x41
mtime = Fri Aug 14 22:01:36 CST 2020
pZxid = 0x42
cversion = 3
dataVersion = 9
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 3

#新shell中去删除
[zk: localhost:2181(CONNECTED) 8] delete /hadoop/node3

#收到时间消息
[zk: localhost:2181(CONNECTED) 21] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/hadoop

zookeeper权限控制ACl

Acl 全称 acess control list

授权模式:

授权模式 描述
world 只是一个用户:anyone,代表登陆到zookeeper的所有人(默认)
ip 对客户端使用IP地址的认证
auth 使用已经添加认证的用户认证
digest 使用“用户名称:密码”方式认证

权限描述:

权限 简写 描述
create c 可以创建子节点
delete d 可以创建子节点(只能操作当前节点的子节点)
read r 可以读取子节点数据及子节点列表
write w 可以设置节点数据
admin a 可以设置节点访问控制权限

权限控制使用到的命令:

命令 使用方式 描述
setAcl setAcl path acl 设置Acl
getAcl getAcl path 获取Acl
addauth addauth scheme auth 添加认证用户
getAcll与setAcll使用
world模式
#首先初始数 我在/node创建了两个节点 node1 node2 节点对用的值都是节点的名字
[zk: localhost:2181(CONNECTED) 37] ls /node
[node2, node1]
#查看node节点权限
[zk: localhost:2181(CONNECTED) 40] getAcl /node
'world,'anyone
: cdrwa
#设置node权限 将create权限去掉
[zk: localhost:2181(CONNECTED) 43] setAcl /node world:anyone:drwa
cZxid = 0x4e
ctime = Fri Aug 14 22:40:25 CST 2020
mZxid = 0x4e
mtime = Fri Aug 14 22:40:25 CST 2020
pZxid = 0x50
cversion = 2
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
[zk: localhost:2181(CONNECTED) 44] getAcl /node
'world,'anyone
: drwa
#无法在改节点下创建
[zk: localhost:2181(CONNECTED) 45] create /node/node3 node3
Authentication is not valid : /node/node3
#将删除权限也去掉
[zk: localhost:2181(CONNECTED) 46] setAcl /node world:anyone:rwa
cZxid = 0x4e
ctime = Fri Aug 14 22:40:25 CST 2020
mZxid = 0x4e
mtime = Fri Aug 14 22:40:25 CST 2020
pZxid = 0x50
cversion = 2
dataVersion = 0
aclVersion = 2
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
#无法删除了
[zk: localhost:2181(CONNECTED) 47] delete /node/node2
Authentication is not valid : /node/node2
#将读权限也去掉
[zk: localhost:2181(CONNECTED) 48] setAcl /node world:anyone:wa 
cZxid = 0x4e
ctime = Fri Aug 14 22:40:25 CST 2020
mZxid = 0x4e
mtime = Fri Aug 14 22:40:25 CST 2020
pZxid = 0x50
cversion = 2
dataVersion = 0
aclVersion = 3
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
[zk: localhost:2181(CONNECTED) 49] get /node
Authentication is not valid : /node
#对子节点的写操作 发现是可以写的
[zk: localhost:2181(CONNECTED) 50] get /node/node2      
node2
cZxid = 0x50
ctime = Fri Aug 14 22:41:03 CST 2020
mZxid = 0x50
mtime = Fri Aug 14 22:41:03 CST 2020
pZxid = 0x50
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
[zk: localhost:2181(CONNECTED) 51] set /node/node2 node22
cZxid = 0x50
ctime = Fri Aug 14 22:41:03 CST 2020
mZxid = 0x56
mtime = Fri Aug 14 22:49:49 CST 2020
pZxid = 0x50
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0

#将该节点的写操作也去掉
zk: localhost:2181(CONNECTED) 52] setAcl /node world:anyone:a 
cZxid = 0x4e
ctime = Fri Aug 14 22:40:25 CST 2020
mZxid = 0x4e
mtime = Fri Aug 14 22:40:25 CST 2020
pZxid = 0x50
cversion = 2
dataVersion = 0
aclVersion = 4
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
#发现可以对该节点下的子节点进行写操作
[zk: localhost:2181(CONNECTED) 53] set /node/node2 node22     
cZxid = 0x50
ctime = Fri Aug 14 22:41:03 CST 2020
mZxid = 0x58
mtime = Fri Aug 14 22:50:34 CST 2020
pZxid = 0x50
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0
#但是对当前节点无法进行操作了
[zk: localhost:2181(CONNECTED) 58] set /node node1
Authentication is not valid : /node
#将admin权限也去掉 发现getAcl也无法使用了
[zk: localhost:2181(CONNECTED) 59] setAcl /node world:anyone: 
cZxid = 0x4e
ctime = Fri Aug 14 22:40:25 CST 2020
mZxid = 0x4e
mtime = Fri Aug 14 22:40:25 CST 2020
pZxid = 0x50
cversion = 2
dataVersion = 0
aclVersion = 5
ephemeralOwner = 0x0
dataLength = 4
numChildren = 2
[zk: localhost:2181(CONNECTED) 60] getAcl node
Command failed: java.lang.IllegalArgumentException: Path must start with / character
ip模式

需要用使用到不同机器的ip,我这里启动了两台虚拟机器
在这里插入图片描述
101启动zookeeper服务,102使用客户去连接101
注意:102必须将zookeeper下载解压,不用启动zkServer。
同时注意防火墙是否关闭了。
不然会报下边的异常

2020-08-15 16:19:57,498 [myid:] - INFO  [main-SendThread(192.168.56.101:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server 192.168.56.101/192.168.56.101:2181. Will not attempt to authenticate using SASL (unknown error)

远程连接zookeeper命令

 ./zkCli.sh -server 192.168.56.101:2181

使用演示:

#创建节点
[zk: localhost:2181(CONNECTED) 65] create /ipnode ip
#给ip 192.168.56.102授权
[zk: localhost:2181(CONNECTED) 70] setAcl /ipnode ip:192.168.56.102:cdrwa
cZxid = 0x61
ctime = Fri Aug 14 23:24:56 CST 2020
mZxid = 0x61
mtime = Fri Aug 14 23:24:56 CST 2020
pZxid = 0x61
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 2
numChildren = 0
#发现192.168.56.101没有改权限了
[zk: localhost:2181(CONNECTED) 71] getAcl /ipnode
Authentication is not valid : /ipnode
#发现192.168.56.102 可以最该节点操作
[zk: 192.168.56.101:2181(CONNECTED) 0] getAcl /ipnode
'ip,'192.168.56.102
: cdrwa

实际开发中我们一般都是集群所以要给多个IP赋权限

setAcl /node1 ip:机器ip:cdrwa,ip:机器ip:cdrwa
#给多个ip分别设置权限
[zk: 192.168.56.101:2181(CONNECTED) 2] setAcl /node1 ip:192.168.56.101:cdrwa,ip:192.168.56.102:cdrwa

[zk: 192.168.56.101:2181(CONNECTED) 7] getAcl node2
Command failed: java.lang.IllegalArgumentException: Path must start with / character
[zk: 192.168.56.101:2181(CONNECTED) 8] getAcl /node2
'ip,'192.168.56.101
: cdrwa
'ip,'192.168.56.102
: cdrwa

注意:本地连接zookeeper ip授权没有权限问题
如果我们在本地连接的zookeeper的服务器也就是使用的默认连接方式[root@10 bin]# ./zkCli.sh 登陆的zookeeper那么在ip设置权限的时候就注意了。
你必须保证 /etc/hosts文件中有localhost 跟本机局域网Ip的映射关系,要不然你设置了本机的ip地址发现你根本无法访问。

为了避免这个坑,建议使用zkCli.sh的时候指定ip加端口方式。

auth模式
#添加认证用户
[zk: 192.168.56.101:2181(CONNECTED) 2] addauth digest root:123456
#设置权限
[zk: 192.168.56.101:2181(CONNECTED) 3] setAcl /node auth:root:acrwd

[zk: 192.168.56.101:2181(CONNECTED) 4] getAcl /node
'digest,'root:u53OoA8hprX59uwFsvQBS3QuI00=
: cdrwa

断开连接或者在其他客户端里边查看该节点
我这里是在102那台机器上测试的

#先获取该节点数据 发现没有权限
[zk: 192.168.56.101:2181(CONNECTED) 15] get /node
Authentication is not valid : /node
#添加认证用户,然后再查看
[zk: 192.168.56.101:2181(CONNECTED) 16] addauth digest root:123456
[zk: 192.168.56.101:2181(CONNECTED) 17] get /node
node
cZxid = 0x17
ctime = Sat Aug 15 00:17:07 CST 2020
mZxid = 0x17
mtime = Sat Aug 15 00:17:07 CST 2020
pZxid = 0x17
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 4
numChildren = 0
digest模式

与auth模式类似

#先生成密文
[root@10 ~]# echo -n root:123456 | openssl dgst -binary -sha1 | openssl base64
u53OoA8hprX59uwFsvQBS3QuI00=

#连接客户端创建节点
[zk: 192.168.56.101:2181(CONNECTED) 7] create /node1 node1
Created /node1
#给节点设置权限
[zk: 192.168.56.101:2181(CONNECTED) 9] setAcl /node1 digest:root:u53OoA8hprX59uwFsvQBS3QuI00=:crdwa
#获取数据
[zk: 192.168.56.101:2181(CONNECTED) 0] get /node1
Authentication is not valid : /node1
#创建授权用户
[zk: 192.168.56.101:2181(CONNECTED) 2] addauth digest root:123456
#再次去获取数据
[zk: 192.168.56.101:2181(CONNECTED) 3] get /node1
node1
cZxid = 0x1b
ctime = Sat Aug 15 00:56:47 CST 2020
mZxid = 0x1b
mtime = Sat Aug 15 00:56:47 CST 2020
pZxid = 0x1b
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
ip auth digest混合授权模式

#先在linux生成一个用户的密钥

[root@localhost ~]# echo -n zl:123456 | openssl dgst -binary -sha1 | openssl base64    
kA/2lgytmE7EijplBMNROqsOlao=

#授权演示

#创建一个节点
[zk: 192.168.56.101:2181(CONNECTED) 9] create /node2
#查看权限
[zk: 192.168.56.101:2181(CONNECTED) 3] getAcl /node2
'world,'anyone
: cdrwa
#添加一个auth认证用户
[zk: 192.168.56.101:2181(CONNECTED) 4] addauth digest root:123456
#这里去做授权
[zk: 192.168.56.101:2181(CONNECTED) 5] setAcl /node2 auth:root:cdrwa,ip:192.168.56.102:cdrwa,digest:zl:kA/2lgytmE7EijplBMNROqsOlao=:cdrwa
cZxid = 0x30
ctime = Sun Aug 16 11:23:08 CST 2020
mZxid = 0x30
mtime = Sun Aug 16 11:23:08 CST 2020
pZxid = 0x30
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
[zk: 192.168.56.101:2181(CONNECTED) 6] getAcl /node2
'ip,'192.168.56.102
: cdrwa
'digest,'zl:kA/2lgytmE7EijplBMNROqsOlao=
: cdrwa
'digest,'root:u53OoA8hprX59uwFsvQBS3QuI00=
: cdrwa

#更新权限 给101机器ip也赋给权限
[zk: 192.168.56.101:2181(CONNECTED) 7] setAcl /node2 auth:root:cdrwa,ip:192.168.56.102:cdrwa,digest:zl:kA/2lgytmE7EijplBMNROqsOlao=:cdrwa,ip:192.168.56.101:cdrwa
cZxid = 0x30
ctime = Sun Aug 16 11:23:08 CST 2020
mZxid = 0x30
mtime = Sun Aug 16 11:23:08 CST 2020
pZxid = 0x30
cversion = 0
dataVersion = 0
aclVersion = 2
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
#查看权限
[zk: 192.168.56.101(CONNECTED) 1] getAcl /node2
'ip,'192.168.56.102
: cdrwa
'digest,'zl:kA/2lgytmE7EijplBMNROqsOlao=
: cdrwa
'ip,'192.168.56.101
: cdrwa
'digest,'root:u53OoA8hprX59uwFsvQBS3QuI00=
: cdrwa
zookeeper中配置超级管理员

设置超级管理员需要在zookeeper的配置文件中添加该超级用户的信息,例如:账号 和 加密后的密码
加密后的密码同样用我们上边用到的密钥生成命令生成:
加入我的超级管理员是:root密码是123456

[root@localhost ~]# echo -n root:123456 | openssl dgst -binary -sha1 | openssl base64     
u53OoA8hprX59uwFsvQBS3QuI00=

修改zkServer.sh 文件 大概是在140行。

#找到
 nohup "$JAVA" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" 
 #在后边添加我们需要配置的超级管理员:
"-Dzookeeper.DigestAuthenticationProvider.superDigest=root:u53OoA8hprX59uwFsvQBS3QuI00=" 

修改后:

 nohup "$JAVA" "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" "-Dzookeeper.DigestAuthent    icationProvider.superDigest=root:u53OoA8hprX59uwFsvQBS3QuI00=" \

然后重启zookeeper的服务

[root@localhost bin]# vim zkServer.sh 
[root@localhost bin]# ./zkServer.sh  stop
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.14/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@localhost bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/zookeeper/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost bin]# ./zkCli.sh -server 192.168.56.101:2181

演示操作:

#创建一个节点,并将它的world权限的读权限去掉
[zk: 192.168.56.101:2181(CONNECTED) 1] create /node3 node3
Created /node3
[zk: 192.168.56.101:2181(CONNECTED) 2] setAcl /node3 world:anyone:cwad
cZxid = 0x3a
ctime = Sun Aug 16 11:50:48 CST 2020
mZxid = 0x3a
mtime = Sun Aug 16 11:50:48 CST 2020
pZxid = 0x3a
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
[zk: 192.168.56.101:2181(CONNECTED) 3] get /node3
Authentication is not valid : /node3

#然后我们使用我们配置的超级用户 root去访问
[zk: 192.168.56.101:2181(CONNECTED) 0] addauth digest root:123456
#可以正常访问
[zk: 192.168.56.101:2181(CONNECTED) 1] get /node3
node3
cZxid = 0x3a
ctime = Sun Aug 16 11:50:48 CST 2020
mZxid = 0x3a
mtime = Sun Aug 16 11:50:48 CST 2020
pZxid = 0x3a
cversion = 0
dataVersion = 0
aclVersion = 1
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
[zk: 192.168.56.101:2181(CONNECTED) 2] getAcl /node3
'world,'anyone
: cdwa

基本命令使用到这里就完了。实际开发中我们一般都会使用代码去连接zookeeper然后对zookeeper进行操作。
关注我的博客,我会抽时间将java 操作zookeeper的笔记也发出来。
可能我写的笔记里边有错误的地方,欢迎评论区指正。

你可能感兴趣的:(zookeeper,zookeeper,分布式,大数据,hadoop)