Kafka在zookeeper里面的存储结构详解

 

当kafka启动的时候，就会向zookeeper里面注册一些信息，这些数据也称为Kafka的元数据信息。

一、zookeeper存储结构总图

1.根目录下的结构

[zk: localhost:2181(CONNECTED) 72] ls /
[isr_change_notification, zookeeper, admin, consumers, cluster, config, latest_producer_id_block, controller, brokers, controller_epoch]

2. admin的结构

[zk: localhost:2181(CONNECTED) 73] ls /admin 
[delete_topics]
[zk: localhost:2181(CONNECTED) 74] ls /admin/delete_topics
[]
[zk: localhost:2181(CONNECTED) 75] get /admin/delete_topics
null
cZxid = 0xe
ctime = Sun Sep 02 20:04:12 PDT 2018
mZxid = 0xe
mtime = Sun Sep 02 20:04:12 PDT 2018
pZxid = 0xe
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
[zk: localhost:2181(CONNECTED) 76] 

3.consumers的结构

 本环境上没有设置消费组

[zk: localhost:2181(CONNECTED) 78] ls /consumers
[]
[zk: localhost:2181(CONNECTED) 79] get /consumers
null
cZxid = 0x2
ctime = Sun Sep 02 20:04:12 PDT 2018
mZxid = 0x2
mtime = Sun Sep 02 20:04:12 PDT 2018
pZxid = 0x2
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
[zk: localhost:2181(CONNECTED) 80] 

4.config的结构

[zk: localhost:2181(CONNECTED) 82] ls /config     
[topics, clients, changes]
[zk: localhost:2181(CONNECTED) 83] ls /config/topics
[__consumer_offsets, test, test2]
[zk: localhost:2181(CONNECTED) 84] ls /config/topics/test2
[]
[zk: localhost:2181(CONNECTED) 85] get /config/topics/test2
{"version":1,"config":{}}
cZxid = 0xcc
ctime = Sun Sep 02 20:18:06 PDT 2018
mZxid = 0xcc
mtime = Sun Sep 02 20:18:06 PDT 2018
pZxid = 0xcc
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 25
numChildren = 0
[zk: localhost:2181(CONNECTED) 86] 

5.controllers的结构

[zk: localhost:2181(CONNECTED) 92] ls /controller
[]
[zk: localhost:2181(CONNECTED) 93] get /controller
{"version":1,"brokerid":0,"timestamp":"1535943857541"} //表示broker0为kafka的控制节点
cZxid = 0x15
ctime = Sun Sep 02 20:04:17 PDT 2018
mZxid = 0x15
mtime = Sun Sep 02 20:04:17 PDT 2018
pZxid = 0x15
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x1659d6268f60000
dataLength = 54
numChildren = 0
[zk: localhost:2181(CONNECTED) 94] 

6.brokers的结构

[zk: localhost:2181(CONNECTED) 95] ls /brokers
[seqid, topics, ids]
[zk: localhost:2181(CONNECTED) 96] ls /brokers/ids
[0]
[zk: localhost:2181(CONNECTED) 97] get /brokers/ids/0
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://localhost:9092"],"jmx_port":-1,"host":"localhost","timestamp":"1535943859255","port":9092,"version":4}
cZxid = 0x1c
ctime = Sun Sep 02 20:04:19 PDT 2018
mZxid = 0x1c
mtime = Sun Sep 02 20:04:19 PDT 2018
pZxid = 0x1c
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x1659d6268f60000
dataLength = 188
numChildren = 0
[zk: localhost:2181(CONNECTED) 98] 

------------------------------brokers/topics----------------------
[zk: localhost:2181(CONNECTED) 100] ls /brokers/topics
[__consumer_offsets, test, test2]
[zk: localhost:2181(CONNECTED) 101] ls /brokers/topics/test

test    test2
[zk: localhost:2181(CONNECTED) 101] ls /brokers/topics/test2
[partitions]
[zk: localhost:2181(CONNECTED) 102] ls /brokers/topics/test2/partitions
[2, 1, 0]
[zk: localhost:2181(CONNECTED) 103] ls /brokers/topics/test2/partitions/2
[state]
[zk: localhost:2181(CONNECTED) 104] ls /brokers/topics/test2/partitions/2/state
[]
[zk: localhost:2181(CONNECTED) 105] get /brokers/topics/test2/partitions/2/state
{"controller_epoch":1,"leader":0,"version":1,"leader_epoch":0,"isr":[0]}
//表示partition 0 的leader是在0 broker上
cZxid = 0xd2
ctime = Sun Sep 02 20:18:06 PDT 2018
mZxid = 0xd2
mtime = Sun Sep 02 20:18:06 PDT 2018
pZxid = 0xd2
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 72
numChildren = 0
[zk: localhost:2181(CONNECTED) 106]