ZooKeeper3.4.6学习笔记（一）集群配置
鲁春利的工作笔记，谁说程序员不能有文艺范？
安装JDK1.6或更高版本
略
下载Zookeeper
从地址http://zookeeper.apache.org/下载zookeeper安装包
解压 % tar -xzv -f zookeeper-x.y.z.tar.gz
配置zookeeper环境变量
# hadoop体系的软件均通过hadoop用户来运行 vim /home/hadoop/.bash_profile export ZOOKEEPER_HOME=/usr/local/zookeeper3.4.6 export PATH=$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf:$PATH
配置文件zoo.cfg
# 将$ZOOKEEPER_HOME$\conf目录下的zoo_sample.cfg文件复制一个并重命名为zoo.cfg [hadoop@nnode conf]$ cat zoo.cfg # Zookeeper基本时间单元(滴答时间，单位为毫秒)，其他运行时时间间隔为tickTime的倍数 tickTime=2000 # 配置Leader服务器等待Follower启动，并完成数据同步的时间。Follower服务器在启动的过程中， # 会与Leader服务器建立连接并完成对数据的同步，从而确定自己对外提供服务的起始状态，Leader # 服务器允许Follower在initLimit时间内(10个tickTime)完成这个工作。 initLimit=10 # 配置Leader服务器和Follower之间进行心跳检测的最大延迟时间。在Zookeeper集群运行过程中， # Leader服务器会与所有的Follower进行心跳检测来确认该服务器是否存活。如果Leader服务器在 # syncLimit时间内(5个tickTime)无法获取到Follower的心跳检测响应，那么Leader就会认为该 # Follower已经脱离了与自己的同步。 syncLimit=5 # 配置Zookeeper服务器存储快照文件的目录。 dataDir=/usr/local/zookeeper3.4.6/data # 配置Zookeeper服务器存储事务日志文件的目录，默认为dataDir dataLogDir=/usr/local/zookeeper3.4.6/logs # 服务器对外提供的服务端口，客户端通过该端口与Zookeeper服务进行连接 clientPort=2181 # 支持的客户端最大连接数 maxClientCnxns=100 # # Be sure to read the maintenance section of the administrator guide before turning on autopurge. # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # 从3.4.0开始，Zookeeper提供了对历史事务日志和快照数据自动清理的支持。 # autopurge.snapRetainCount配置Zookeeper在自动清理时保留的快照文件数量和对应的事务日志文件。 # autopurge.snapRetainCount最小值为3，如果小于3则自动调整为3(至少保留3个)。 # autopurge.snapRetainCount=3 # 自动清除的时间间隔，以小时为单，如果设置为0，将不会执行该任务。默认值为0。 # autopurge.purgeInterval=1 # server.id=host:port:port # 单机模式 # server.1=nnode:2888:3888 # 伪集群模式 # server.1=nnode:2888:3888 # server.2=nnode:2889:3889 # server.3=nnode:2890:3890 # 集群模式 server.1=nnode:2888:3888 server.2=dnode1:2888:3888 server.3=dnode2:2888:3888 # 参数说明： # id：ServerID(1-255)，标识机器在集群中的序号，与dataDir目录下myid文件中的内容对应； # host：机器的主机名； # port：Follower服务器与Leader进行运行时通信和数据同步时使用的端口； # port：执行Leader选举时服务器相互通信的端口。 # 伪集群host都一样，Zookeeper实例通信端口号不能一样，所以要给它们分配不同的端口。
创建myid文件
在集群模式下，每台机器的dataDir目录下都需要创建一个myid文件，文件只有一行，写的是一个数字，该数字需要与zoo.cfg中当前机器的ServerID一致，如：
server.1=host1:2888:3888，那么host1主机的dataDir目录下的myid文件中只有一个数字为1； server.2=hosta:2888:3888，那么host1主机的dataDir目录下的myid文件中只有一个数字为2。
将配置好的zookeeper分发到其他机器，并修改myid文件
# nnode [hadoop@nnode data]$ pwd /usr/local/zookeeper3.4.6/data [hadoop@nnode data]$ cat myid 1 [hadoop@nnode data]$ # dnode1 [hadoop@dnode1 data]$ pwd /usr/local/zookeeper3.4.6/data [hadoop@dnode1 data]$ cat myid 2 [hadoop@dnode1 data]$ # dnode2 [hadoop@dnode2 data]$ pwd /usr/local/zookeeper3.4.6/data [hadoop@dnode2 data]$ cat myid 3 [hadoop@dnode2 data]$
ZooKeeper可执行脚本
[hadoop@nnode bin]$ ll total 60 -rwxr-xr-x 1 hadoop hadoop 238 Feb 20 2014 README.txt -rwxr-xr-x 1 hadoop hadoop 1937 Feb 20 2014 zkCleanup.sh -rwxr-xr-x 1 hadoop hadoop 1049 Feb 20 2014 zkCli.cmd -rwxr-xr-x 1 hadoop hadoop 1534 Feb 20 2014 zkCli.sh -rwxr-xr-x 1 hadoop hadoop 1333 Feb 20 2014 zkEnv.cmd -rwxr-xr-x 1 hadoop hadoop 2756 Jun 8 14:10 zkEnv.sh -rwxr-xr-x 1 hadoop hadoop 1084 Feb 20 2014 zkServer.cmd -rwxr-xr-x 1 hadoop hadoop 5742 Feb 20 2014 zkServer.sh [hadoop@nnode bin]$
脚本说明
zkServer.sh：ZooKeeper服务器的启动、停止和重启脚本； zkCli.sh：ZooKeeper的简易客户端； zkEnv.sh：设置ZooKeeper的环境变量； zkCleanup.sh：清理ZooKeeper历史数据，包括事务日志文件和快照数据文件。
启动服务
[hadoop@nnode ~]$ zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [hadoop@nnode ~]$ [hadoop@dnode1 ~]$ zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [hadoop@dnode1 ~]$ [hadoop@dnode1 ~]$ zkServer.sh start JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [hadoop@dnode1 ~]$
验证状态
[hadoop@nnode ~]$ zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Mode: follower [hadoop@nnode ~]$ [hadoop@dnode1 ~]$ zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Mode: leader [hadoop@dnode1 ~]$ [hadoop@dnode2 data]$ zkServer.sh status JMX enabled by default Using config: /usr/local/zookeeper3.4.6/bin/../conf/zoo.cfg Mode: follower [hadoop@dnode2 data]$
通过telnet验证
# nnode [hadoop@nnode ~]$ telnet 127.0.0.1 2181 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. stat Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT Clients: /127.0.0.1:38315[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/0 Received: 2 Sent: 1 Connections: 1 Outstanding: 0 Zxid: 0x3200000025 Mode: follower # follower Node count: 39 Connection closed by foreign host. # dnode1 [hadoop@nnode ~]$ telnet dnode1 2181 Trying 192.168.137.118... Connected to dnode1. Escape character is '^]'. stat Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT Clients: /192.168.137.117:45071[0](queued=0,recved=1,sent=0) Latency min/avg/max: 0/0/0 Received: 2 Sent: 1 Connections: 1 Outstanding: 0 Zxid: 0x3300000000 Mode: leader # leader Node count: 39 Connection closed by foreign host.
客户端脚本
[hadoop@nnode bin]$ zkCli.sh Connecting to localhost:2181 // ...... WATCHER:: WatchedEvent state:SyncConnected type:None path:null # 读取znode数据 [zk: localhost:2181(CONNECTED) 0] ls / [hbase, hadoop-ha, zookeeper] # 第一次部署Zookeeper集群，默认创建名为/zookeeper的节点 [zk: localhost:2181(CONNECTED) 1] ls /zookeeper [quota] [zk: localhost:2181(CONNECTED) 2] ls /zookeeper/quota [] # 读取znode数据 [zk: localhost:2181(CONNECTED) 3] get /zookeeper/quota cZxid = 0x0 ctime = Thu Jan 01 08:00:00 CST 1970 mZxid = 0x0 mtime = Thu Jan 01 08:00:00 CST 1970 pZxid = 0x0 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 0 # 创建znode [zk: localhost:2181(CONNECTED) 4] create /zk-book 123 Created /zk-book [zk: localhost:2181(CONNECTED) 5] ls / [zk-book, hbase, hadoop-ha, zookeeper] [zk: localhost:2181(CONNECTED) 6] get /zk-book 123 cZxid = 0x3400000006 ctime = Sat Aug 08 21:18:47 CST 2015 mZxid = 0x3400000006 mtime = Sat Aug 08 21:18:47 CST 2015 pZxid = 0x3400000006 cversion = 0 dataVersion = 0 # version的版本为0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 # 更新znode的值 [zk: localhost:2181(CONNECTED) 7] set /zk-book 456 cZxid = 0x3400000006 ctime = Sat Aug 08 21:18:47 CST 2015 mZxid = 0x3400000007 mtime = Sat Aug 08 21:19:54 CST 2015 pZxid = 0x3400000006 cversion = 0 dataVersion = 1 # 更新后version的版本为1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 [zk: localhost:2181(CONNECTED) 8] get /zk-book 456 cZxid = 0x3400000006 ctime = Sat Aug 08 21:18:47 CST 2015 mZxid = 0x3400000007 mtime = Sat Aug 08 21:19:54 CST 2015 pZxid = 0x3400000006 cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 # 删除 [zk: localhost:2181(CONNECTED) 9] delete /zk-book WATCHER:: WatchedEvent state:SyncConnected type:NodeDeleted path:/zk-book [zk: localhost:2181(CONNECTED) 10] ls / # 再次查看发现/zk-book节点已被成功删除 [hbase, hadoop-ha, zookeeper] # 退出zkCli [zk: localhost:2181(CONNECTED) 11] quit Quitting... 2015-08-08 21:21:06,508 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x14f0d5f49760002 closed 2015-08-08 21:21:06,510 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@512] - EventThread shut down [hadoop@nnode bin]$
说明：
连接远程Zookeeper服务器zkCli.sh -server ip:port
通过help命令可以查询命令使用的详细信息
停止服务
# 在每台机器上依次执行zkServer.sh stop [hadoop@nnode~]$zkServer.sh stop
zkCli.sh脚本解析
#!/bin/sh
# $0：脚本本身的名字，${ZOOKEEPER_HOME}/bin/zkCli.sh
ZOOBIN="${BASH_SOURCE-$0}" # ${ZOOKEEPER_HOME}/bin/zkCli.sh
ZOOBIN=`dirname ${ZOOBIN}` # ${ZOOKEEPER_HOME}/bin
ZOOBINDIR=`cd ${ZOOBIN}; pwd` # ${ZOOKEEPER_HOME}/bin
# 如果存在${ZOOKEEPER_HOME}/libexec/zkEnv.sh文件
if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
. "$ZOOBINDIR"/../libexec/zkEnv.sh
else # 否则调用${ZOOKEEPER_HOME}/bin/zkEnv.sh，点表示当前目录
. "$ZOOBINDIR"/zkEnv.sh
fi
# ZOO_LOG_DIR在zkEnv.sh中配置，如${ZOOKEEPER_HOME}/log，zkServer.sh中会将zookeeper.out生成在该目录下
# ZOO_LOG4J_PROP : INFO,CONSOLE
# CLASSPATH : Zookeeper的类路径
# $@ ：传给脚本的所有参数的列表
# java ：zkEnv.sh里面配置的$JAVA_HOME/bin/java
$JAVA "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
-cp "$CLASSPATH" $CLIENT_JVMFLAGS $JVMFLAGS org.apache.zookeeper.ZooKeeperMain "$@"
zkCli.sh中实际调用的是ZooKeeperMain类，在该类中实际是接收客户端的输入，并在后台调用：
package org.apache.zookeeper;
/**
* The command line client to ZooKeeper.
*/
public class ZooKeeperMain {
protected static final Map<String,String> commandMap = new HashMap<String,String>( );
protected MyCommandOptions cl = new MyCommandOptions(); // 内部类
/**
public MyCommandOptions() {
options.put("server", "localhost:2181");
options.put("timeout", "30000");
}
*/
protected HashMap<Integer,String> history = new HashMap<Integer,String>( );
protected int commandCount = 0;
protected boolean printWatches = true;
public static void main(String args[]) throws KeeperException, IOException, InterruptedException {
ZooKeeperMain main = new ZooKeeperMain(args);
main.run();
}
@SuppressWarnings("unchecked")
void run() throws KeeperException, IOException, InterruptedException {
if (cl.getCommand() == null) {
System.out.println("Welcome to ZooKeeper!");
boolean jlinemissing = false;
try {
Class consoleC = Class.forName("jline.ConsoleReader");
Class completorC = Class.forName("org.apache.zookeeper.JLineZNodeCompletor");
System.out.println("JLine support is enabled");
Object console = consoleC.getConstructor().newInstance();
Object completor = completorC.getConstructor(ZooKeeper.class).newInstance(zk);
Method addCompletor = consoleC.getMethod("addCompletor", Class.forName("jline.Completor"));
addCompletor.invoke(console, completor);
String line;
Method readLine = consoleC.getMethod("readLine", String.class);
while ((line = (String)readLine.invoke(console, getPrompt())) != null) {
executeLine(line);
}
} catch (ClassNotFoundException e) {
// 略
jlinemissing = true;
}
if (jlinemissing) {
System.out.println("JLine support is disabled"); // System.in
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String line;
while ((line = br.readLine()) != null) {
executeLine(line);
}
}
}
}
// 其他代码略
}
# 实际上在ZooKeeperMain中调用了Zookeeper的客户端类org.apache.zookeeper.ZooKeeper。