Zookeeper的简单运用 -- 基于ZK的分布式共享锁、应用系统服务器上下线动态感知
Zookeeper的简单运用
1. ZooKeeper Java API
-
org.apache.zookeeper.Zookeeper:
Zookeeper 是在 Java 中客户端主类，负责建立与 zookeeper 集群的会话， 并提供方法进行操作。 -
org.apache.zookeeper.Watcher
Watcher 接口表示一个标准的事件处理器，其定义了事件通知相关的逻辑， 包含 KeeperState 和 EventType 两个枚举类，分别代表了通知状态和事件类型， 同时定义了事件的回调方法：process（WatchedEvent event）。
process 方法是 Watcher 接口中的一个回调方法，当 ZooKeeper 向客户端发送一个 Watcher 事件通知时，客户端就会对相应的 process 方法进行回调，从而实现对事件的处理。
2. 简单使用
先新建项目，解压zookeeper-3.4.5.tar.gz ，并导入解压包中的zookeeper-3.4.5.jar 以及 解压包中的该路径zookeeper-3.4.5\lib下的所有jar包
节点的增、删、改、查代码如下：
package com.mylove.bigdata.zk;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
public class SimpleZkClient {
private static final String connectString = "mini1:2181,mini2:2181,mini3:2181";
private static final int sessionTimeout = 2*1000;
ZooKeeper zkClient = null;
@Before
public void init() throws Exception{
zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher(){
@Override
public void process(WatchedEvent event) {
// 收到事件通知的回调函数
System.out.println(event.getType() +"----"+event.getPath());
try {
zkClient.getChildren("/", true);
} catch (KeeperException | InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
/**
* 创建数据节点
* @param args
* @throws Exception
*/
@Test
public void testCreate() throws Exception {
// args[0]:要创建的节点路径 args[1]:节点的数据 args[2]:节点的权限 args[3]:节点的类型
//存入的数据可以是任何类型，必须转化成byte
String nodeCreated = zkClient.create("/eclipse", "hellozk".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}
@Test
public void testExist() throws Exception{
Stat stat = zkClient.exists("/eclipse", false);
System.out.println(stat == null ? "not exist":"exist");
}
/**
* 获取子节点
* @throws Exception
*/
@Test
public void getChildren() throws Exception{
List children = zkClient.getChildren("/", true);
for(String child : children){
System.out.println(child);
}
Thread.sleep(Long.MAX_VALUE);
}
/**
* 获取zookeeper中的数据
* @throws Exception
*/
@Test
public void getData() throws Exception{
byte[] data = zkClient.getData("/eclipse", false, null);
System.out.println(new String(data,"utf-8"));
}
/**
* 删除znode
* @throws Exception
*/
@Test
public void deleteZnode() throws Exception{
//参数2指定要删除的版本，-1表示删除所有版本
zkClient.delete("/eclipse", -1);
}
/**
* 修改节点数据
* @throws Exception
*/
@Test
public void setData() throws Exception{
zkClient.setData("/app1", "i miss you mylove".getBytes(), -1);
byte[] data = zkClient.getData("/app1", false, null);
System.out.println(new String(data,"utf-8"));
}
}
3.服务器的动态上下线感知的简单实现
需求：某分布式系统中，主节点可以有多台，可以动态上下线，任意一台客户端都能实时感知到主节点服务器的上下线
服务提供方代码实现如下：
package com.mylove.bigdata.zkdist;
import java.io.File;
import java.io.IOException;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
public class DistributedServer {
private static final String connectString = "mini1:2181,mini2:2181,mini3:2181";
private static final int sessionTimeout = 2*1000;
private static final String PARENTNODE = "/servers";
private ZooKeeper zkClient = null;
//建立连接
public void getConnect() throws Exception{
zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher(){
@Override
public void process(WatchedEvent event) {
// 收到事件通知的回调函数
System.out.println(event.getType() +"----"+event.getPath());
try {
zkClient.getChildren(PARENTNODE, true);
} catch (KeeperException | InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
//向zk集群注册服务器信息
public void registerServer(String hostName) throws Exception{
String create = zkClient.create(PARENTNODE+"/server", hostName.getBytes(),Ids.OPEN_ACL_UNSAFE , CreateMode.EPHEMERAL_SEQUENTIAL);
System.out.println(hostName + " is online ..." + create);
}
//业务功能
public void handleBussiness(String hostName) throws InterruptedException{
System.out.println(hostName + " start working.... ");
Thread.sleep(Long.MAX_VALUE);
}
public static void main(String[] args) throws Exception {
//获取zk连接
DistributedServer distributedServer = new DistributedServer();
distributedServer.getConnect();
//利用zk连接注册服务器信息
distributedServer.registerServer(args[0]);
//启动业务功能
distributedServer.handleBussiness(args[0]);
}
}
服务消费方代码实现如下：
package com.mylove.bigdata.zkdist;
import java.util.ArrayList;
import java.util.List;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
public class DistributedClient {
private static final String connectString = "mini1:2181,mini2:2181,mini3:2181";
private static final int sessionTimeout = 2*1000;
private static final String PARENTNODE = "/servers";
private volatile List serverList;
private ZooKeeper zkClient = null;
//建立连接
public void getConnect() throws Exception{
zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher(){
@Override
public void process(WatchedEvent event) {
// 收到事件通知的回调函数
System.out.println(event.getType() +"----"+event.getPath());
try {
//重新更新服务器列表，并注册监听
getServerList();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
/**
* 获取服务器信息列表
* @throws Exception
*/
public void getServerList() throws Exception{
//获取服务器子节点信息，并对父节点进行监听
List children = zkClient.getChildren(PARENTNODE, true);
List servers = new ArrayList();
for(String child:children){
byte[] data = zkClient.getData(PARENTNODE+"/"+child, false, null);
servers.add(new String(data));
}
// 把serverse赋值给成员变量serverList，已提供给个业务线程使用
serverList = servers;
System.out.println(serverList);
}
//业务功能
public void handleBussiness() throws InterruptedException{
System.out.println("Client start working.... ");
Thread.sleep(Long.MAX_VALUE);
}
public static void main(String[] args) throws Exception {
//获取zk连接
DistributedClient client = new DistributedClient();
client.getConnect();
//获取servers的子节点信息，从中获取服务器的服务列表
client.getServerList();
//客户端业务功能
client.handleBussiness();
}
}
运行服务提供方的main函数，并传入参数mini1，再启动服务消费方，消费方立刻感知服务服务器mini1；再次运行服务提供方的main函数，并传入参数mini2，消费方此时感知服务服务器mini1、mini2在线；停止mini1服务2s后，消费方感知此时只有服务器mini2在线。
4.分布式共享锁的简单实现
分布式应用场景下，多台客户端都需要访问服务端的某个资源，但同一时刻只能有一个客户端访问。
实现逻辑：
- 客户端节点启动时到zk上注册一个 ”短暂+序号“的znode，并监听父节点
- 获取父节点下的所有子节点，比较序号的大小
- 序号最小的客户端获取到"锁"，去访问新的资源，访问完成后，删除自己的节点(相当于释放锁)，并重新注册一个新的子节点。
- 其他程序节点会收到事件通知，则可以去zk上获取锁
分析流程图如下：
代码实现：
package com.mylove.bigdata.zklock;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.Watcher.Event.EventType;
import org.apache.zookeeper.ZooDefs.Ids;
public class DistributedClientLock {
private static final String connectString = "mini1:2181,mini2:2181,mini3:2181";
private static final int sessionTimeout = 2*1000;
private static final String GROUPNODE = "/locks";
private static final String SUBNODE = "sub";
private ZooKeeper zkClient = null;
private String clientName;
//记录自己创建的子节点路径
private volatile String thisPath;
public void connectZookeeper() throws Exception{
zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher(){
@Override
public void process(WatchedEvent event) {
try {
// 判断事件类型，此处只处理子节点变化事件
if(event.getType() == EventType.NodeChildrenChanged && event.getPath().equals(GROUPNODE)){
//获取子节点并对父节点进行监听
List childrenNodes = zkClient.getChildren(GROUPNODE, true);
String thisNode = thisPath.substring((GROUPNODE+"/").length());
//排序后比较自己是否是最小的id
Collections.sort(childrenNodes);
if(childrenNodes.indexOf(thisNode) == 0 ){
//访问共享资源处理业务，并且在处理完成之后删除锁
handleBussiness();
//重新注册一把新的锁
thisPath = zkClient.create(GROUPNODE+"/"+SUBNODE, null, Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL);
}
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
//1、建立连接时先注册一把锁
thisPath = zkClient.create(GROUPNODE+"/"+SUBNODE, null, Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
//休眠便于观察
Thread.sleep(new Random().nextInt(1000));
//从zk锁的父目录下，获取所有子节点，并且注册对父节点的监听
List childrenNodes = zkClient.getChildren(GROUPNODE, true);
//如果争抢资源的程序就只有自己，则可以直接去访问共享资源
if(childrenNodes.size() == 1 ){
handleBussiness();
thisPath = zkClient.create(GROUPNODE+"/"+SUBNODE, null, Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
}
}
//业务逻辑处理，并且在最后释放锁
private void handleBussiness() throws Exception {
System.out.println( this.clientName + " gain lock : "+thisPath);
//处理具体的业务逻辑
try {
Thread.sleep(5*1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
System.out.println(this.clientName +" finished : " + thisPath );
//业务逻辑处理完成之后，删除thisPath节点,参数-1表示删除所有版本
zkClient.delete(this.thisPath, -1);
}
}
public String getClientName() {
return clientName;
}
public void setClientName(String clientName) {
this.clientName = clientName;
}
public static void main(String[] args) throws Exception {
DistributedClientLock distributedClientLock = new DistributedClientLock();
distributedClientLock.connectZookeeper();
Thread.sleep(Long.MAX_VALUE);
}
}