[Curator] Tree Cache 的使用与分析

为什么80%的码农都做不了架构师?>>>   hot3.png

Tree Cache

使用zk路径下所有节点数据作为缓存。这个类会监控ZK的一个路径,路径下所有的节点变动,数据变动都会被响应。 还可以通过注册自定义监听器来更细节的控制这些数据变动操作。

1. 关键 API

org.apache.curator.framework.recipes.cache.TreeCache

org.apache.curator.framework.recipes.cache.TreeCacheListener

org.apache.curator.framework.recipes.cache.TreeCacheEvent

org.apache.curator.framework.recipes.cache.ChildData

2. 机制说明

与Path Cache类似

  • Builder模式
    • TreeCache属性较多,所以提供了一个TreeCache.Builder
  • Path Cache是一层结构
    • 按cache的name,挂载到path下
  • TreeCache
    • 顾名思义,内部是一个树结构
    • 所以有一个内部类TreeCache.TreeNode封装树节点信息

TreeCache使用一个内部类TreeNode来维护这个一个树结构。并将这个树结构与ZK节点进行了映射。

3. 用法

3.1 创建

public TreeCache(CuratorFramework client,
                         String path,
                         boolean cacheData)
  • cacheData
    • 如果设置true,是否需要缓存数据

3.2 使用

还是一样的套路,在使用前需要调用start();用完之后需要调用close()方法。

随时都可以调用getCurrentData()获取当前缓存的状态和数据。

也可以通过getListenable()获取监听器容器,并在此基础上增加自定义监听器:

public void addListener(NodeCacheListener listener)

不过与Path Cache,以及Node Cache不一样的是:

  • 多了一个getCurrentChildren()方法
    • 返回path下多个子节点的缓存数据
    • 封装成一个Map返回
    • 没有很精准的进行数据同步
      • 可以当作一份快照使用

4. 错误处理

TreeCache实例自带一个ConnectionStateListener处理链接状态的变化,并处理数据变动情况

5. 源码分析

5.1 类定义

public class TreeCache implements Closeable{}

  • 实现了java.io.Closeable

5.1.1 TreeCache.Builder

org.apache.curator.framework.recipes.cache.TreeCache.Builder

public static final class Builder{}

5.1.2 TreeCache.TreeNode

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode

private final class TreeNode implements Watcher, BackgroundCallback{}

5.2 成员变量

public class TreeCache implements Closeable
{
    private static final Logger LOG = LoggerFactory.getLogger(TreeCache.class);
    private final boolean createParentNodes;
    private final TreeCacheSelector selector;
    
    private static final AtomicReferenceFieldUpdater nodeStateUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, NodeState.class, "nodeState");

    private static final AtomicReferenceFieldUpdater childDataUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, ChildData.class, "childData");

    private static final AtomicReferenceFieldUpdater childrenUpdater =
            AtomicReferenceFieldUpdater.newUpdater(TreeNode.class, ConcurrentMap.class, "children");
            
    private final AtomicLong outstandingOps = new AtomicLong(0);

    private final AtomicBoolean isInitialized = new AtomicBoolean(false);

    private final TreeNode root;
    private final CuratorFramework client;
    private final ExecutorService executorService;
    private final boolean cacheData;
    private final boolean dataIsCompressed;
    private final int maxDepth;
    private final ListenerContainer listeners = new ListenerContainer();
    private final ListenerContainer errorListeners = new ListenerContainer();
    private final AtomicReference treeState = new AtomicReference(TreeState.LATENT);

    private final ConnectionStateListener connectionStateListener = new ConnectionStateListener()
    {
        @Override
        public void stateChanged(CuratorFramework client, ConnectionState newState)
        {
            handleStateChange(newState);
        }
    };

    static final ThreadFactory defaultThreadFactory = ThreadUtils.newThreadFactory("TreeCache");
}

可以看出,定义了很多变量。(所以额外的使用了Builder模式)

  • LOG
    • 私有常量
    • 静态内部类也要使用日志
  • createParentNodes
    • 是否需要创建父节点
    • 默认情况下,不会自动创建path的父节点
  • selector
    • org.apache.curator.framework.recipes.cache.TreeCacheSelector
    • 默认使用org.apache.curator.framework.recipes.cache.DefaultTreeCacheSelector
    • 用于区分哪些节点作为缓存使用
  • nodeStateUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的nodeState属性提供原子操作
  • childDataUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的childData属性提供原子操作
  • childrenUpdater
    • java.util.concurrent.atomic.AtomicReferenceFieldUpdater
    • 为TreeNode的children属性提供原子操作
  • outstandingOps
    • 记录未完成回调处理的任务数量
  • isInitialized
    • 是否已初始化
    • 是否已发出INITIALIZED事件
  • root
    • 树结构的根节点
    • 也可看作一个哨兵
  • client
  • executorService
    • 线程池
    • 老套路,操作异步处理。类似Path Cache
  • cacheData
    • 是否缓存数据
  • dataIsCompressed
    • 数据是否压缩
  • maxDepth
    • 树结构的最大深度
  • listeners
    • 监听器容器
    • TreeCacheListener缓存监听器
  • errorListeners
    • 监听器容器
    • UnhandledErrorListener监听器单独一个容器
    • TreeCacheListener监听器分开,利于对不同种类的监听器单独管理
  • treeState
    • AtomicReference
    • 内部枚举
      • LATENT
      • STARTED
      • CLOSED
  • defaultThreadFactory
    • 老套路,线程工厂。类似Path Cache
  • connectionStateListener
    • zk链接状态监听器
    • 老套路,回调状态机,异步处理事件

5.2.1 TreeNode

private final class TreeNode implements Watcher, BackgroundCallback
{

    volatile NodeState nodeState = NodeState.PENDING;
    volatile ChildData childData;
    final TreeNode parent;
    final String path;
    volatile ConcurrentMap children;
    final int depth;

    TreeNode(String path, TreeNode parent)
    {
        this.path = path;
        this.parent = parent;
        this.depth = parent == null ? 0 : parent.depth + 1;
    }
}
  • 实现了org.apache.zookeeper.Watcher接口
  • 实现了org.apache.curator.framework.api.BackgroundCallback接口
  • nodeState
    • org.apache.curator.framework.recipes.cache.TreeCache.NodeState
    • 节点状态
      • PENDING 等待处理
      • LIVE 有效
      • DEAD 无效
  • childData
    • org.apache.curator.framework.recipes.cache.ChildData
    • 节点数据
  • parent
    • 父节点
  • path
    • 对应的zk节点路径
  • children
    • java.util.concurrent.ConcurrentMap
    • 子节点列表
  • depth
    • 当前节点所处深度(层级)

5.3 构造器

public TreeCache(CuratorFramework client, String path)
{
    this(client, path, true, false, Integer.MAX_VALUE, Executors.newSingleThreadExecutor(defaultThreadFactory), false, new DefaultTreeCacheSelector());
}

TreeCache(CuratorFramework client, String path, boolean cacheData, boolean dataIsCompressed, int maxDepth, final ExecutorService executorService, boolean createParentNodes, TreeCacheSelector selector)
{
    this.createParentNodes = createParentNodes;
    this.selector = Preconditions.checkNotNull(selector, "selector cannot be null");
    this.root = new TreeNode(validatePath(path), null);
    this.client = Preconditions.checkNotNull(client, "client cannot be null");
    this.cacheData = cacheData;
    this.dataIsCompressed = dataIsCompressed;
    this.maxDepth = maxDepth;
    this.executorService = Preconditions.checkNotNull(executorService, "executorService cannot be null");
}

可以发现:

  • 默认处理线程池是一个单线程线程池
  • 默认缓存节点数据
  • 默认不对数据进行压缩
  • 默认不控制树的深度
  • 默认不创建父节点
  • 使用默认的DefaultTreeCacheSelector
  • 初始化了root节点信息

除了两个构造器,还有Builderbuild方法来创建TreeCache

private final CuratorFramework client;
private final String path;
private boolean cacheData = true;
private boolean dataIsCompressed = false;
private ExecutorService executorService = null;
private int maxDepth = Integer.MAX_VALUE;
private boolean createParentNodes = false;
private TreeCacheSelector selector = new DefaultTreeCacheSelector();

public TreeCache build()
{
    ExecutorService executor = executorService;
    if ( executor == null )
    {
        executor = Executors.newSingleThreadExecutor(defaultThreadFactory);
    }
    return new TreeCache(client, path, cacheData, dataIsCompressed, maxDepth, executor, createParentNodes, selector);
}

创建过程也没有特殊的设定,各属性默认值也都一致

5.4 启动

使用之前需要调用start()方法

public TreeCache start() throws Exception
{
    Preconditions.checkState(treeState.compareAndSet(TreeState.LATENT, TreeState.STARTED), "already started");
    if ( createParentNodes )
    {
        client.createContainers(root.path);
    }
    client.getConnectionStateListenable().addListener(connectionStateListener);
    if ( client.getZookeeperClient().isConnected() )
    {
        root.wasCreated();
    }
    return this;
}
  1. 原子更新状态
  2. 如果需要创建父节点,则创建
  3. 在链接上增加监听器
  4. 如果链接是已连接,则调用root.wasCreated()root节点开始逐个同步数据

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#wasCreated:

void wasCreated() throws Exception
{
    refresh();
}

private void refresh() throws Exception
{
    if ((depth < maxDepth) && selector.traverseChildren(path))
    {
        outstandingOps.addAndGet(2);
        doRefreshData();
        doRefreshChildren();
    } else {
        refreshData();
    }
}
private void refreshData() throws Exception
{
    outstandingOps.incrementAndGet();
    doRefreshData();
}

  1. 如果深度(树层级)允许,则下探(开始两个任务)
    1. 刷新当前节点数据
    2. 刷新下层节点列表
  2. 如果无需下探,则仅刷新当前结点的数据
  • 每次调用doXxxxx方法,则创建一个任务
    • 使用outstandingOps跟踪任务数量
刷新节点数据
private void doRefreshData() throws Exception
{
    if ( dataIsCompressed )
    {
        client.getData().decompressed().usingWatcher(this).inBackground(this).forPath(path);
    }
    else
    {
        client.getData().usingWatcher(this).inBackground(this).forPath(path);
    }
}
  • 根据是否使用数据压缩,调用不同的获取数据的方法
  • 都使用了回掉处理
    • TreeNode本身已经实现了Watcher, BackgroundCallback两个接口,所以回调/监听对象都是this

回调: org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#processResult:

public void processResult(CuratorFramework client, CuratorEvent event) throws Exception
    {
        LOG.debug("processResult: {}", event);
        Stat newStat = event.getStat();
        switch ( event.getType() )
        {
        case EXISTS:
            Preconditions.checkState(parent == null, "unexpected EXISTS on non-root node");
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                nodeStateUpdater.compareAndSet(this, NodeState.DEAD, NodeState.PENDING);
                wasCreated();
            }
            break;
        case CHILDREN:
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                ChildData oldChildData = childData;
                if ( oldChildData != null && oldChildData.getStat().getMzxid() == newStat.getMzxid() )
                {
                    // Only update stat if mzxid is same, otherwise we might obscure
                    // GET_DATA event updates.
                    childDataUpdater.compareAndSet(this, oldChildData, new ChildData(oldChildData.getPath(), newStat, oldChildData.getData()));
                }

                if ( event.getChildren().isEmpty() )
                {
                    break;
                }

                ConcurrentMap childMap = children;
                if ( childMap == null )
                {
                    childMap = Maps.newConcurrentMap();
                    if ( !childrenUpdater.compareAndSet(this, null, childMap) )
                    {
                        childMap = children;
                    }
                }

                // Present new children in sorted order for test determinism.
                List newChildren = new ArrayList();
                for ( String child : event.getChildren() )
                {
                    if ( !childMap.containsKey(child) && selector.acceptChild(ZKPaths.makePath(path, child)) )
                    {
                        newChildren.add(child);
                    }
                }

                Collections.sort(newChildren);
                for ( String child : newChildren )
                {
                    String fullPath = ZKPaths.makePath(path, child);
                    TreeNode node = new TreeNode(fullPath, this);
                    if ( childMap.putIfAbsent(child, node) == null )
                    {
                        node.wasCreated();
                    }
                }
            }
            else if ( event.getResultCode() == KeeperException.Code.NONODE.intValue() )
            {
                wasDeleted();
            }
            break;
        case GET_DATA:
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                ChildData toPublish = new ChildData(event.getPath(), newStat, event.getData());
                ChildData oldChildData;
                if ( cacheData )
                {
                    oldChildData = childDataUpdater.getAndSet(this, toPublish);
                }
                else
                {
                    oldChildData = childDataUpdater.getAndSet(this, new ChildData(event.getPath(), newStat, null));
                }

                boolean added;
                if (parent == null) {
                    // We're the singleton root.
                    added = nodeStateUpdater.getAndSet(this, NodeState.LIVE) != NodeState.LIVE;
                } else {
                    added = nodeStateUpdater.compareAndSet(this, NodeState.PENDING, NodeState.LIVE);
                    if (!added) {
                        // Ordinary nodes are not allowed to transition from dead -> live;
                        // make sure this isn't a delayed response that came in after death.
                        if (nodeState != NodeState.LIVE) {
                            return;
                        }
                    }
                }

                if ( added )
                {
                    publishEvent(TreeCacheEvent.Type.NODE_ADDED, toPublish);
                }
                else
                {
                    if ( oldChildData == null || oldChildData.getStat().getMzxid() != newStat.getMzxid() )
                    {
                        publishEvent(TreeCacheEvent.Type.NODE_UPDATED, toPublish);
                    }
                }
            }
            else if ( event.getResultCode() == KeeperException.Code.NONODE.intValue() )
            {
                wasDeleted();
            }
            break;
        default:
            // An unknown event, probably an error of some sort like connection loss.
            LOG.info(String.format("Unknown event %s", event));
            // Don't produce an initialized event on error; reconnect can fix this.
            outstandingOps.decrementAndGet();
            return;
        }

        if ( outstandingOps.decrementAndGet() == 0 )
        {
            if ( isInitialized.compareAndSet(false, true) )
            {
                publishEvent(TreeCacheEvent.Type.INITIALIZED);
            }
        }
    }
}
  • 状态机 这里先来看看在启动时获取数据的逻辑:

  • GET_DATA

    1. 如果成功获取了zk节点数据
      1. 构建新的ChildData
      2. 如果需要对数据进行缓存
        1. 使用AtomicReferenceFieldUpdater原子化操作覆盖当前结点的数据
      3. 如果不需要对数据进行缓存
        1. 则使用一个不含数据的ChildData覆盖当前结点的数据
      4. 检查当前结点是否已经加入到缓存中
        • 通过判断节点状态来实现
      5. 如果发现为节点已经DEAD无效了,则只是覆盖更新数据,而再不处理,直接返回
      6. 如果当前结点是第一次被添加到树中
        1. 触发NODE_ADDED事件
      7. 如果已经在树中,则判断数据是否有更新
        1. 如果数据有更新,则触发NODE_UPDATED事件
    2. 如果发现zk节点已经不存在了
      1. 则进行删除书节点操作wasDeleted()
  • 当任务计数器outstandingOps为0时,说明同步任务全部完成

    • 更新isInitialized状态
    • 触发了INITIALIZED事件

org.apache.curator.framework.recipes.cache.TreeCache.TreeNode#wasDeleted:

void wasDeleted() throws Exception
{
    ChildData oldChildData = childDataUpdater.getAndSet(this, null);
    client.clearWatcherReferences(this);
    ConcurrentMap childMap = childrenUpdater.getAndSet(this,null);
    if ( childMap != null )
    {
        ArrayList childCopy = new ArrayList(childMap.values());
        childMap.clear();
        for ( TreeNode child : childCopy )
        {
            child.wasDeleted();
        }
    }

    if ( treeState.get() == TreeState.CLOSED )
    {
        return;
    }

    NodeState oldState = nodeStateUpdater.getAndSet(this, NodeState.DEAD);
    if ( oldState == NodeState.LIVE )
    {
        publishEvent(TreeCacheEvent.Type.NODE_REMOVED, oldChildData);
    }

    if ( parent == null )
    {
        // Root node; use an exist query to watch for existence.
        client.checkExists().usingWatcher(this).inBackground(this).forPath(path);
    }
    else
    {
        // Remove from parent if we're currently a child
        ConcurrentMap parentChildMap = parent.children;
        if ( parentChildMap != null )
        {
            parentChildMap.remove(ZKPaths.getNodeFromPath(path), this);
        }
    }
}
  1. 将当前节点数据清空
  2. 清除节点上的监听器
  3. 制空子节点列表
    1. 调用每一个节点的wasDeleted()
    • 树结构从上到下的清理动作
  4. 如果整个树状态都已经关闭,则不需要在处理了
  5. 如果树状态正常,只是清理某个分支则:
    1. 则将当前结点状态更新为DEAD无效
    2. 触发NODE_REMOVED事件
    3. 如果父节点为空(自身就是根节点)
      1. 则对zk节点进行检查(检查树对应的zk节点根节点是否还存在)
    4. 如果父节点不为空(树中的某个分支节点)
      1. 则将自己从父节点的子节点列表中移除
刷新子节点列表
private void doRefreshChildren() throws Exception
{
    client.getChildren().usingWatcher(this).inBackground(this).forPath(path);
}

仍然是一个回调自身processResult状态机的方式。

  • 这里看看CHILDREN的逻辑
    1. 如果以及成功获取了ZK子节点列表
      1. 如果子节点列表没有变化(ZK节点没变化,不是说本地列表没变化)
        1. 覆盖更新当前结点的子节点列表
        • 只是更新了Stat信息,后续逻辑会判断data的相等性,如果这里也覆盖了data部分,则会导致后续逻辑任务数据发生了更新,从而导致不必要的GET_DATA事件
      2. 如果子节点列表是空的,自然不需要再做什么处理了
      3. 开始构建children
        • 考虑到了children的并发
          • 先用原子操作
          • 如果失败,说明有其他线程已经更新了,则使用新值
      4. 对节点列表进行梳理排序后,加入子节点列表
        1. 逐个调用子节点的wasCreated
          • 从上到下的加入一个树分支
    2. 如果发现ZK节点不存在了
      1. 则进行删除书节点操作wasDeleted()
        • 参见上一节

5.5 获取缓存

有两个方式来获取缓存

5.5.1 获取指定节点缓存数据

调用:ChildData getCurrentData(String fullPath)

看看源码:

public ChildData getCurrentData(String fullPath)
{
    TreeNode node = find(fullPath);
    if ( node == null || node.nodeState != NodeState.LIVE )
    {
        return null;
    }
    ChildData result = node.childData;
    // Double-check liveness after retreiving data.
    return node.nodeState == NodeState.LIVE ? result : null;
}
  1. 查找到对应的节点
  2. 获取节点数据
  3. Double check

所以,重点在于节点的查找过程:

private TreeNode find(String findPath)
{
    PathUtils.validatePath(findPath);
    LinkedList rootElements = new LinkedList(ZKPaths.split(root.path));
    LinkedList findElements = new LinkedList(ZKPaths.split(findPath));
    while (!rootElements.isEmpty()) {
        if (findElements.isEmpty()) {
            // Target path shorter than root path
            return null;
        }
        String nextRoot = rootElements.removeFirst();
        String nextFind = findElements.removeFirst();
        if (!nextFind.equals(nextRoot)) {
            // Initial root path does not match
            return null;
        }
    }

    TreeNode current = root;
    while (!findElements.isEmpty()) {
        String nextFind = findElements.removeFirst();
        ConcurrentMap map = current.children;
        if ( map == null )
        {
            return null;
        }
        current = map.get(nextFind);
        if ( current == null )
        {
            return null;
        }
    }
    return current;
}
  1. 分离出缓存节点对应路径
    1. root.pathfindPath的每一层(/分隔)逐一比较
    2. findPath中,去掉root.path的部分
  2. 从root节点开始,按剩余的findPath部分逐一查找
  3. 直到最后匹配的节点

5.5.2 获取一组子节点缓存数据

调用:Map getCurrentChildren(String fullPath)

public Map getCurrentChildren(String fullPath)
{
    TreeNode node = find(fullPath);
    if ( node == null || node.nodeState != NodeState.LIVE )
    {
        return null;
    }
    ConcurrentMap map = node.children;
    Map result;
    if ( map == null )
    {
        result = ImmutableMap.of();
    }
    else
    {
        ImmutableMap.Builder builder = ImmutableMap.builder();
        for ( Map.Entry entry : map.entrySet() )
        {
            TreeNode childNode = entry.getValue();
            ChildData childData = childNode.childData;
            // Double-check liveness after retreiving data.
            if ( childData != null && childNode.nodeState == NodeState.LIVE )
            {
                builder.put(entry.getKey(), childData);
            }
        }
        result = builder.build();
    }

    // Double-check liveness after retreiving children.
    return node.nodeState == NodeState.LIVE ? result : null;
}
  1. 查找对应的节点
  2. 逐个获取子节点数据
  • 返回的是一个不可变Map

5.6 关闭缓存

使用完缓存需要调用close()

public void close()
{
    if ( treeState.compareAndSet(TreeState.STARTED, TreeState.CLOSED) )
    {
        client.getConnectionStateListenable().removeListener(connectionStateListener);
        listeners.clear();
        executorService.shutdown();
        try
        {
            root.wasDeleted();
        }
        catch ( Exception e )
        {
            ThreadUtils.checkInterrupted(e);
            handleException(e);
        }
    }
}
  1. 原子化更新状态
  2. 移除链接状态监听器
  3. 清理监听器容器
  4. 关闭线程池
  5. 删除root节点

6. 小结

对比项 Path Cache Node Cache Tree Cache
缓存数量 多个 单个 多个
使用 管理着一组缓存,类似一个CacheManager 单个name的缓存 有层级关系的CacheManager
机制 状态机,事件驱动(对重复操作进行过滤),单线程,任务队列 状态机,回调驱动 状态机,单线程,事件驱动,使用任务计数器跟踪

转载于:https://my.oschina.net/roccn/blog/956654

你可能感兴趣的:([Curator] Tree Cache 的使用与分析)