Nacos源码分析之Raft协议(三)----数据的同步

前言

上两篇文章主要分析了leader选举以及发送心跳包的代码，如果还有疑问的小伙伴，可以点击传送门再去温习一下。

leader选举源码分析
leader发送心跳源码分析

接下来进入正题，今天我们主要是分析上篇文章没有说完的数据同步相关的代码。

Nacos数据的同步

以服务实例的数据同步为例，数据同步的主要核心的地方在于ConsistencyService的put()方法，ConsistencyService是一个接口，而针对于raft算法的实现类有RaftConsistencyServiceImpl，我们找到RaftConsistencyServiceImpl的put方法

    @Override
    public void put(String key, Record value) throws NacosException {
        try {
            raftCore.signalPublish(key, value);
        } catch (Exception e) {
            Loggers.RAFT.error("Raft put failed.", e);
            throw new NacosException(NacosException.SERVER_ERROR, "Raft put failed, key:" + key + ", value:" + value,
                    e);
        }
    }

这段代码主要的核心在于 raftCore.signalPublish(key, value);所以我们需要进入到raftCore的signalPublish()方法中

public void signalPublish(String key, Record value) throws Exception {

        //不是leader，转发到leader
        if (!isLeader()) {
            ObjectNode params = JacksonUtils.createEmptyJsonNode();
            params.put("key", key);
            params.replace("value", JacksonUtils.transferToJsonNode(value));
            Map parameters = new HashMap<>(1);
            parameters.put("key", key);

            final RaftPeer leader = getLeader();

            raftProxy.proxyPostLarge(leader.ip, API_PUB, params.toString(), parameters);
            return;
        }

        try {
            OPERATE_LOCK.lock();
            final long start = System.currentTimeMillis();
            final Datum datum = new Datum();
            datum.key = key;
            datum.value = value;
            //如果是新增数据
            if (getDatum(key) == null) {
                datum.timestamp.set(1L);
            } else {
                datum.timestamp.set(getDatum(key).timestamp.incrementAndGet());
            }

            ObjectNode json = JacksonUtils.createEmptyJsonNode();
            //放入数据和本机信息
            json.replace("datum", JacksonUtils.transferToJsonNode(datum));
            json.replace("source", JacksonUtils.transferToJsonNode(peers.local()));
            //将数据写入本地磁盘
            onPublish(datum, peers.local());

            final String content = json.toString();
            //CountDownLatch 用于控制过半提交
            final CountDownLatch latch = new CountDownLatch(peers.majorityCount());
            //循环所有机器发送请求 （经过前面的判断，只有leader能走到这里）
            for (final String server : peers.allServersIncludeMyself()) {
                if (isLeader(server)) {
                    latch.countDown();
                    continue;
                }
                final String url = buildUrl(server, API_ON_PUB);
                HttpClient.asyncHttpPostLarge(url, Arrays.asList("key=" + key), content,
                    new AsyncCompletionHandler() {
                        @Override
                        public Integer onCompleted(Response response) throws Exception {
                            if (response.getStatusCode() != HttpURLConnection.HTTP_OK) {
                                Loggers.RAFT
                                    .warn("[RAFT] failed to publish data to peer, datumId={}, peer={}, http code={}",
                                        datum.key, server, response.getStatusCode());
                                return 1;
                            }
                            //请求成功 执行latch.countDown();
                            latch.countDown();
                            return 0;
                        }

                        @Override
                        public STATE onContentWriteCompleted() {
                            return STATE.CONTINUE;
                        }
                    });

            }
            // latch.await（long timeout，TimeUnit unit）如果在5s内没有过半的机器同意 那么抛出异常
            if (!latch.await(UtilsAndCommons.RAFT_PUBLISH_TIMEOUT, TimeUnit.MILLISECONDS)) {
                // only majority servers return success can we consider this update success
                Loggers.RAFT.error("data publish failed, caused failed to notify majority, key={}", key);
                throw new IllegalStateException("data publish failed, caused failed to notify majority, key=" + key);
            }

            long end = System.currentTimeMillis();
            Loggers.RAFT.info("signalPublish cost {} ms, key: {}", (end - start), key);
        } finally {
            OPERATE_LOCK.unlock();
        }
    }

在这段代码中主要做了一下几件事:
1.判断本机是不是leader，不是则将请求转发到leader
2.加锁，对数据写入到本地磁盘中（onPublish()方法）
3.向集群中的机器发起请求(/v1/ns/raft/datum/commit)，利用countDownLatch控制过半提交（如果5s内没有过半的机器同意，那么抛出异常）
4.释放锁

我们首先看onPublish()方法做了什么

public void onPublish(Datum datum, RaftPeer source) throws Exception {
        RaftPeer local = peers.local();
        if (datum.value == null) {
            Loggers.RAFT.warn("received empty datum");
            throw new IllegalStateException("received empty datum");
        }

        if (!peers.isLeader(source.ip)) {
            Loggers.RAFT
                .warn("peer {} tried to publish data but wasn't leader, leader: {}", JacksonUtils.toJson(source),
                    JacksonUtils.toJson(getLeader()));
            throw new IllegalStateException("peer(" + source.ip + ") tried to publish " + "data but wasn't leader");
        }

        if (source.term.get() < local.term.get()) {
            Loggers.RAFT.warn("out of date publish, pub-term: {}, cur-term: {}", JacksonUtils.toJson(source),
                JacksonUtils.toJson(local));
            throw new IllegalStateException(
                "out of date publish, pub-term:" + source.term.get() + ", cur-term: " + local.term.get());
        }

        local.resetLeaderDue();

        // if data should be persisted, usually this is true:
        //如果是持久化数据 写入到磁盘
        if (KeyBuilder.matchPersistentKey(datum.key)) {
            raftStore.write(datum);
        }

        datums.put(datum.key, datum);

        if (isLeader()) {
            //写入一次数据任期增加100
            local.term.addAndGet(PUBLISH_TERM_INCREASE_COUNT);
        } else {
            //更新任期
            if (local.term.get() + PUBLISH_TERM_INCREASE_COUNT > source.term.get()) {
                //set leader term:
                getLeader().term.set(source.term.get());
                local.term.set(getLeader().term.get());
            } else {
                //term增加100
                local.term.addAndGet(PUBLISH_TERM_INCREASE_COUNT);
            }
        }
        //修改磁盘的任期
        raftStore.updateTerm(local.term.get());
        //在监听中的队列中添加一个修改的事件
        notifier.addTask(datum.key, ApplyAction.CHANGE);

        Loggers.RAFT.info("data added/updated, key={}, term={}", datum.key, local.term);
    }

这段代码主要做一下几件事：
1.首先做一系列的判断，排除异常逻辑，包括数据的value是不是空，任期是否正常等
2.先把数据写入到磁盘，然后写入到本地内存中
3.更新任期 没修改一次数据term +100
4.在监听中的队列中添加一个修改的事件

然后我们看其他节点在接受到leader请求时是如何处理的，我们查看/v1/ns/raft/datum/commit接口的代码

    @PostMapping("/datum/commit")
    public String onPublish(HttpServletRequest request, HttpServletResponse response) throws Exception {

        response.setHeader("Content-Type", "application/json; charset=" + getAcceptEncoding(request));
        response.setHeader("Cache-Control", "no-cache");
        response.setHeader("Content-Encode", "gzip");

        String entity = IoUtils.toString(request.getInputStream(), "UTF-8");
        String value = URLDecoder.decode(entity, "UTF-8");

        JsonNode jsonObject = JacksonUtils.toObj(value);
        String key = "key";

        RaftPeer source = JacksonUtils.toObj(jsonObject.get("source").toString(), RaftPeer.class);
        JsonNode datumJson = jsonObject.get("datum");

        Datum datum = null;
        //根据不同数据类型进行处理
        if (KeyBuilder.matchInstanceListKey(datumJson.get(key).asText())) {
            datum = JacksonUtils.toObj(jsonObject.get("datum").toString(), new TypeReference>() {
            });
        } else if (KeyBuilder.matchSwitchKey(datumJson.get(key).asText())) {
            datum = JacksonUtils.toObj(jsonObject.get("datum").toString(), new TypeReference>() {
            });
        } else if (KeyBuilder.matchServiceMetaKey(datumJson.get(key).asText())) {
            datum = JacksonUtils.toObj(jsonObject.get("datum").toString(), new TypeReference>() {
            });
        }

        raftConsistencyService.onPut(datum, source);
        return "ok";
    }

主要的核心在于 raftConsistencyService.onPut(datum, source);我们进入到该方法中

public void onPut(Datum datum, RaftPeer source) throws NacosException {
        try {
            //在本地写入数据
            raftCore.onPublish(datum, source);
        } catch (Exception e) {
            Loggers.RAFT.error("Raft onPut failed.", e);
            throw new NacosException(NacosException.SERVER_ERROR,
                    "Raft onPut failed, datum:" + datum + ", source: " + source, e);
        }
    }

其实这个方法很简单，就是同样调用raftCore.onPublish()方法，将数据写入到本地中

至此，nacos的数据同步也就完成了。