通过查看LongPollingClient的构造方法，我们看到，只有一个地方构造了该类
//org.dromara.soul.admin.listener.http.HttpLongPollingDataChangedListener#doLongPolling
public void doLongPolling(final HttpServletRequest request, final HttpServletResponse response) {
// 比较当前的请求是否有group配置发生变化
List changedGroup = compareChangedGroup(request);
String clientIp = getRemoteIp(request);
// 如果变化的group不为空，则立即返回结果
if (CollectionUtils.isNotEmpty(changedGroup)) {
this.generateResponse(response, changedGroup);
log.info("send response with the changed group, ip={}, group={}", clientIp, changedGroup);
return;
}
// 开启异步处理
final AsyncContext asyncContext = request.startAsync();
// AsyncContext.settimeout() does not timeout properly, so you have to control it yourself
//这里关闭了timeout，所以我们需要自己控制好
asyncContext.setTimeout(0L);
// 这里便是创建LongPollingClient，即承接上篇文章的构造部分，并交给scheduler处理
scheduler.execute(new LongPollingClient(asyncContext, clientIp, HttpConstants.SERVER_MAX_HOLD_TIMEOUT));
}
这里看出，是接受了一个请求并，判断是否需要立刻返回变化结果，如果没有立刻发现有配置变化，则会走异步处理
我们在看下这个方法在哪里调用。发现是在ConfigController
public class ConfigController {
@Resource
private HttpLongPollingDataChangedListener longPollingListener;
/**
* Fetch configs soul result.
*
* @param groupKeys the group keys
* @return the soul result
*/
@GetMapping("/fetch")
public SoulAdminResult fetchConfigs(@NotNull final String[] groupKeys) {
Map> result = Maps.newHashMap();
for (String groupKey : groupKeys) {
ConfigData data = longPollingListener.fetchConfig(ConfigGroupEnum.valueOf(groupKey));
result.put(groupKey, data);
}
return SoulAdminResult.success(SoulResultMessage.SUCCESS, result);
}
/**
* Listener.
*
* @param request the request
* @param response the response
*/
@PostMapping(value = "/listener")
public void listener(final HttpServletRequest request, final HttpServletResponse response) {
longPollingListener.doLongPolling(request, response);
}
}
这里我们看到，提供了两种方式，一种是主动拉的方式，fetch。
fetch方式主要实现：就是拉取对应group的配置信息
public ConfigData fetchConfig(final ConfigGroupEnum groupKey) {
ConfigDataCache config = CACHE.get(groupKey.name());
switch (groupKey) {
case APP_AUTH:
List appAuthList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken>() {
}.getType());
return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), appAuthList);
case PLUGIN:
List pluginList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken>() {
}.getType());
return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), pluginList);
case RULE:
List ruleList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken>() {
}.getType());
return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), ruleList);
case SELECTOR:
List selectorList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken>() {
}.getType());
return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), selectorList);
case META_DATA:
List metaList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken>() {
}.getType());
return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), metaList);
default:
throw new IllegalStateException("Unexpected groupKey: " + groupKey);
}
}
另一种是监听，就是org.dromara.soul.admin.listener.http.HttpLongPollingDataChangedListener#doLongPolling的调用方。
我们在看下这里分别是哪里在调用
先看下fetch。通过查找controller的http全局搜索，找到是在包soul-sync-data-http中的HttpSyncDataService，这个类是由HttpSyncDataConfiguration来注入到spring容器中的，这里用到了一个spring的特性，ObjectProvider
//org.dromara.soul.spring.boot.starter.sync.data.http.HttpSyncDataConfiguration
@Configuration
@ConditionalOnClass(HttpSyncDataService.class)
@ConditionalOnProperty(prefix = "soul.sync.http", name = "url")
@Slf4j
public class HttpSyncDataConfiguration {
@Bean
public SyncDataService httpSyncDataService(final ObjectProvider httpConfig, final ObjectProvider pluginSubscriber,
final ObjectProvider> metaSubscribers, final ObjectProvider> authSubscribers) {
log.info("you use http long pull sync soul data");
return new HttpSyncDataService(Objects.requireNonNull(httpConfig.getIfAvailable()), Objects.requireNonNull(pluginSubscriber.getIfAvailable()),
metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
}
/**
* Http config http config.
*
* @return the http config
*/
@Bean
@ConfigurationProperties(prefix = "soul.sync.http")
public HttpConfig httpConfig() {
return new HttpConfig();
}
}
HttpSyncDataConfiguration 注入了HttpConfig和SyncDataService，但是如果想构造这两个对象，有一个条件是@ConditionalOnProperty(prefix = "soul.sync.http", name = "url")需要有soul.sync.http配置存在
Soul的很多配置，都设置了条件，需要满足条件才可以注入相关的类，这样也大大减少了spring中无用类的管理
如果网关想要开启http长轮训数据同步方式，只需要配置下soul.sync.http，url配置admin的地址即可.http://localhost:9095
我们再说回HttpSyncDataService.
//org.dromara.soul.sync.data.http.HttpSyncDataService
@Slf4j
public class HttpSyncDataService implements SyncDataService, AutoCloseable {
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final Gson GSON = new Gson();
/**
* default: 10s.
*/
private Duration connectionTimeout = Duration.ofSeconds(10);
/**
* only use for http long polling.
*/
private RestTemplate httpClient;
private ExecutorService executor;
private HttpConfig httpConfig;
//soul-admin的http的server列表
private List serverList;
//数据刷新工厂
private DataRefreshFactory factory;
public HttpSyncDataService(final HttpConfig httpConfig, final PluginDataSubscriber pluginDataSubscriber,
final List metaDataSubscribers, final List authDataSubscribers) {
this.factory = new DataRefreshFactory(pluginDataSubscriber, metaDataSubscribers, authDataSubscribers);
this.httpConfig = httpConfig;
this.serverList = Lists.newArrayList(Splitter.on(",").split(httpConfig.getUrl()));
this.httpClient = createRestTemplate();
this.start();
}
private RestTemplate createRestTemplate() {
OkHttp3ClientHttpRequestFactory factory = new OkHttp3ClientHttpRequestFactory();
factory.setConnectTimeout((int) this.connectionTimeout.toMillis());
factory.setReadTimeout((int) HttpConstants.CLIENT_POLLING_READ_TIMEOUT);
return new RestTemplate(factory);
}
}
当构造器调用后，初始化属性后，调用start
//org.dromara.soul.sync.data.http.HttpSyncDataService#start
private void start() {
// 保证只初始化一次
if (RUNNING.compareAndSet(false, true)) {
// 1.1 fetch所有的配置
this.fetchGroupConfig(ConfigGroupEnum.values());
int threadSize = serverList.size();
this.executor = new ThreadPoolExecutor(threadSize, threadSize, 60L, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(),
SoulThreadFactory.create("http-long-polling", true));
// 1.2 用定时线程池，每60秒执行一次http长轮训任务
this.serverList.forEach(server -> this.executor.execute(new HttpLongPollingTask(server)));
} else {
log.info("soul http long polling was started, executor=[{}]", executor);
}
}
1.1 我们先看下fetchGroupConfig是如何实现的
//org.dromara.soul.sync.data.http.HttpSyncDataService#fetchGroupConfig
//依次从server端fetch数据，只要有一个服务fetch成功，即可跳出循环，如果fetch最终都失败了，抛出异常
private void fetchGroupConfig(final ConfigGroupEnum... groups) throws SoulException {
for (int index = 0; index < this.serverList.size(); index++) {
String server = serverList.get(index);
try {
//1.1.1
this.doFetchGroupConfig(server, groups);
break;
} catch (SoulException e) {
// no available server, throw exception.
if (index >= serverList.size() - 1) {
throw e;
}
log.warn("fetch config fail, try another one: {}", serverList.get(index + 1));
}
}
}
1.1.1 这里终于找到了/configs/fetch的调用的地方
//org.dromara.soul.sync.data.http.HttpSyncDataService#doFetchGroupConfig
private void doFetchGroupConfig(final String server, final ConfigGroupEnum... groups) {
StringBuilder params = new StringBuilder();
for (ConfigGroupEnum groupKey : groups) {
params.append("groupKeys").append("=").append(groupKey.name()).append("&");
}
String url = server + "/configs/fetch?" + StringUtils.removeEnd(params.toString(), "&");
log.info("request configs: [{}]", url);
String json = null;
try {
//通过httpClient调用获取到拿到的json
json = this.httpClient.getForObject(url, String.class);
} catch (RestClientException e) {
String message = String.format("fetch config fail from server[%s], %s", url, e.getMessage());
log.warn(message);
throw new SoulException(message, e);
}
// 1.1.1.1 更新本地缓存
boolean updated = this.updateCacheWithJson(json);
if (updated) {
log.info("get latest configs: [{}]", json);
return;
}
// not updated. it is likely that the current config server has not been updated yet. wait a moment.
log.info("The config of the server[{}] has not been updated or is out of date. Wait for 30s to listen for changes again.", server);
ThreadUtils.sleep(TimeUnit.SECONDS, 30);
}
1.1.1.1 通过DataRefreshFactory factory执行更新逻辑
//org.dromara.soul.sync.data.http.HttpSyncDataService#updateCacheWithJson
private boolean updateCacheWithJson(final String json) {
JsonObject jsonObject = GSON.fromJson(json, JsonObject.class);
JsonObject data = jsonObject.getAsJsonObject("data");
// if the config cache will be updated?
return factory.executor(data);
}
DataRefreshFactory 是HttpSyncDataService之前构造器构造的一个数据更新工厂，里面封装了不同种数据的更新策略，这里用到了工厂方法的设计模式
//org.dromara.soul.sync.data.http.refresh.DataRefreshFactory
public final class DataRefreshFactory {
private static final EnumMap ENUM_MAP = new EnumMap<>(ConfigGroupEnum.class);
/**
* 工厂 内部 封装了一个 ConfigGroupEnum, DataRefresh 对应关系，针对不同的ConfigGroup，并使用不同的数据监听方法
*/
public DataRefreshFactory(final PluginDataSubscriber pluginDataSubscriber,
final List metaDataSubscribers,
final List authDataSubscribers) {
ENUM_MAP.put(ConfigGroupEnum.PLUGIN, new PluginDataRefresh(pluginDataSubscriber));
ENUM_MAP.put(ConfigGroupEnum.SELECTOR, new SelectorDataRefresh(pluginDataSubscriber));
ENUM_MAP.put(ConfigGroupEnum.RULE, new RuleDataRefresh(pluginDataSubscriber));
ENUM_MAP.put(ConfigGroupEnum.APP_AUTH, new AppAuthDataRefresh(authDataSubscribers));
ENUM_MAP.put(ConfigGroupEnum.META_DATA, new MetaDataRefresh(metaDataSubscribers));
}
/**
* 对所有的数据做刷新处理
*/
public boolean executor(final JsonObject data) {
final boolean[] success = {false};
ENUM_MAP.values().parallelStream().forEach(dataRefresh -> success[0] = dataRefresh.refresh(data));
return success[0];
}
}
DataRefresh是一个接口，定义了数据刷新的行为
//org.dromara.soul.sync.data.http.refresh.DataRefresh
public interface DataRefresh {
/**
* 刷新数据并返回是否刷新成功
*/
Boolean refresh(JsonObject data);
/**
* 拿到缓存中的配置信息
*/
ConfigData cacheConfigData();
}
看到DataRefresh有一个抽象实现，以及针对不同的数据的不同实现
image.png
AbstractDataRefresh定义了一些通用模版，并且定义了一个泛型，因为之前我们传入的是一个JsonObject，这里通过泛型，再将泛型转化为对应的类，这里技巧值得学习，比如refresh
// org.dromara.soul.sync.data.http.refresh.AbstractDataRefresh
/**
*/
protected abstract void refresh(List data);
/**
这里将 JsonObject转化为对应的ConfigData抽象起来，让每个子类再去实现
*/
@Override
public Boolean refresh(final JsonObject data) {
boolean updated = false;
//转换JsonObject
JsonObject jsonObject = convert(data);
if (null != jsonObject) {
//将JsonObject转换为ConfigData
ConfigData result = fromJson(jsonObject);
//是否需要更新缓存
if (this.updateCacheIfNeed(result)) {
updated = true;
//刷新缓存
refresh(result.getData());
}
}
return updated;
}
我们挑一个PluginDataRefresh看下具体的逻辑
//org.dromara.soul.sync.data.http.refresh.PluginDataRefresh
//通过调用父类的updateCacheIfNeed来判断是否需要更新
@Override
protected boolean updateCacheIfNeed(final ConfigData result) {
return updateCacheIfNeed(result, ConfigGroupEnum.PLUGIN);
}
//org.dromara.soul.sync.data.http.refresh.AbstractDataRefresh#updateCacheIfNeed(org.dromara.soul.common.dto.ConfigData, org.dromara.soul.common.enums.ConfigGroupEnum)
protected boolean updateCacheIfNeed(final ConfigData newVal, final ConfigGroupEnum groupEnum) {
// 第一次初始化
if (GROUP_CACHE.putIfAbsent(groupEnum, newVal) == null) {
return true;
}
ResultHolder holder = new ResultHolder(false);
GROUP_CACHE.merge(groupEnum, newVal, (oldVal, value) -> {
// 比对md5以及更新时间，如果新的数据即更新到GROUP_CACHE中，否则保留原来的值
if (!StringUtils.equals(oldVal.getMd5(), newVal.getMd5()) && oldVal.getLastModifyTime() < newVal.getLastModifyTime()) {
log.info("update {} config: {}", groupEnum, newVal);
holder.result = true;
return newVal;
}
log.info("Get the same config, the [{}] config cache will not be updated, md5:{}", groupEnum, oldVal.getMd5());
return oldVal;
});
//返回 更新结果，这里的Holder实际上就是为了能够拿到merge里面的更新状态的一个持有类而已
return holder.result;
}
如果我们发现需要更新，并更新了本地缓存数据，则进行refresh
//org.dromara.soul.sync.data.http.refresh.PluginDataRefresh#refresh
@Override
protected void refresh(final List data) {
//如果data是空，代表是要清空缓存，调用refreshPluginDataAll
if (CollectionUtils.isEmpty(data)) {
log.info("clear all plugin data cache");
pluginDataSubscriber.refreshPluginDataAll();
} else {
//这里存疑，为何要在这里还要执行一次refreshAll
pluginDataSubscriber.refreshPluginDataAll();
//对每一个数据通过pluginDataSubscriber做处理
data.forEach(pluginDataSubscriber::onSubscribe);
}
}
我们看下PluginDataSubscriber,它里面有很多的默认方法实现，只有一个实现类CommonPluginDataSubscriber，我们重点看下CommonPluginDataSubscriber的refreshPluginDataAll和onSubscribe
//org.dromara.soul.plugin.base.cache.CommonPluginDataSubscriber
//这里调用了
@Override
public void refreshPluginDataAll() {
BaseDataCache.getInstance().cleanPluginData();
}
@Override
public void onSubscribe(final PluginData pluginData) {
subscribeDataHandler(pluginData, DataEventTypeEnum.UPDATE);
}
/**
这里就是根据不同的事件和data类型做不同的处理
*/
private void subscribeDataHandler(final T classData, final DataEventTypeEnum dataType) {
Optional.ofNullable(classData).ifPresent(data -> {
if (data instanceof PluginData) {
PluginData pluginData = (PluginData) data;
if (dataType == DataEventTypeEnum.UPDATE) {
//缓存数据
BaseDataCache.getInstance().cachePluginData(pluginData);
//对要处理的pluginData对应的数据进行相关处理
Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.handlerPlugin(pluginData));
} else if (dataType == DataEventTypeEnum.DELETE) {
BaseDataCache.getInstance().removePluginData(pluginData);
Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.removePlugin(pluginData));
}
} else if (data instanceof SelectorData) {
SelectorData selectorData = (SelectorData) data;
if (dataType == DataEventTypeEnum.UPDATE) {
BaseDataCache.getInstance().cacheSelectData(selectorData);
Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.handlerSelector(selectorData));
} else if (dataType == DataEventTypeEnum.DELETE) {
BaseDataCache.getInstance().removeSelectData(selectorData);
Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.removeSelector(selectorData));
}
} else if (data instanceof RuleData) {
RuleData ruleData = (RuleData) data;
if (dataType == DataEventTypeEnum.UPDATE) {
BaseDataCache.getInstance().cacheRuleData(ruleData);
Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.handlerRule(ruleData));
} else if (dataType == DataEventTypeEnum.DELETE) {
BaseDataCache.getInstance().removeRuleData(ruleData);
Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.removeRule(ruleData));
}
}
});
}
我们在看下BaseDataCache是什么，它用到了单例模式，里面持有不同的缓存数据，将操作封装了一下
//org.dromara.soul.plugin.base.cache.BaseDataCache
/**
* pluginName -> PluginData.
*/
private static final ConcurrentMap PLUGIN_MAP = Maps.newConcurrentMap();
/**
* pluginName -> SelectorData.
*/
private static final ConcurrentMap> SELECTOR_MAP = Maps.newConcurrentMap();
/**
* selectorId -> RuleData.
*/
private static final ConcurrentMap> RULE_MAP = Maps.newConcurrentMap();
/**
* Clean plugin data.
*/
public void cleanPluginData() {
PLUGIN_MAP.clear();
}
/**
* Cache plugin data.
*
* @param pluginData the plugin data
*/
public void cachePluginData(final PluginData pluginData) {
Optional.ofNullable(pluginData).ifPresent(data -> PLUGIN_MAP.put(data.getName(), data));
}
刚才看到我们数据刷新缓存后还有一个handler处理，handlerMap是CommonPluginDataSubscriber在构造器构造的一个Map
//org.dromara.soul.plugin.base.cache.CommonPluginDataSubscriber
private final Map handlerMap;
/**
* Instantiates a new Common plugin data subscriber.
*
* @param pluginDataHandlerList the plugin data handler list
*/
public CommonPluginDataSubscriber(final List pluginDataHandlerList) {
this.handlerMap = pluginDataHandlerList.stream().collect(Collectors.toConcurrentMap(PluginDataHandler::pluginNamed, e -> e));
}
我们在看下PluginDataHandler做了什么,PluginDataHandler是一个接口，这里每一个插件都有一个对应的实现类，我们看到了熟悉的身影，Divide插件，Sofa插件，Dubbo插件等等
image.png
因为我们现在看的是PluginData的处理逻辑所以我们看下handlerPlugin的其中一个实现，ApacheDubboPluginDataHandler
//org.dromara.soul.plugin.apache.dubbo.handler.ApacheDubboPluginDataHandler
public class ApacheDubboPluginDataHandler implements PluginDataHandler {
@Override
public void handlerPlugin(final PluginData pluginData) {
//判断是否开启
if (null != pluginData && pluginData.getEnabled()) {
//拿到dubbo的注册信息
DubboRegisterConfig dubboRegisterConfig = GsonUtils.getInstance().fromJson(pluginData.getConfig(), DubboRegisterConfig.class);
//Singleton.INST是一个单例模式，可以自己看下，很简单
DubboRegisterConfig exist = Singleton.INST.get(DubboRegisterConfig.class);
//如果注册信息不存在直接返回
if (Objects.isNull(dubboRegisterConfig)) {
return;
}
//当前exit没有，或者不想等
if (Objects.isNull(exist) || !dubboRegisterConfig.equals(exist)) {
// 初始化新的配置，废弃当前缓存
ApplicationConfigCache.getInstance().init(dubboRegisterConfig);
ApplicationConfigCache.getInstance().invalidateAll();
}
Singleton.INST.single(DubboRegisterConfig.class, dubboRegisterConfig);
}
}
@Override
public String pluginNamed() {
return PluginEnum.DUBBO.getName();
}
}
这里主要讲一下数据更新部分，对于插件部分的详细内容之后单拉一个插件系列再讲
到这里我们捋清楚了数据fetch的流程。