Sentinel学习(八) —— 规则持久化
规则持久化分成两种方式：拉模式和推模式。
拉模式
原理简述
FileRefreshableDataSource定时从指定文件中读取规则JSON文件【图中的本地文件】，如果发现文件发生变化，就更新规则缓存。FileWritableDataSource接收控制台规则推送，并根据配置，修改规则JSON文件【图中的本地文件】。
功能实现
-
加依赖
<dependency> <groupId>com.alibaba.csp</groupId> <artifactId>sentinel-datasource-extension</artifactId> </dependency> -
增加拉模式规则持久化类
/** * 拉模式规则持久化 * * @author itmuch.com */ public class FileDataSourceInit implements InitFunc { @Override public void init() throws Exception { // TIPS: 如果你对这个路径不喜欢，可修改为你喜欢的路径 String ruleDir = System.getProperty("user.home") + "/sentinel/rules"; String flowRulePath = ruleDir + "/flow-rule.json"; String degradeRulePath = ruleDir + "/degrade-rule.json"; String systemRulePath = ruleDir + "/system-rule.json"; String authorityRulePath = ruleDir + "/authority-rule.json"; String paramFlowRulePath = ruleDir + "/param-flow-rule.json"; this.mkdirIfNotExits(ruleDir); this.createFileIfNotExits(flowRulePath); this.createFileIfNotExits(degradeRulePath); this.createFileIfNotExits(systemRulePath); this.createFileIfNotExits(authorityRulePath); this.createFileIfNotExits(paramFlowRulePath); // 流控规则 ReadableDataSource<String, List<FlowRule>> flowRuleRDS = new FileRefreshableDataSource<>( flowRulePath, flowRuleListParser ); // 将可读数据源注册至FlowRuleManager // 这样当规则文件发生变化时，就会更新规则到内存 FlowRuleManager.register2Property(flowRuleRDS.getProperty()); WritableDataSource<List<FlowRule>> flowRuleWDS = new FileWritableDataSource<>( flowRulePath, this::encodeJson ); // 将可写数据源注册至transport模块的WritableDataSourceRegistry中 // 这样收到控制台推送的规则时，Sentinel会先更新到内存，然后将规则写入到文件中 WritableDataSourceRegistry.registerFlowDataSource(flowRuleWDS); // 降级规则 ReadableDataSource<String, List<DegradeRule>> degradeRuleRDS = new FileRefreshableDataSource<>( degradeRulePath, degradeRuleListParser ); DegradeRuleManager.register2Property(degradeRuleRDS.getProperty()); WritableDataSource<List<DegradeRule>> degradeRuleWDS = new FileWritableDataSource<>( degradeRulePath, this::encodeJson ); WritableDataSourceRegistry.registerDegradeDataSource(degradeRuleWDS); // 系统规则 ReadableDataSource<String, List<SystemRule>> systemRuleRDS = new FileRefreshableDataSource<>( systemRulePath, systemRuleListParser ); SystemRuleManager.register2Property(systemRuleRDS.getProperty()); WritableDataSource<List<SystemRule>> systemRuleWDS = new FileWritableDataSource<>( systemRulePath, this::encodeJson ); WritableDataSourceRegistry.registerSystemDataSource(systemRuleWDS); // 授权规则 ReadableDataSource<String, List<AuthorityRule>> authorityRuleRDS = new FileRefreshableDataSource<>( authorityRulePath, authorityRuleListParser ); AuthorityRuleManager.register2Property(authorityRuleRDS.getProperty()); WritableDataSource<List<AuthorityRule>> authorityRuleWDS = new FileWritableDataSource<>( authorityRulePath, this::encodeJson ); WritableDataSourceRegistry.registerAuthorityDataSource(authorityRuleWDS); // 热点参数规则 ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleRDS = new FileRefreshableDataSource<>( paramFlowRulePath, paramFlowRuleListParser ); ParamFlowRuleManager.register2Property(paramFlowRuleRDS.getProperty()); WritableDataSource<List<ParamFlowRule>> paramFlowRuleWDS = new FileWritableDataSource<>( paramFlowRulePath, this::encodeJson ); ModifyParamFlowRulesCommandHandler.setWritableDataSource(paramFlowRuleWDS); } private Converter<String, List<FlowRule>> flowRuleListParser = source -> JSON.parseObject( source, new TypeReference<List<FlowRule>>() { } ); private Converter<String, List<DegradeRule>> degradeRuleListParser = source -> JSON.parseObject( source, new TypeReference<List<DegradeRule>>() { } ); private Converter<String, List<SystemRule>> systemRuleListParser = source -> JSON.parseObject( source, new TypeReference<List<SystemRule>>() { } ); private Converter<String, List<AuthorityRule>> authorityRuleListParser = source -> JSON.parseObject( source, new TypeReference<List<AuthorityRule>>() { } ); private Converter<String, List<ParamFlowRule>> paramFlowRuleListParser = source -> JSON.parseObject( source, new TypeReference<List<ParamFlowRule>>() { } ); private void mkdirIfNotExits(String filePath) throws IOException { File file = new File(filePath); if (!file.exists()) { file.mkdirs(); } } private void createFileIfNotExits(String filePath) throws IOException { File file = new File(filePath); if (!file.exists()) { file.createNewFile(); } } private <T> String encodeJson(T t) { return JSON.toJSONString(t); } } -
配置
在项目的 resources/META-INF/services 目录下创建文件，名为 com.alibaba.csp.sentinel.init.InitFunc ，内容为：
# 改成上面FileDataSourceInit的包名类名全路径即可。 com.itmuch.contentcenter.FileDataSourceInit -
优缺点分析
- 优点
- 简单易懂
- 没有多余依赖（比如配置中心、缓存等）
- 缺点
- 由于规则是用
FileRefreshableDataSource定时更新的，所以规则更新会有延迟。如果FileRefreshableDataSource定时时间过大，可能长时间延迟；如果FileRefreshableDataSource过小，又会影响性能； - 规则存储在本地文件，如果有一天需要迁移微服务，那么需要把规则文件一起迁移，否则规则会丢失。
- 由于规则是用
- 优点
推模式
原理简述
- 控制台推送规则：
- 将规则推送到Nacos或其他远程配置中心
- Sentinel客户端链接Nacos，获取规则配置；并监听Nacos配置变化，如发生变化，就更新本地缓存（从而让本地缓存总是和Nacos一致）
- 控制台监听Nacos配置变化，如发生变化就更新本地缓存（从而让控制台本地缓存总是和Nacos一致）
整合Apollo
控制面板改造
首先我们来改造sentinel的控制面板。在源码中官方已经给出来了单元测试。我们来进行改造。
我们在java包下面的com.alibaba.csp.sentinel.dashboard.rule创建一个apollo包，开始创建我们自己的类。
ApolloConfig
由于我们要将限流配置保存到Apollo中，所以我们需要配置地址和调用OpenAPI的token。
@Configuration
@EnableApolloConfig
public class ApolloConfig {
/**
* apollo地址
*/
@Value("${apollo.sentinel.portal.url}")
private String apolloPortalUrl;
/**
* apollo token
*/
@Value("${apollo.sentinel.token}")
private String apolloApplicationToken;
/**
* @Author xyhua
* @Description apollo openApi
* @Date 09:39 2020-06-02
* @param
* @return
**/
@Bean
public ApolloOpenApiClient apolloOpenApiClient() {
ApolloOpenApiClient client = ApolloOpenApiClient.newBuilder()
.withPortalUrl(apolloPortalUrl)
.withToken(apolloApplicationToken)
.build();
return client;
}
}
ApolloCommonService
主要用来对Apollo里的值和Sentinel中的实体类相互转换。在保存规则和读取规则都会调这里面的方法。
@Service
public class ApolloCommonService {
/**
* 没有找到配置项，apollo 返回的错误码
*/
private static final int NO_FOUND_ERROR_CODE = 404;
@Autowired
private ApolloOpenApiClient apolloOpenApiClient;
@Value("${apollo.sentinel.env}")
private String env;
@Value("${apollo.sentinel.appid}")
private String appId;
@Value("${apollo.sentinel.cluster.name}")
private String clusterName;
@Value("${apollo.sentinel.namespace.name}")
private String namespaceName;
@Value("${apollo.sentinel.modify.user}")
private String modifyUser;
@Value("${apollo.sentinel.modify.comment}")
private String modifyComment;
@Value("${apollo.sentinel.release.comment}")
private String releaseComment;
@Value("${apollo.sentinel.release.user}")
private String releaseUser;
/**
* @Author xyhua
* @Description 从apollo中获取规则
* @Date 19:52 2020-06-02
* @param
* @return
**/
public <T> List<T> getRules(String appName, String flowDataIdSuffix, Class<T> ruleClass) {
// flowDataId
String flowDataId = appName + flowDataIdSuffix;
OpenNamespaceDTO openNamespaceDTO = apolloOpenApiClient.getNamespace(appId, env, clusterName, namespaceName);
String rules = openNamespaceDTO
.getItems()
.stream()
.filter(p -> p.getKey().equals(flowDataId))
.map(OpenItemDTO::getValue)
.findFirst()
.orElse("");
if (StringUtil.isEmpty(rules)) {
return new ArrayList<>();
}
List<T> flow = JSON.parseArray(rules, ruleClass);
if (Objects.isNull(flow)) {
return new ArrayList<>();
}
return flow;
}
/**
* @Author xyhua
* @Description 设置规则类型
* @Date 01:31 2020-06-03
* @param
* @return
**/
public void publishRules(String appName, String flowDataIdSuffix, String rules) {
// flowDataId
String flowDataId = appName + flowDataIdSuffix;
AssertUtil.notEmpty(appName, "app name cannot be empty");
if (rules == null) {
return;
}
OpenItemDTO openItemDTO = new OpenItemDTO();
openItemDTO.setKey(flowDataId);
openItemDTO.setValue(rules);
openItemDTO.setComment(modifyComment);
openItemDTO.setDataChangeCreatedBy(modifyUser);
try {
apolloOpenApiClient.createOrUpdateItem(appId, env, clusterName, namespaceName, openItemDTO);
} catch (Exception e) {
if (e.getCause() instanceof ApolloOpenApiException) {
ApolloOpenApiException apolloOpenApiException = (ApolloOpenApiException) e.getCause();
if (Integer.valueOf(NO_FOUND_ERROR_CODE).equals(apolloOpenApiException.getStatus())) {
apolloOpenApiClient.createItem(appId, env, clusterName, namespaceName, openItemDTO);
System.out.println("初始化应用配置 -> {}" + flowDataId);
}
} else {
e.printStackTrace();
}
}
// Release configuration
NamespaceReleaseDTO namespaceReleaseDTO = new NamespaceReleaseDTO();
namespaceReleaseDTO.setEmergencyPublish(true);
namespaceReleaseDTO.setReleaseComment(releaseComment);
namespaceReleaseDTO.setReleasedBy(releaseUser);
namespaceReleaseDTO.setReleaseTitle(releaseComment);
apolloOpenApiClient.publishNamespace(appId, env, clusterName, namespaceName, namespaceReleaseDTO);
}
/**
* @Author xyhua
* @Description 删除规则
* @Date 01:33 2020-06-03
* @param
* @return
**/
public void deleteRules(String rulekey, String operator) {
try {
apolloOpenApiClient.removeItem(appId, env, clusterName, namespaceName, rulekey, operator);
} catch (Exception e) {
if (e.getCause() instanceof ApolloOpenApiException) {
ApolloOpenApiException apolloOpenApiException = (ApolloOpenApiException) e.getCause();
if (Integer.valueOf(NO_FOUND_ERROR_CODE).equals(apolloOpenApiException.getStatus())) {
apolloOpenApiClient.removeItem(appId, env, clusterName, namespaceName, rulekey, operator);
}
} else {
e.printStackTrace();
}
}
// Release configuration
NamespaceReleaseDTO namespaceReleaseDTO = new NamespaceReleaseDTO();
namespaceReleaseDTO.setEmergencyPublish(true);
namespaceReleaseDTO.setReleaseComment(releaseComment);
namespaceReleaseDTO.setReleasedBy(releaseUser);
namespaceReleaseDTO.setReleaseTitle(releaseComment);
apolloOpenApiClient.publishNamespace(appId, env, clusterName, namespaceName, namespaceReleaseDTO);
}
}
AbstractApolloCommonService
基础抽象类，主要定义类Apollo中配置的不同规则前缀和上面提到的Apollo规则的处理类。
@Service
public abstract class AbstractApolloCommonService {
@Autowired
protected ApolloCommonService apolloCommonService;
/**
* 流控规则前缀标示
*/
@Value("${apollo.sentinel.flow.key.suffix:-flow}")
String flowDataIdSuffix;
/**
* 熔断降级规则前缀标示
*/
@Value("${apollo.sentinel.degrade.key.suffix:-degrade}")
String degradeDataIdSuffix;
/**
* 热点规则前缀标示
*/
@Value("${apollo.sentinel.paramFlow.key.suffix:-paramFlow}")
String paramFlowDataIdSuffix;
/**
* 系统规则前缀标示
*/
@Value("${apollo.sentinel.system.key.suffix:-system}")
String systemDataIdSuffix;
/**
* 授权规则前缀标示
*/
@Value("${apollo.sentinel.authority.key.suffix:-authority}")
String authorityDataIdSuffix;
}
DynamicRuleProvider和DynamicRulePublisher
public interface DynamicRuleProvider<T> {
T getRules(String appName) throws Exception;
}
public interface DynamicRulePublisher<T> {
void publish(String app, T rules) throws Exception;
}
这个是Sentinel提供的对规则的处理接口。分别包含getRules和publish方法用来对规则进行获取和保存，针对不同的限流规则我们实现不同的实体类就可以了。我们用限流规则来举例。
FlowRuleApolloProvider和FlowRuleApolloPublisher
@Component("flowRuleApolloProvider")
public class FlowRuleApolloProvider extends AbstractApolloCommonService implements
DynamicRuleProvider<List<FlowRuleEntity>> {
@Override
public List<FlowRuleEntity> getRules(String appName) {
return apolloCommonService.getRules(appName, flowDataIdSuffix, FlowRuleEntity.class);
}
}
@Component("flowRuleApolloPublisher")
public class FlowRuleApolloPublisher extends AbstractApolloCommonService implements
DynamicRulePublisher<List<FlowRuleEntity>> {
@Override
public void publish(String app, List<FlowRuleEntity> rules) {
apolloCommonService.publishRules(app, flowDataIdSuffix, JSON.toJSONString(rules));
}
}
FlowControllerV2
这是控制台的限流处理类，需要引用我们刚刚创建的类。
@RestController
@RequestMapping(value = "/v2/flow")
public class FlowControllerV2 {
...
@Autowired
@Qualifier("flowRuleApolloProvider")
private DynamicRuleProvider<List<FlowRuleEntity>> ruleProvider;
@Autowired
@Qualifier("flowRuleApolloPublisher")
private DynamicRulePublisher<List<FlowRuleEntity>> rulePublisher;
...
}
sidebar.html
修改页面，让他调用V2的controller
<li ui-sref-active="active" ng-if="!entry.isGateway">
<a ui-sref="dashboard.flow({app: entry.app})">
<i class="glyphicon glyphicon-filter"></i> 流控规则</a>
</li>
重启下sentinel控制台，设置一下流控规则，我们就看到规则已经保存到Apollo中了。
客户端改造
引入依赖
<dependency>
<groupId>com.alibaba.csp</groupId>
<artifactId>sentinel-datasource-apollo</artifactId>
</dependency>
增加Apollo配置
apollo:
meta: http://39.106.161.250:8080
bootstrap:
enabled: true
namespaces: FISH.Sentinel-Common
增加限流规则配置，告诉客户端如何读取Apollo中的配置规则。
spring:
cloud:
sentinel:
filter:
enabled: true
transport:
dashboard: localhost:8080
datasource:
ds:
apollo:
namespaceName: FISH.Sentinel-Common
flowRulesKey: order-flow
ruleType: flow
这样客户端就能动态读取Apollo配置的规则了。