Spring Boot 2.4.2 + Spring Cloud 2020.0.1 + Nacos 1.4.1 + spring-cloud-loadbalancer + gateway 实现灰度发布
Spring Boot 2.4.2 + Spring Cloud 2020.0.1 + Nacos 1.4.1 + spring-cloud-loadbalancer + gateway 实现灰度发布
- 引言
- Nacos Server 1.4.1
- Producer
-
- pom.xml
- Controller
- application.properties
- Consumer
-
- pom.xml
- Controller
- Gateway
-
- pom.xml
- application.yml
- spring-cloud-loadbalancer 实现灰度发布
-
- 原理
- 实现灰度发布
-
- 根据官网推荐的注解实现
- 所有服务使用灰度发布
- 写在最后
引言
Spring Cloud 慢慢已经移除了Netflix套件， 所以这次主要是整个项目调用链替换spring-cloud-loadbalancer，然后我们说说spring-cloud-loadbalancer + gateway如何实现灰度发布。
Nacos Server 1.4.1
在github中下载nacos-server-1.4.1，默认mode被改为了cluster模式，并且存储方式会强制你使用mysql，直接启动会一直Nacos is starting...，然后load jdbc.properties error再然后ERROR Nacos failed to start, please see \nacos-server-1.4.1\nacos\logs\nacos.log for more details. ，所以单机测试需要改一下。
startup.cmd修改set MODE="cluster"为set MODE="standalone"startup.sh修改export MODE="cluster"为export MODE="standalone"
Producer
pom.xml
其中spring-cloud-starter-alibaba-nacos-discovery目前的版本还是依赖于ribbon，所以我们需要exclusion ribbon依赖。
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<parent>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-parentartifactId>
<version>2.4.2version>
<relativePath/>
parent>
<groupId>com.doub1groupId>
<artifactId>springcloud-testartifactId>
<version>0.0.1-SNAPSHOTversion>
<name>springcloud-testname>
<description>Demo project for Spring Bootdescription>
<properties>
<java.version>1.8java.version>
<spring-cloud.version>2020.0.1spring-cloud.version>
properties>
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webartifactId>
dependency>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-testartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starterartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-loadbalancerartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-openfeignartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-configartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discoveryartifactId>
<exclusions>
<exclusion>
<artifactId>spring-cloud-starter-netflix-ribbonartifactId>
<groupId>org.springframework.cloudgroupId>
exclusion>
exclusions>
dependency>
dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-dependenciesartifactId>
<version>${spring-cloud.version}version>
<type>pomtype>
<scope>importscope>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-alibaba-dependenciesartifactId>
<version>2.2.1.RELEASEversion>
<type>pomtype>
<scope>importscope>
dependency>
dependencies>
dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-maven-pluginartifactId>
plugin>
plugins>
build>
project>
Controller
package com.doub1.producer.controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.UUID;
@RestController
public class HomeController {
private static final String randomStr = UUID.randomUUID().toString();
@GetMapping("/index")
public String index() {
return randomStr + "我是V1";
}
}
application.properties
我创建了2个相同的producer,只是注册进nacos的元数据的version不同。
server.port=18081
spring.application.name=Producer
spring.cloud.loadbalancer.ribbon.enabled=false
spring.cloud.nacos.discovery.metadata.version=1
Consumer
pom.xml
额外增加了一个spring-cloud-starter-openfeign
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<parent>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-parentartifactId>
<version>2.4.2version>
<relativePath/>
parent>
<groupId>com.doub1groupId>
<artifactId>springcloud-testartifactId>
<version>0.0.1-SNAPSHOTversion>
<name>springcloud-testname>
<description>Demo project for Spring Bootdescription>
<properties>
<java.version>1.8java.version>
<spring-cloud.version>2020.0.1spring-cloud.version>
properties>
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webartifactId>
dependency>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-testartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starterartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-loadbalancerartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-openfeignartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-configartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discoveryartifactId>
<exclusions>
<exclusion>
<artifactId>spring-cloud-starter-netflix-ribbonartifactId>
<groupId>org.springframework.cloudgroupId>
exclusion>
exclusions>
dependency>
dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-dependenciesartifactId>
<version>${spring-cloud.version}version>
<type>pomtype>
<scope>importscope>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-alibaba-dependenciesartifactId>
<version>2.2.1.RELEASEversion>
<type>pomtype>
<scope>importscope>
dependency>
dependencies>
dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-maven-pluginartifactId>
plugin>
plugins>
build>
project>
Controller
仅仅是通过Feign调用Producer。
package com.doub1.consumer.controller;
import com.doub1.consumer.services.IProducer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class HomeController {
@Autowired
IProducer producer;
@GetMapping("/index")
public String index(){
return producer.index();
}
}
Gateway
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<parent>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-parentartifactId>
<version>2.4.2version>
<relativePath/>
parent>
<groupId>com.doub1groupId>
<artifactId>springcloud-testartifactId>
<version>0.0.1-SNAPSHOTversion>
<name>springcloud-testname>
<description>Demo project for Spring Bootdescription>
<properties>
<java.version>1.8java.version>
<spring-cloud.version>2020.0.1spring-cloud.version>
properties>
<dependencies>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-webfluxartifactId>
dependency>
<dependency>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-starter-testartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starterartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-loadbalancerartifactId>
dependency>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-starter-gatewayartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-configartifactId>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discoveryartifactId>
<exclusions>
<exclusion>
<artifactId>spring-cloud-starter-netflix-ribbonartifactId>
<groupId>org.springframework.cloudgroupId>
exclusion>
exclusions>
dependency>
dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloudgroupId>
<artifactId>spring-cloud-dependenciesartifactId>
<version>${spring-cloud.version}version>
<type>pomtype>
<scope>importscope>
dependency>
<dependency>
<groupId>com.alibaba.cloudgroupId>
<artifactId>spring-cloud-alibaba-dependenciesartifactId>
<version>2.2.1.RELEASEversion>
<type>pomtype>
<scope>importscope>
dependency>
dependencies>
dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.bootgroupId>
<artifactId>spring-boot-maven-pluginartifactId>
plugin>
plugins>
build>
project>
application.yml
一个简单的规则路由
spring:
cloud:
gateway:
routes:
- id: consumer
uri: lb://Consumer
filters:
- StripPrefix=1
predicates:
- Path=/Consumer/**
- id: producer
uri: lb://Producer
filters:
- StripPrefix=1
predicates:
- Path=/Producer/**
global-filter:
reactive-load-balancer-client:
enabled: true
load-balancer-client:
enabled: true
nacos:
discovery:
register-enabled: false
loadbalancer:
ribbon:
enabled: false
spring-cloud-loadbalancer 实现灰度发布
之前有写过Ribbon的灰度发布，我们主要靠自定义Rule通过Predicate就能实现，spring-cloud-loadbalancer如何实现呢。
原理
gateway中有一个ReactiveLoadBalancerClientFilter，其中获取choose的时候实际上使用的是RoundRobinLoadBalancer。
还有org.springframework.cloud.loadbalancer.config.BlockingLoadBalancerClientAutoConfiguration 中我们可以看见创建一个BlockingLoadBalancerClient的bean.
BlockingLoadBalancerClient的关键代码
@Override
public <T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException {
String hint = getHint(serviceId);
LoadBalancerRequestAdapter<T, DefaultRequestContext> lbRequest = new LoadBalancerRequestAdapter<>(request,
new DefaultRequestContext(request, hint));
Set<LoadBalancerLifecycle> supportedLifecycleProcessors = LoadBalancerLifecycleValidator
.getSupportedLifecycleProcessors(
loadBalancerClientFactory.getInstances(serviceId, LoadBalancerLifecycle.class),
DefaultRequestContext.class, Object.class, ServiceInstance.class);
supportedLifecycleProcessors.forEach(lifecycle -> lifecycle.onStart(lbRequest));
ServiceInstance serviceInstance = choose(serviceId, lbRequest);
if (serviceInstance == null) {
supportedLifecycleProcessors.forEach(lifecycle -> lifecycle.onComplete(
new CompletionContext<>(CompletionContext.Status.DISCARD, lbRequest, new EmptyResponse())));
throw new IllegalStateException("No instances available for " + serviceId);
}
return execute(serviceId, serviceInstance, lbRequest);
}
其中ServiceInstance serviceInstance = choose(serviceId, lbRequest);为
@Override
public <T> ServiceInstance choose(String serviceId, Request<T> request) {
ReactiveLoadBalancer<ServiceInstance> loadBalancer = loadBalancerClientFactory.getInstance(serviceId);
if (loadBalancer == null) {
return null;
}
Response<ServiceInstance> loadBalancerResponse = Mono.from(loadBalancer.choose(request)).block();
if (loadBalancerResponse == null) {
return null;
}
return loadBalancerResponse.getServer();
}
loadBalancer.choose实际上就是调用ReactiveLoadBalancer的choose方法
ReactiveLoadBalancer的实现类有2个RandomLoadBalancer和RoundRobinLoadBalancer
public Mono<Response<ServiceInstance>> choose(Request request) {
ServiceInstanceListSupplier supplier = serviceInstanceListSupplierProvider
.getIfAvailable(NoopServiceInstanceListSupplier::new);
//返回flux的supplier通过next迭代，最后映射为Response实现灰度发布
最早的思路是拷贝RoundRobinLoadBalancer，修改choose，getInstanceResponse中的position保留，因为如果多个同版本服务，我们还可以让他继续进行轮询规则。
在choose中我们使用自己的ServiceInstanceListSupplier，但是这种方法改动较大，网上大多数是直接添加了一个在gateway的filter，将之前的根据服务名懒加载创建的单例bean改为了不停的new一个新的LoadBalancer。
我们只是希望能用我们自己的ServiceInstanceListSupplier而已，没有必要改动如此之大，且网上这么做的性能会变差。
根据官网推荐的注解实现
官网推荐使用
@LoadBalancerClients({@LoadBalancerClient(value = "Producer", configuration = CustomLoadBalancerClientConfiguration.class)})，为指定的服务使用不同的配置，这和Ribbon中使用注解对应不同的Rule同理。
@LoadBalancerClient的value为服务名称，configuration为对应的配置文件。
GrayServiceInstanceListSupplier实现
package test.test.demo.loadBalancer;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.boot.convert.DurationStyle;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.discovery.ReactiveDiscoveryClient;
import org.springframework.cloud.client.loadbalancer.Request;
import org.springframework.cloud.client.loadbalancer.RequestDataContext;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.core.env.Environment;
import reactor.core.publisher.Flux;
import reactor.core.scheduler.Schedulers;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import static org.springframework.cloud.loadbalancer.support.LoadBalancerClientFactory.PROPERTY_NAME;
public class GrayServiceInstanceListSupplier implements ServiceInstanceListSupplier {
/**
* Property that establishes the timeout for calls to service discovery.
*/
public static final String SERVICE_DISCOVERY_TIMEOUT = "spring.cloud.loadbalancer.service-discovery.timeout";
private static final Log LOG = LogFactory.getLog(GrayServiceInstanceListSupplier.class);
private Duration timeout = Duration.ofSeconds(30);
private final String serviceId;
private final Flux<List<ServiceInstance>> serviceInstances;
public GrayServiceInstanceListSupplier(DiscoveryClient delegate, Environment environment) {
this.serviceId = environment.getProperty(PROPERTY_NAME);
resolveTimeout(environment);
this.serviceInstances = Flux.defer(() -> Flux.just(delegate.getInstances(serviceId)))
.subscribeOn(Schedulers.boundedElastic()).timeout(timeout, Flux.defer(() -> {
logTimeout();
return Flux.just(new ArrayList<>());
})).onErrorResume(error -> {
logException(error);
return Flux.just(new ArrayList<>());
});
}
public GrayServiceInstanceListSupplier(ReactiveDiscoveryClient delegate, Environment environment) {
this.serviceId = environment.getProperty(PROPERTY_NAME);
resolveTimeout(environment);
this.serviceInstances = Flux
.defer(() -> delegate.getInstances(serviceId).collectList().flux().timeout(timeout, Flux.defer(() -> {
logTimeout();
return Flux.just(new ArrayList<>());
})).onErrorResume(error -> {
logException(error);
return Flux.just(new ArrayList<>());
}));
}
@Override
public String getServiceId() {
return serviceId;
}
//主要重写方法
@Override
public Flux<List<ServiceInstance>> get(Request request) {
final String requestVersion = ((RequestDataContext) request.getContext()).getClientRequest().getHeaders().getFirst("version");
final AtomicInteger instanceSize = new AtomicInteger();
if (requestVersion != null){
Flux<List<ServiceInstance>> tmpInstance = serviceInstances.map(list -> list.stream().filter(
x -> {
final String version = x.getMetadata().get("version");
final boolean bOk = requestVersion.equals(version);
if (bOk) instanceSize.addAndGet(1);
return bOk;
}
).collect(Collectors.toList()));
//容错，如header无匹配的则轮询
if (instanceSize.get()>0) return tmpInstance;
}
return serviceInstances;
}
@Override
public Flux<List<ServiceInstance>> get() {
return serviceInstances;
}
private void resolveTimeout(Environment environment) {
String providedTimeout = environment.getProperty(SERVICE_DISCOVERY_TIMEOUT);
if (providedTimeout != null) {
timeout = DurationStyle.detectAndParse(providedTimeout);
}
}
private void logTimeout() {
if (LOG.isDebugEnabled()) {
LOG.debug(String.format("Timeout occurred while retrieving instances for service %s."
+ "The instances could not be retrieved during %s", serviceId, timeout));
}
}
private void logException(Throwable error) {
if (LOG.isDebugEnabled()) {
LOG.debug(String.format("Exception occurred while retrieving instances for service %s", serviceId), error);
}
}
}
CustomLoadBalancerClientConfiguration配置文件
需要注意的是不要使用@Configuration，否则会成为全局生效，无法针对指定服务。
package test.test.demo.config;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import test.test.demo.loadBalancer.GrayServiceInstanceListSupplier;
public class CustomLoadBalancerClientConfiguration {
@Autowired
private ApplicationContext applicationContext;
@Bean
@Primary
ServiceInstanceListSupplier serviceInstanceListSupplier() {
DiscoveryClient discoveryClient = applicationContext.getBean(DiscoveryClient.class);
return new GrayServiceInstanceListSupplier(discoveryClient, applicationContext.getEnvironment());
}
}
所有服务使用灰度发布
只需要去除@LoadBalancerClients({@LoadBalancerClient(value = "Producer", configuration = CustomLoadBalancerClientConfiguration.class)})
然后将CustomLoadBalancerClientConfiguration添加一个@Configuration注解，使其全局生效。
写在最后
全链路灰度发布需要在每个服务中增加http头部转发和相同逻辑的代码，这里不写了，大家自己动动手，挺简单的。
不过还是建议，如果用Istio，灰度发布之类的还是尽量在基础架构吧，这些东西集成在项目代码中确实有些臃肿。