Java流控的各种实现方案
Java流控的各种实现方案
1.使用Guava的RateLimiter
原理：RateLimiter使用的是一种叫令牌桶的流控算法，RateLimiter会按照一定的频率往桶里扔令牌，线程拿到令牌才能执行，比如你希望自己的应用程序QPS不要超过1000，那么RateLimiter设置1000的速率后，就会每秒往桶里扔1000个令牌。
代码：
LxRateLimit.java
import java.lang.annotation.*;
import java.util.concurrent.TimeUnit;
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface LxRateLimit {
/**
*
* @return
*/
String value() default "";
/**
* 每秒向桶中放入令牌的数量 默认最大即不做限流
* @return
*/
double perSecond() default Double.MAX_VALUE;
/**
* 获取令牌的等待时间 默认0
* @return
*/
int timeOut() default 0;
/**
* 超时时间单位
* @return
*/
TimeUnit timeOutUnit() default TimeUnit.MILLISECONDS;
}
LxRateLimitAspect.java
import com.google.common.util.concurrent.RateLimiter;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.Signature;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.ResponseBody;
import java.lang.reflect.Method;
@Aspect
@Component
public class LxRateLimitAspect {
private final static Logger logger = LoggerFactory.getLogger(LxRateLimitAspect.class);
private RateLimiter rateLimiter = RateLimiter.create(Double.MAX_VALUE);
/**
* 定义切点
* 1、通过扫包切入
* 2、带有指定注解切入
*/
@Pointcut("@annotation(com.currentlimiting.demo.guava.LxRateLimit)")
public void checkPointcut() { }
@ResponseBody
@Around(value = "checkPointcut()")
public Object aroundNotice(ProceedingJoinPoint pjp) throws Throwable {
Signature signature = pjp.getSignature();
MethodSignature methodSignature = (MethodSignature)signature;
//获取目标方法
Method targetMethod = methodSignature.getMethod();
if (targetMethod.isAnnotationPresent(LxRateLimit.class)) {
//获取目标方法的@LxRateLimit注解
LxRateLimit lxRateLimit = targetMethod.getAnnotation(LxRateLimit.class);
rateLimiter.setRate(lxRateLimit.perSecond());
if (!rateLimiter.tryAcquire(lxRateLimit.timeOut(), lxRateLimit.timeOutUnit())){
logger.error("fail");
return "fail";
}
}
return pjp.proceed();
}
}
GuavaController.java
import com.currentlimiting.demo.guava.LxRateLimit;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class GuavaController {
private final static Logger logger = LoggerFactory.getLogger(GuavaController.class);
@RequestMapping("/guava")
@LxRateLimit(perSecond = 10)
public String guava() throws InterruptedException {
logger.info("guava");
Thread.sleep(20);
return "guava";
}
}
ab测试结果：
条件：QPS=10
[D:\Apache24\bin]$ ab -n 20 -c 20 "http://localhost:8080/guava"
多次模拟并发请求20次，平均有9次失败，11次成功，设置的QPS为10，guava的RateLimiter总会比设置的多1，是因为guava的懒加载的缘故。
工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/guava
2.使用Hystrix限流
介绍：Hystrix是SOA/微服务架构中提供服务隔离、熔断、降级机制的工具/框架。Hystrix是断路器的一种实现，用于高微服务架构的可用性，是防止服务出现雪崩的利器。
原理：Hystrix将请求的逻辑进行封装，一种是把相关逻辑封装在独立的线程中执行实现隔离，另一种是利用信号量来控制并发数实现隔离，如图：
线程池实现：
代码：
HystrixController.java
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.concurrent.TimeUnit;
@RestController
public class HystrixController {
private final static Logger logger = LoggerFactory.getLogger(HystrixController.class);
@RequestMapping("/hystrix1")
@HystrixCommand(
fallbackMethod = "error",
threadPoolProperties = {
@HystrixProperty(name = "coreSize", value = "10"),//10个核心线程
@HystrixProperty(name = "maxQueueSize", value = "100"),//最大线程数100
@HystrixProperty(name = "queueSizeRejectionThreshold", value = "10")},//队列阈值10
commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "10000"), //命令执行超时时间
@HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "1000"), //若干10s一个窗口内失败1000次, 则达到触发熔断的最少请求量
@HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "1") //断路1s后尝试执行, 默认为5s
})
public String hystrix() throws InterruptedException {
Thread.sleep(500);
logger.info("hystrix1");
return "hystrix1";
}
public String error() {
logger.info("fail");
return "fail";
}
}
ab测试结果：
条件：线程池核心线程=10，线程池队列=100，队列拒绝策略阈值=10
[D:\Apache24\bin]$ ab -n 42 -c 42 "http://localhost:8080/hystrix"
在多次模拟42个并发请求时，平均失败了20个，理论上核心线程=10（因为最大线程数默认是10），队列拒绝策略阈值=10（小于线程池队列，可以忽略队列值），容量就是20，应该成功22失败20。由于并发测试并不能百分百保证结果完全等于理论值，但是会在理论值上下浮动。hystrix是限制并发数的，所以会比guava的测试结果浮动大一些。
信号量实现：
代码：
HystrixController.java
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.concurrent.TimeUnit;
@RestController
public class HystrixController {
@RequestMapping("/hystrix2")
@HystrixCommand(
fallbackMethod = "error",
commandProperties = {
@HystrixProperty(name = "execution.isolation.semaphore.maxConcurrentRequests", value = "1"),
@HystrixProperty(name = "execution.isolation.strategy", value = "SEMAPHORE")
})
public String hystrix2() throws InterruptedException {
Thread.sleep(500);
logger.info("hystrix2");
return "hystrix2";
}
public String error() {
logger.info("fail");
return "fail";
}
}
并发测试结果：
条件：最大并发量=1
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.util.Date;
import java.util.concurrent.CountDownLatch;
public class TestBench {
private final static Logger logger = LoggerFactory.getLogger(TestBench.class);
//模拟的并发量
private static final int CONCURRENT_NUM = 42;
private static String url = "http://localhost:8080/hystrix2";
private static CountDownLatch cdl = new CountDownLatch(CONCURRENT_NUM);
public static void main(String[] args) {
for (int i = 0; i < CONCURRENT_NUM; i++) {
new Thread(new Demo()).start();
cdl.countDown();
}
}
public static class Demo implements Runnable{
@Override
public void run() {
try {
cdl.await();
} catch (Exception e) {
e.printStackTrace();
}
//使用工具类发送http请求
String json2 = requestPost(url, "");
logger.info(new Date().getTime()+"::"+json2);
}
}
/**
* 发送POST请求，参数是JSON
*/
public static String requestPost(String url, String jsonParam){
logger.info("HttpTool.requestPost 开始 请求url：" + url + ", 参数：" + jsonParam);
//创建HttpClient对象
CloseableHttpClient client = HttpClients.createDefault();
//创建HttpPost对象
HttpPost httpPost = new HttpPost(url);
//配置请求参数
RequestConfig requestConfig = RequestConfig.custom()
.setCookieSpec(CookieSpecs.DEFAULT)
.setExpectContinueEnabled(true)
.setSocketTimeout(5000)
.setConnectTimeout(5000)
.setConnectionRequestTimeout(5000)
.build();
httpPost.setConfig(requestConfig);
String respContent = null;
//设置参数和请求方式
StringEntity entity = new StringEntity(jsonParam,"UTF-8");//解决中文乱码问题
entity.setContentEncoding("UTF-8");
entity.setContentType("application/json");
httpPost.setEntity(entity);
HttpResponse resp;
try {
//执行请求
resp = client.execute(httpPost);
if(resp.getStatusLine().getStatusCode() == 200) {
HttpEntity responseObj = resp.getEntity();
respContent = EntityUtils.toString(responseObj,"UTF-8");
}
} catch (IOException e) {
e.printStackTrace();
logger.info("HttpTool.requestPost 异常 请求url：" + url + ", 参数：" + jsonParam + "，异常信息：" + e);
}
logger.info("HttpTool.requestPost 结束 请求url：" + url + ", 参数：" + jsonParam + "");
//返回数据
return respContent;
}
}
在多次模拟42个并发请求时，由于不能严格的模拟测试脚本和server之间的并发请求，通过前后日志输出，来验证，基本符合预期。
hystrix不但可以通过线程池限流，也可以通过信号量来进行限流，不过都是通过控制并发数来实现的，没有直接提供类似QPS的限流方式，不过也可以估算并相互转化。
一般设置标准：
**线程池大小 = 峰值QPS * （99耗时/1s） + 预留线程数 **
工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/hystrix
3.使用resilience限流
介绍：这个库受到Hystrix的启发，但提供了更方便的API和许多其他特性，如RateLimiter(阻塞太频繁的请求)、Bulkhead(避免太多并发请求)等。
原理：Resilience4j的限流器RateLimiter实现了令牌桶限流和基于信号量的固定并发数限流。
代码：
CurrentLimiterConfig.java
import io.github.resilience4j.bulkhead.Bulkhead;
import io.github.resilience4j.bulkhead.BulkheadConfig;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RateLimiterConfig;
import org.springframework.context.annotation.Configuration;
import java.time.Duration;
@Configuration
public class CurrentLimiterConfig {
protected final RateLimiter rateLimiter;
protected final RateLimiter rateLimiter2;
protected final Bulkhead bulkhead;
public CurrentLimiterConfig(){
rateLimiter = RateLimiter.of("Limiter1",RateLimiterConfig.custom()
.limitForPeriod(20)
.limitRefreshPeriod(Duration.ofSeconds(1))
.timeoutDuration(Duration.ofMillis(5))
.build());
rateLimiter2 = RateLimiter.of("Limiter2",RateLimiterConfig.custom()
.limitForPeriod(100)
.limitRefreshPeriod(Duration.ofSeconds(5))
.timeoutDuration(Duration.ofMillis(5))
.build());
bulkhead = Bulkhead.of("Bulkhead", BulkheadConfig.custom()
.maxConcurrentCalls(1)
.build());
}
}
RateLimiterController.java
package com.example.resilience;
import io.github.resilience4j.bulkhead.Bulkhead;
import io.github.resilience4j.bulkhead.BulkheadFullException;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RequestNotPermitted;
import io.vavr.control.Try;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class RateLimiterController extends CurrentLimiterConfig {
private final static Logger logger = LoggerFactory.getLogger(RateLimiterController.class);
private final RateLimiterService service = new RateLimiterService();
@RequestMapping("/resilience")
public String resilience() {
String result = Try.ofSupplier(RateLimiter.decorateSupplier(rateLimiter, service::service))
.recover(CommonException.class, "fail")
.recover(RequestNotPermitted.class,"Request not permitted for limiter")
.get();
logger.info(result);
return result;
}
@RequestMapping("/resilience2")
public String resilience2() {
logger.info("enter resilience2");
String result = Try.ofSupplier(Bulkhead.decorateSupplier(bulkhead, service::service2))
.recover(CommonException.class, "fail")
.recover(RequestNotPermitted.class,"Request not permitted for limiter")
.recover(BulkheadFullException.class,"Bulkhead name is full")
.get();
logger.info(result);
return result;
}
}
RateLimiterService.java
package com.example.resilience;
import org.springframework.stereotype.Service;
@Service
public class RateLimiterService {
public String service() {
return "resilience";
}
public String service2() {
return "resilience2";
}
}
ab测试结果：
RateLimiter:
条件：单位时间周期允许个数=20，时间周期=1秒，请求超时时间=5毫秒
[D:\Apache24\bin]$ ab -n 42 -c 42 "http://localhost:8080/resilience"
在多次模拟42个并发请求时，平均失败了20个，符合预期。
并发测试结果：
Bulkhead:
条件：最大并发数=1
在多次模拟42个并发请求时，同hystrix的测试，基本符合预期。
工程代码地址：https://github.com/xiedeyantu/currentlimiting/tree/master/resilience
总结：
如果侧重控制单位时间请求量，可以使用guava的RateLimiter或Resilience的RateLimiter，如果侧重控制并发请求量，可以使用hystrix或Resilience的Bulkhead。
对于一般需求个人认为Resilience的RateLimiter比较好，参数清晰，配置也比较辩解，但是基于Java8的函数式编程需要花时间熟悉一下。