API 限流器（三） 在Spring Cloud 微服务体系中集成RedisRateLimiter
这篇是API限流器这个系列的终章，就是讲述如何在Spring Cloud 微服务开发中应用我发明的先进限流器。
本项目的代码地址为：https://github.com/tangaiyun/spring-cloud-microservice-ratelimiter
限流器的代码地址为：https://github.com/tangaiyun/RedisRateLimiter
开篇明义，基本思路如下：
1. 定义一个annotation - RedisLimiter
2. 在RestController 中有URL Mapping 的方法上应用RedisLimiter注解
3. 定义一个拦截器 - RateLimitCheckInterceptor，继承org.springframework.web.servlet.HandlerInterceptor
4. 定义一个WebMvcConfigurerAdapter，把上面的RateLimitCheckInterceptor启用，拦截所有的URL请求
5. 定义一个RedisRateLimiterFactory，这个工厂对RedisLimiter对象做了一些缓存。
项目的整个代码结构如图所示：
基本的调用时序图如下：
下面我将就一些重要的类和配置逐一讲解：
1. 首先是application.yml
这个文件是Spring Cloud 微服务的重点配置文件，内容为：
server:
port: 8080 #微服务端口
spring:
application:
name: ratelimitersample #微服务名称
jpa: #JPA的配置信息
generate-ddl: false
show-sql: true
hibernate:
ddl-auto: none
datasource: # 指定数据源
platform: h2 # 指定数据源类型
schema: classpath:schema.sql # 指定h2数据库的建表脚本
data: classpath:data.sql # 指定h2数据库的数据脚本
redis: # Redis配置信息
host: 192.168.0.135
port: 6379
password: foobared
timeout: 2000
pool: #Jedis Pool的配置信息
maxIdle: 300
minIdle: 100
max-wait: -1
max-active: 600
logging: # 配置日志级别，让hibernate打印出执行的SQL
level:
root: INFO
org.hibernate: INFO
org.hibernate.type.descriptor.sql.BasicBinder: TRACE
org.hibernate.type.descriptor.sql.BasicExtractor: TRACE
eureka: #eureka的配置信息
client:
healthcheck: #打开服务健康检查
enabled: true
serviceUrl:
defaultZone: http://192.168.0.118:8761/eureka/
instance: #服务实例细信息
app-group-name: product
prefer-ip-address: true
instance-id: ${spring.application.name}:${server.port}
lease-expiration-duration-in-seconds: 10
lease-renewal-interval-in-seconds: 5
2. RateLimiter自定义annotation
package com.tay.ratelimitersample.ratelimiter;
import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;
@Retention(RUNTIME)
@Target({ TYPE, METHOD })
public @interface RateLimiter {
public enum Base {
General, //统一控制
IP, //按IP限制
User //按用户控制
};
Base base();
String path() default "";
TimeUnit timeUnit() default TimeUnit.SECONDS;
int permits();
}
这个注解有4个属性，第一个为base,意思是限流的基准是什么，内置的枚举类型支持三种基准，统一控制、基于IP控制、基于用户控制。
第二属性为path, 一般来说这个不用配置，但是如果你的RestController方法映射的URL上带有URL参数，类似@GetMapping("/testforid/{id}") 这种，这时候这个path配置就很有必要了，本例中，你可以将path配置为path="/testforid"。
第三个属性为timeUnit, 限流的时间单位，支持TimeUnit.SECONDS,TimeUnit.MINUTES,TimeUnit.HOURS,TimeUnit.DAYS, 即秒，分，小时，天
第四个属性为permits,意思是单位时间允许访问的次数限制。
3. RedisRateLimiter, 此类为核心类，算法原理在此系列文章二中做了详细讲解，本文中略过。
4. RedisRateLimiterFactory， RedisRateLimiter的工厂类，主要是负责创建RedisRateLimiter对象，缓存RedisRateLimiter对象。
package com.tay.ratelimitersample.ratelimiter;
import java.util.WeakHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import redis.clients.jedis.JedisPool;
@Component
public class RedisRateLimiterFactory {
@Autowired
private JedisPool jedisPool;
private final WeakHashMap limiterMap = new WeakHashMap();
private ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
public JedisPool getJedisPool() {
return jedisPool;
}
public void setJedisPool(JedisPool jedisPool) {
this.jedisPool = jedisPool;
}
public RedisRateLimiter get(String keyPrefix, TimeUnit timeUnit, int permits) {
RedisRateLimiter redisRateLimiter = null;
try {
lock.readLock().lock();
if(limiterMap.containsKey(keyPrefix)) {
redisRateLimiter = limiterMap.get(keyPrefix);
}
}
finally {
lock.readLock().unlock();
}
if(redisRateLimiter == null) {
try {
lock.writeLock().lock();
if(limiterMap.containsKey(keyPrefix)) {
redisRateLimiter = limiterMap.get(keyPrefix);
}
if(redisRateLimiter == null) {
redisRateLimiter = new RedisRateLimiter(jedisPool, timeUnit, permits);
limiterMap.put(keyPrefix, redisRateLimiter);
}
}
finally {
lock.writeLock().unlock();
}
}
return redisRateLimiter;
}
}
5. RedisConfig， Redis配置信息读取类，并在其中声明了一个RedisRateLimiterFactory Bean.
package com.tay.ratelimitersample.ratelimiter;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
@Configuration
public class RedisConfig {
@Value("${spring.redis.host}")
private String host;
@Value("${spring.redis.port}")
private int port;
@Value("${spring.redis.password}")
private String password;
@Value("${spring.redis.timeout}")
private int timeout;
@Value("${spring.redis.pool.maxIdle}")
private int maxIdle;
@Value("${spring.redis.pool.minIdle}")
private int minIdle;
@Value("${spring.redis.pool.max-wait}")
private long maxWaitMillis;
@Value("${spring.redis.pool.max-active}")
private int maxActive;
@Bean
public JedisPool redisPoolFactory() {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxIdle(maxIdle);
jedisPoolConfig.setMinIdle(minIdle);
jedisPoolConfig.setMaxWaitMillis(maxWaitMillis);
jedisPoolConfig.setMaxTotal(maxActive);
JedisPool jedisPool = new JedisPool(jedisPoolConfig, host, port, timeout, password);
return jedisPool;
}
}
6. RateLimitCheckInterceptor类，此类为Spring cloud集成的关键类。
package com.tay.ratelimitersample.ratelimiter;
import java.lang.reflect.Method;
import java.util.concurrent.TimeUnit;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import org.springframework.web.servlet.ModelAndView;
@Component
public class RateLimitCheckInterceptor implements HandlerInterceptor {
private static final String[] IP_HEADER_KYES = {
"X-Forwarded-For",
"X-Real-IP",
"Proxy-Client-IP",
"WL-Proxy-Client-IP",
"HTTP_CLIENT_IP",
"HTTP_X_FORWARDED_FOR" };
private static final String USER_TOKEN_KEY = "UserToken";
@Autowired
private RedisRateLimiterFactory redisRateLimiterFactory;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
if (!(handler instanceof HandlerMethod)) {
return true;
}
boolean isSuccess = true;
HandlerMethod handlerMethod = (HandlerMethod) handler;
Method method = handlerMethod.getMethod();
if (method.isAnnotationPresent(RateLimiter.class)) {
RateLimiter rateLimiterAnnotation = method.getAnnotation(RateLimiter.class);
int permits = rateLimiterAnnotation.permits();
TimeUnit timeUnit = rateLimiterAnnotation.timeUnit();
String path = rateLimiterAnnotation.path();
if ("".equals(path)) {
path = request.getRequestURI();
}
if (rateLimiterAnnotation.base() == RateLimiter.Base.General) {
String rateLimiterKey = path;
RedisRateLimiter redisRatelimiter = redisRateLimiterFactory.get(path, timeUnit,
permits);
isSuccess = rateCheck(redisRatelimiter, rateLimiterKey, response);
} else if (rateLimiterAnnotation.base() == RateLimiter.Base.IP) {
String ip = getIP(request);
if (ip != null) {
String rateLimiterKey = path + ":" + ip;
RedisRateLimiter redisRatelimiter = redisRateLimiterFactory.get(rateLimiterKey, timeUnit,
permits);
isSuccess = rateCheck(redisRatelimiter, rateLimiterKey, response);
}
} else if (rateLimiterAnnotation.base() == RateLimiter.Base.User) {
String userToken = getUserToken(request);
if (userToken != null) {
String rateLimiterKey = path + ":" + userToken;
RedisRateLimiter redisRatelimiter = redisRateLimiterFactory.get(rateLimiterKey, timeUnit,
permits);
isSuccess =rateCheck(redisRatelimiter, rateLimiterKey, response);
}
}
}
return isSuccess;
}
private boolean rateCheck(RedisRateLimiter redisRatelimiter, String keyPrefix, HttpServletResponse response)
throws Exception {
if (!redisRatelimiter.acquire(keyPrefix)) {
response.setStatus(HttpStatus.FORBIDDEN.value());
response.getWriter().print("Access denied because of exceeding access rate");
return false;
}
return true;
}
private String getIP(HttpServletRequest request) {
for (String ipHeaderKey : IP_HEADER_KYES) {
String ip = request.getHeader(ipHeaderKey);
if (ip != null && ip.length() != 0 && (!"unknown".equalsIgnoreCase(ip))) {
return ip;
}
else {
return request.getRemoteHost();
}
}
return null;
}
private String getUserToken(HttpServletRequest request) {
String userToken = request.getHeader(USER_TOKEN_KEY);
return userToken;
}
@Override
public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler,
ModelAndView modelAndView) throws Exception {
}
@Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex)
throws Exception {
}
}
此类尤为关键，当我们api限流是基于IP或用户的话，那么我们有一些约定：
基于IP限流的话，则http request的中header里面必须带有ip信息，ip的header支持
"X-Forwarded-For",
"X-Real-IP",
"Proxy-Client-IP",
"WL-Proxy-Client-IP",
"HTTP_CLIENT_IP",
"HTTP_X_FORWARDED_FOR"
特别注意，现在流行会把Nginx作为分发服务器，则Nginx forward正式客户端请求到下游微服务时，务必要把客户的真实IP塞入到header中往下传递，需要在Nginx server配置中加入
proxy_set_header X-Real-IP $remote_addr;7. WebMvcConfigurer， 此类的作用即将拦截器RateLimitCheckInterceptor对所有的API访问生效
package com.tay.ratelimitersample.ratelimiter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
@Configuration
public class WebMvcConfigurer extends WebMvcConfigurerAdapter {
@Autowired
private RateLimitCheckInterceptor rateLimitCheckInterceptor;
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(rateLimitCheckInterceptor).addPathPatterns("/**");
super.addInterceptors(registry);
}
}
地址patterns指定为"/**",即对所有请求生效。
8. RestController
package com.tay.ratelimitersample.controller;
import java.util.concurrent.TimeUnit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.tay.ratelimitersample.entity.User;
import com.tay.ratelimitersample.ratelimiter.RateLimiter;
import com.tay.ratelimitersample.repository.UserRepository;
@RestController
@RequestMapping("/api")
public class UserController {
@Autowired
private UserRepository userRepository;
@GetMapping("/{id}")
public User findById(@PathVariable Long id, @RequestHeader HttpHeaders headers) {
User findOne = this.userRepository.findOne(id);
return findOne;
}
@RateLimiter(base = RateLimiter.Base.General, permits = 2, timeUnit = TimeUnit.MINUTES)
@GetMapping("/test")
public String test() {
return "test!";
}
@RateLimiter(base = RateLimiter.Base.IP, path="/testforid", permits = 4, timeUnit = TimeUnit.MINUTES)
@GetMapping("/testforid/{id}")
public String testforid(@PathVariable Long id) {
return "test! " + id;
}
}
其中，/api/test限流是对所有请求限定为1分钟内仅限2次。 对于/api/testforid/{id}， 是针对于每个请求ip，限定为每分钟4次，注意，这里跟上面一个配置有点小区别，就是指定了path, 因为这里的url中带了PathVariable, 但实际上我们限流时并不关心具体的PathVariable是什么，所以要指定path为/testforid, 就是所有请求都会忽略{id}, 都归结到/testforid请求。
8. 运行环境， 本例运行是需要redis服务，和一个Eureka服务。如果你不想启动Eureka亦可，把ProviderUserApplication类的注解@EnableDiscoveryClient去掉，并把application.yml中的关于Eureka的配置去掉：
eureka:
client:
healthcheck:
enabled: true
serviceUrl:
defaultZone: http://192.168.0.118:8761/eureka/
instance:
app-group-name: product
prefer-ip-address: true
instance-id: ${spring.application.name}:${server.port}
lease-expiration-duration-in-seconds: 10
lease-renewal-interval-in-seconds: 5
9.测试
测试工具firefox上的插件RESTED Client和Redis Desktop Manager
a. 通过RESTED client 访问 http://192.168.0.118:8080/api/testforid/1，访问成功，则：
连续点击访问，达到访问频率限制，则：
观察redis中数据：
