高并发项目总结
功能实现
两次MD5加密密码
- 第一次MD5
Http是明文传输，为了不泄露密码，用户在前端表单输入时，在前端对密钥做第一次MD5处理，发送给后端。 - 第二次MD5
后端收到密码后做第二次MD5处理，并存入数据库中。这是防止数据库泄露根据彩虹表反向推到出用户密钥。
为了增强密码安全，增加随机盐值混淆密码
JSR303参数校验
对VO层对象的属性设定限制，如格式、非空等，无需在每个方法里重复判断。
使用注解的方式对VO属性进行限制，如对LoginVO的mobile属性做手机号格式匹配，新建IsMobile注解。
@Target({ METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER })
@Retention(RUNTIME)
@Documented
@Constraint(validatedBy = {IsMobileValidator.class })
public @interface IsMobile {
boolean required() default true;
String message() default "手机号码格式错误";
Class<?>[] groups() default { };
Class<? extends Payload>[] payload() default { };
}
public class IsMobileValidator implements ConstraintValidator<IsMobile, String> {
private boolean required = false;
public void initialize(IsMobile constraintAnnotation) {
required = constraintAnnotation.required();
}
public boolean isValid(String value, ConstraintValidatorContext context) {
if(required) {
return ValidatorUtil.isMobile(value);
}else {
if(StringUtils.isEmpty(value)) {
return true;
}else {
return ValidatorUtil.isMobile(value);
}
}
}
}
Redis集成
- 创建与配置文件对应的Bean，方便访问使用
@Component
@ConfigurationProperties(prefix="redis")
public class RedisConfig {
private String host;
private int port;
private int timeout;//秒
private String password;
private int poolMaxTotal;
private int poolMaxIdle;
private int poolMaxWait;//秒
//get and set
}
- 构建redis连接池
@Service
public class RedisPoolFactory {
@Autowired
RedisConfig redisConfig;
@Bean
public JedisPool JedisPoolFactory() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxIdle(redisConfig.getPoolMaxIdle());
poolConfig.setMaxTotal(redisConfig.getPoolMaxTotal());
poolConfig.setMaxWaitMillis(redisConfig.getPoolMaxWait() * 1000);
JedisPool jp = new JedisPool(poolConfig, redisConfig.getHost(), redisConfig.getPort(),
redisConfig.getTimeout()*1000, redisConfig.getPassword(), 0);
return jp;
}
}
- 手写redis客户端
仅贴出核心get和set方法。为了解决冲突问题，对key进行分组化管理，不同模块拥有不同的key前缀，拼接成真正的key。key前缀包含过期时间和前缀字符串。
public <T> T get(KeyPrefix prefix, String key, Class<T> clazz) {
Jedis jedis = null;
try {
jedis = jedisPool.getResource();
//生成真正的key
String realKey = prefix.getPrefix() + key;
String str = jedis.get(realKey);
T t = stringToBean(str, clazz);
return t;
}finally {
returnToPool(jedis);
}
}
public <T> boolean set(KeyPrefix prefix, String key, T value) {
Jedis jedis = null;
try {
jedis = jedisPool.getResource();
String str = beanToString(value);
if(str == null || str.length() <= 0) {
return false;
}
//生成真正的key
String realKey = prefix.getPrefix() + key;
int seconds = prefix.expireSeconds();
if(seconds <= 0) {
jedis.set(realKey, str);
}else {
jedis.setex(realKey, seconds, str);
}
return true;
}finally {
returnToPool(jedis);
}
}
private void returnToPool(Jedis jedis) {
if(jedis != null) {
jedis.close();
}
}
全局异常处理
创建异常监听处理器负责处理所有项目异常。
@ControllerAdvice
@ResponseBody
public class GlobalExceptionHandler {
@ExceptionHandler(value=Exception.class)
public Result<String> exceptionHandler(HttpServletRequest request, Exception e){
e.printStackTrace();
if(e instanceof GlobalException) {
GlobalException ex = (GlobalException)e;
return Result.error(ex.getCm());
}else if(e instanceof BindException) {
BindException ex = (BindException)e;
List<ObjectError> errors = ex.getAllErrors();
ObjectError error = errors.get(0);
String msg = error.getDefaultMessage();
return Result.error(CodeMsg.BIND_ERROR.fillArgs(msg));
}else {
return Result.error(CodeMsg.SERVER_ERROR);
}
}
}
分布式Session
在多台服务器上进行Session的共享，将用户的Session存储在单独的Redis服务器上。
每次用户登录时，后端产生随机唯一UUID值作为token，加入到用户的cookie中，同时将用户对象和该token存储在redis中。
在每次请求时会从cookie中拿到这个token值并从redis中取出对象，每次有效访问都会延长本次session的存活时间。
//生成cookie
String token = UUIDUtil.uuid();
addCookie(response, token, user);
private void addCookie(HttpServletResponse response, String token, MiaoshaUser user) {
redisService.set(MiaoshaUserKey.token, token, user);
Cookie cookie = new Cookie(COOKI_NAME_TOKEN, token);
cookie.setMaxAge(MiaoshaUserKey.token.expireSeconds());
cookie.setPath("/");
response.addCookie(cookie);
}
user自动封装
实现WebMvcConfigurerAdapter的addArgumentResolvers方法对请求中的User进行自动封装。
@Configuration
public class WebConfig extends WebMvcConfigurerAdapter{
@Autowired
UserArgumentResolver userArgumentResolver;
@Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {
argumentResolvers.add(userArgumentResolver);
}
}
@Service
public class UserArgumentResolver implements HandlerMethodArgumentResolver {
@Autowired
MiaoshaUserService userService;
public boolean supportsParameter(MethodParameter parameter) {
Class<?> clazz = parameter.getParameterType();
return clazz==MiaoshaUser.class;
}
public Object resolveArgument(MethodParameter parameter, ModelAndViewContainer mavContainer,
NativeWebRequest webRequest, WebDataBinderFactory binderFactory) throws Exception {
HttpServletRequest request = webRequest.getNativeRequest(HttpServletRequest.class);
HttpServletResponse response = webRequest.getNativeResponse(HttpServletResponse.class);
String paramToken = request.getParameter(MiaoshaUserService.COOKI_NAME_TOKEN);
String cookieToken = getCookieValue(request, MiaoshaUserService.COOKI_NAME_TOKEN);
if(StringUtils.isEmpty(cookieToken) && StringUtils.isEmpty(paramToken)) {
return null;
}
String token = StringUtils.isEmpty(paramToken)?cookieToken:paramToken;
return userService.getByToken(response, token);
}
private String getCookieValue(HttpServletRequest request, String cookiName) {
Cookie[] cookies = request.getCookies();
if(cookies == null || cookies.length <= 0){
return null;
}
for(Cookie cookie : cookies) {
if(cookie.getName().equals(cookiName)) {
return cookie.getValue();
}
}
return null;
}
}
高并发优化
页面级优化
页面缓存
ajax方式请求，后端返回html字节流。使用thymeleafViewResolver进行手动渲染页面，并将html缓存到redis中。
@RequestMapping(value="/to_list", produces="text/html")
@ResponseBody
public String list(HttpServletRequest request, HttpServletResponse response, Model model,MiaoshaUser user) {
model.addAttribute("user", user);
//取缓存
// String html = redisService.get(GoodsKey.getGoodsList, "", String.class);
// if(!StringUtils.isEmpty(html)) {
// return html;
// }
List<GoodsVo> goodsList = goodsService.listGoodsVo();
model.addAttribute("goodsList", goodsList);
// return "goods_list";
SpringWebContext ctx = new SpringWebContext(request,response,
request.getServletContext(),request.getLocale(), model.asMap(), applicationContext );
//手动渲染
String html = thymeleafViewResolver.getTemplateEngine().process("goods_list", ctx);
if(!StringUtils.isEmpty(html)) {
redisService.set(GoodsKey.getGoodsList, "", html);
}
return html;
}
对象缓存
将数据库中查到的对象缓存到redis中，减少数据库访问。
页面静态化
Spring提供页面在前端静态化缓存，如下配置文件缓存有效期3600秒
#static
spring.resources.add-mappings=true
spring.resources.cache-period= 3600
spring.resources.chain.cache=true
spring.resources.chain.enabled=true
spring.resources.chain.gzipped=true
spring.resources.chain.html-application-cache=true
spring.resources.static-locations=classpath:/static/
在缓存有效期内，前端页面无需请求页面内容，仅需要请求关键数据，使用ajax请求接口数据，再使用js进行动态渲染。
@RequestMapping(value="/detail/{goodsId}")
@ResponseBody
public Result<GoodsDetailVo> detail(HttpServletRequest request, HttpServletResponse response, Model model,MiaoshaUser user,
@PathVariable("goodsId")long goodsId) {
GoodsVo goods = goodsService.getGoodsVoByGoodsId(goodsId);
long startAt = goods.getStartDate().getTime();
long endAt = goods.getEndDate().getTime();
long now = System.currentTimeMillis();
int miaoshaStatus = 0;
int remainSeconds = 0;
if(now < startAt ) {//秒杀还没开始，倒计时
miaoshaStatus = 0;
remainSeconds = (int)((startAt - now )/1000);
}else if(now > endAt){//秒杀已经结束
miaoshaStatus = 2;
remainSeconds = -1;
}else {//秒杀进行中
miaoshaStatus = 1;
remainSeconds = 0;
}
GoodsDetailVo vo = new GoodsDetailVo();
vo.setGoods(goods);
vo.setUser(user);
vo.setRemainSeconds(remainSeconds);
vo.setMiaoshaStatus(miaoshaStatus);
return Result.success(vo);
}
服务端优化
Redis和Map减库存
首先初始化时将秒杀物品从数据库读取出来添加到redis和本地map中，本地map仅用来标志库存是否还有。
在秒杀请求进来时，先从map查询库存标志，不为空则在redis中进行减库存操作，若库存清空，设置map标志位即可。
public class MiaoshaController implements InitializingBean {
private HashMap<Long, Boolean> localOverMap = new HashMap<Long, Boolean>();
/**
* 系统初始化
* */
public void afterPropertiesSet() throws Exception {
List<GoodsVo> goodsList = goodsService.listGoodsVo();
if(goodsList == null) {
return;
}
for(GoodsVo goods : goodsList) {
redisService.set(GoodsKey.getMiaoshaGoodsStock, ""+goods.getId(), goods.getStockCount());
localOverMap.put(goods.getId(), false);
}
}
@RequestMapping(value="/{path}/do_miaosha", method=RequestMethod.POST)
@ResponseBody
public Result<Integer> miaosha(Model model,MiaoshaUser user,
@RequestParam("goodsId")long goodsId,
@PathVariable("path") String path) {
model.addAttribute("user", user);
if(user == null) {
return Result.error(CodeMsg.SESSION_ERROR);
}
//内存标记，减少redis访问
boolean over = localOverMap.get(goodsId);
if(over) {
return Result.error(CodeMsg.MIAO_SHA_OVER);
}
//预减库存
long stock = redisService.decr(GoodsKey.getMiaoshaGoodsStock, ""+goodsId);//10
if(stock < 0) {
localOverMap.put(goodsId, true);
return Result.error(CodeMsg.MIAO_SHA_OVER);
}
//判断是否已经秒杀到了
MiaoshaOrder order = orderService.getMiaoshaOrderByUserIdGoodsId(user.getId(), goodsId);
if(order != null) {
return Result.error(CodeMsg.REPEATE_MIAOSHA);
}
//秒杀
}
}
Rabbitmq
MiaoshaMessage mm = new MiaoshaMessage();
mm.setUser(user);
mm.setGoodsId(goodsId);
sender.sendMiaoshaMessage(mm);
return Result.success(0);//排队中
public class MiaoshaMessage {
private MiaoshaUser user;
private long goodsId;
}
@Service
public class MQSender {
private static Logger log = LoggerFactory.getLogger(MQSender.class);
@Autowired
AmqpTemplate amqpTemplate ;
public void sendMiaoshaMessage(MiaoshaMessage mm) {
String msg = RedisService.beanToString(mm);
log.info("send message:"+msg);
amqpTemplate.convertAndSend(MQConfig.MIAOSHA_QUEUE, msg);
}
}
@RabbitListener(queues=MQConfig.MIAOSHA_QUEUE)
public void receive(String message) {
log.info("receive message:"+message);
MiaoshaMessage mm = RedisService.stringToBean(message, MiaoshaMessage.class);
MiaoshaUser user = mm.getUser();
long goodsId = mm.getGoodsId();
GoodsVo goods = goodsService.getGoodsVoByGoodsId(goodsId);
int stock = goods.getStockCount();
if(stock <= 0) {
return;
}
//判断是否已经秒杀到了
MiaoshaOrder order = orderService.getMiaoshaOrderByUserIdGoodsId(user.getId(), goodsId);
if(order != null) {
return;
}
//减库存 下订单 写入秒杀订单
miaoshaService.miaosha(user, goods);
}
安全问题
验证码防刷
后端生成数据公式使用JFX绘出图片返回给前端，并将结果写入redis，在秒杀之前需要做验证码才能请求到真实秒杀地址。
隐藏秒杀地址
在秒杀时，先向服务端获取一个随机数
@AccessLimit(seconds=5, maxCount=5, needLogin=true)
@RequestMapping(value="/path", method=RequestMethod.GET)
@ResponseBody
public Result<String> getMiaoshaPath(HttpServletRequest request, MiaoshaUser user,
@RequestParam("goodsId")long goodsId,
@RequestParam(value="verifyCode", defaultValue="0")int verifyCode
) {
if(user == null) {
return Result.error(CodeMsg.SESSION_ERROR);
}
boolean check = miaoshaService.checkVerifyCode(user, goodsId, verifyCode);
if(!check) {
return Result.error(CodeMsg.REQUEST_ILLEGAL);
}
String path =miaoshaService.createMiaoshaPath(user, goodsId);
return Result.success(path);
}
后端会将该随机数计入到redis中，然后前端达到随机数进行请求秒杀，后端接到后验证随机数是否正确。若正确，则进行秒杀，异步返回结果排队中。前端轮询秒杀结果。
@RequestMapping(value="/{path}/do_miaosha", method=RequestMethod.POST)
通用接口限流
限流思路：使用redis的过期时间进行计时，如在5秒内仅允许访问5次，则设置redis过期时间5秒，value为次数并计数。
为了解耦和更加通用，此类非业务代码使用拦截器实现。
新建注解@AccessLimit(seconds=5, maxCount=5, needLogin=true)代表5秒5次且需要登录。
@Retention(RUNTIME)
@Target(METHOD)
public @interface AccessLimit {
int seconds();
int maxCount();
boolean needLogin() default true;
}
@Service
public class AccessInterceptor extends HandlerInterceptorAdapter{
@Autowired
MiaoshaUserService userService;
@Autowired
RedisService redisService;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
if(handler instanceof HandlerMethod) {
MiaoshaUser user = getUser(request, response);
UserContext.setUser(user);
HandlerMethod hm = (HandlerMethod)handler;
AccessLimit accessLimit = hm.getMethodAnnotation(AccessLimit.class);
if(accessLimit == null) {
return true;
}
int seconds = accessLimit.seconds();
int maxCount = accessLimit.maxCount();
boolean needLogin = accessLimit.needLogin();
String key = request.getRequestURI();
if(needLogin) {
if(user == null) {
render(response, CodeMsg.SESSION_ERROR);
return false;
}
key += "_" + user.getId();
}else {
//do nothing
}
AccessKey ak = AccessKey.withExpire(seconds);
Integer count = redisService.get(ak, key, Integer.class);
if(count == null) {
redisService.set(ak, key, 1);
}else if(count < maxCount) {
redisService.incr(ak, key);
}else {
render(response, CodeMsg.ACCESS_LIMIT_REACHED);
return false;
}
}
return true;
}
private void render(HttpServletResponse response, CodeMsg cm)throws Exception {
response.setContentType("application/json;charset=UTF-8");
OutputStream out = response.getOutputStream();
String str = JSON.toJSONString(Result.error(cm));
out.write(str.getBytes("UTF-8"));
out.flush();
out.close();
}
private MiaoshaUser getUser(HttpServletRequest request, HttpServletResponse response) {
String paramToken = request.getParameter(MiaoshaUserService.COOKI_NAME_TOKEN);
String cookieToken = getCookieValue(request, MiaoshaUserService.COOKI_NAME_TOKEN);
if(StringUtils.isEmpty(cookieToken) && StringUtils.isEmpty(paramToken)) {
return null;
}
String token = StringUtils.isEmpty(paramToken)?cookieToken:paramToken;
return userService.getByToken(response, token);
}
private String getCookieValue(HttpServletRequest request, String cookiName) {
Cookie[] cookies = request.getCookies();
if(cookies == null || cookies.length <= 0){
return null;
}
for(Cookie cookie : cookies) {
if(cookie.getName().equals(cookiName)) {
return cookie.getValue();
}
}
return null;
}
}
这里将User的获取挪到了拦截器中，并存储到了UserContext中
public class UserContext {
private static ThreadLocal<MiaoshaUser> userHolder = new ThreadLocal<MiaoshaUser>();
}
原参数解析器UserArgumentResolver直接从UserContext中取值即可。
注册拦截器：
@Configuration
public class WebConfig extends WebMvcConfigurerAdapter{
@Autowired
AccessInterceptor accessInterceptor;
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(accessInterceptor);
}
}
超买超卖
在减库存的时候添加判定条件and stock_count > 0，即使用数据库自身的锁，解决超卖问题。
@Update("update miaosha_goods set stock_count = stock_count - 1 where goods_id = #{goodsId} and stock_count > 0")
public int reduceStock(MiaoshaGoods g);
在秒杀订单数据库中，增加用户和商品的唯一约束，在重复购买插入订单时会因为唯一约束导致订单插入失败，从而解决超买问题。