SpringBoot之RabbitMQ实现延时队列（订单延迟取消，消费端丢失消息的防止）

SpringBoot之RabbitMQ实现延时队列（订单延迟取消，消费端丢失消息的防止）

1.1. 什么是死信队列

• DLX，全称为 Dead-Letter-Exchange，可以称之为死信交换器，也有人称之为死信邮箱。当
  消息在一个队列中变成死信 (dead message) 之后，它能被重新被发送到另一个交换器中，这个
  交换器就是 DLX，绑定 DLX 的队列就称之为死信队列。
• 消息变成死信一般是由于以下几种情况 :
• 消息被拒绝 (Basic.Reject/Basic .Nack)，井且设置 requeue 参数为 false;
• 消息过期;
• 队列达到最大长度。
• DLX 也是一个正常的交换器，和一般的交换器没有区别，它能在任何的队列上被指定， 实际上就是设置某个队列的属性。当这个队列中存在死信时，RabbitMQ 就会自动地将这个消息重新发布到设置的 DLX 上去，进而被路由到另一个队列，即死信队列。可以监听这个队列中的消 息、以进行相应的处理，这个特性与将消息的 TTL 设置为 0 配合使用可以弥补 imrnediate 参数
  的功能。
  通过在 channel.queueDeclare 方法中设置 x-dead-letter-exchange 参数来为这 个队列添加 DLX
  

1.2 什么是延迟队列

延迟队列存储的对象是对应的延迟消息，所谓"延迟消息"是指当消息被发送以后，并不
想让消费者立刻拿到消息，而是等待特定时间后，消费者才能拿到这个消息进行消费。

• 延迟队列的使用场景 ：

  • 令在订单系统中， 一个用户下单之后通常有 30 分钟的时间进行支付，如果 30 分钟之内
    没有支付成功，那么这个订单将进行异常处理，这时就可以使用延迟队列来处理这些订单了
  • 用户希望通过手机远程遥控家里的智能设备在指定的时间进行工作。这时候就可以将 用户指令发送到延迟队列，当指令设定的时间到了再将指令推送到智能设备。

• 在 AMQP 协议中，或者 RabbitMQ 本身没有直接支持延迟队列的功能，但是可以通过前面
  所介绍的 DLX 和 TTL 模拟出延迟队列的功能。

• Rabbitmq实现延时队列一般而言有两种形式：

  • 第一种方式：利用两个特性： Time To Live(TTL)、Dead Letter Exchanges（DLX）
  • 第二种方式：利用rabbitmq中的插件x-delay-message

Time-To-Live Extensions
RabbitMQ允许我们为消息或者队列设置TTL（time to live），也就是过期时间。TTL表明了一条消息可在队列中存活的最大时间，单位为毫秒。也就是说，当某条消息被设置了TTL或者当某条消息进入了设置了TTL的队列时，这条消息会在经过TTL秒后“死亡”，成为Dead Letter。如果既配置了消息的TTL，又配置了队列的TTL，那么较小的那个值会被取用。更多资料请查阅官方文档。

Dead Letter Exchange
刚才提到了，被设置了TTL的消息在过期后会成为Dead Letter。其实在RabbitMQ中，一共有三种消息的“死亡”形式：

• 消息被拒绝。通过调用basic.reject或者basic.nack并且设置的requeue参数为false。
• 消息因为设置了TTL而过期。
• 消息进入了一条已经达到最大长度的队列。
• 如果队列设置了Dead Letter Exchange（DLX），那么这些Dead Letter就会被重新publish到Dead Letter Exchange，通过Dead Letter Exchange路由到其他队列。更多资料请查阅官方文档。

1.3 延迟消费

延迟消费是延迟队列最为常用的使用模式。如下图所示，生产者产生的消息首先会进入缓冲队列（图中红色队列）。通过RabbitMQ提供的TTL扩展，这些消息会被设置过期时间，也就是延迟消费的时间。等消息过期之后，这些消息会通过配置好的DLX转发到实际消费队列（图中蓝色队列），以此达到延迟消费的效果。

1.4 延迟重试

延迟重试本质上也是延迟消费的一种，但是这种模式的结构与普通的延迟消费的流程图较为不同，所以单独拎出来介绍。

如下图所示，消费者发现该消息处理出现了异常，比如是因为网络波动引起的异常。那么如果不等待一段时间，直接就重试的话，很可能会导致在这期间内一直无法成功，造成一定的资源浪费。那么我们可以将其先放在缓冲队列中（图中红色队列），等消息经过一段的延迟时间后再次进入实际消费队列中（图中蓝色队列），此时由于已经过了“较长”的时间了，异常的一些波动通常已经恢复，这些消息可以被正常地消费。

1.5 代码实现

1.5.1 application.properties配置

# tomcat config
server.port=9999
# log config
logging.file=rabbitmq.log
# tomcat db pool
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/test?useSSL=false&useUnicode=true&characterEncoding=utf-8
spring.datasource.username=root
spring.datasource.password=123
spring.datasource.tomcat.max-active=10
spring.datasource.tomcat.initial-size=2
spring.datasource.tomcat.min-idle=3
spring.datasource.tomcat.max-idle=10

spring.jackson.date-format=yyyy-MM-dd HH:mm:ss
spring.jackson.date-format-exact=yyyy-MM-dd HH:mm:ss SSS
spring.jackson.time-zone=GMT+8

#rabbitmq配置
mq.env=local
#此项必须有，否则无法连接，因为如果是默认的话，则不需要
spring.rabbitmq.virtual-host=/vhost_mmr
spring.rabbitmq.host=127.0.0.1
spring.rabbitmq.port=5672
spring.rabbitmq.username=LiXiwen
spring.rabbitmq.password=178415
spring.rabbitmq.listener.concurrency=10
spring.rabbitmq.listener.max-concurrency=20
spring.rabbitmq.listener.prefetch=5
#后面三个参数主要是用于“并发量的配置”
#并发消费者的初始化值，并发消费者的最大值，每个消费者每次监听时可拉取处理的消息数量。
spring.rabbitmq.listener.transaction-size=1

########################### queue 配置 ##########################
#正常发送消息队列，文章中的：正常发送消息的交换机，正常发送消息的队列
register.delay.queue.name=${mq.env}.user.register.delay.queue
register.delay.exchange.name=${mq.env}.user.register.delay.exchange
#存放过期消息的队列，文章中的：死信交换机，死信队列
register.receive.exchange.name=${mq.env}.user.register.receive.exchange
register.receive.queue.name=${mq.env}.user.register.receive.queue

#交易记录失效时间：10s
trade.record.ttl=10000

#mybatis
mybatis.checkConfigLocation = true
mybatis.mapper-locations=classpath:mapper/*.xml

1.5.2 pom文件

引入正常的依赖即可

1.5.3 RabbitMQConfig

可以直接拿来用,注意导包别导错了

import com.softlab.logger.web.api.ConsumerController;
import org.springframework.amqp.core.*;
import org.springframework.amqp.rabbit.config.SimpleRabbitListenerContainerFactory;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.core.env.Environment;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.connection.CachingConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.amqp.SimpleRabbitListenerContainerFactoryConfigurer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.HashMap;
import java.util.Map;

@Configuration
public class RabbitmqConfig {
    private static final Logger log = LoggerFactory.getLogger(RabbitmqConfig.class);

    private final Environment env;
    private final CachingConnectionFactory connectionFactory;
    private final SimpleRabbitListenerContainerFactoryConfigurer factoryConfigurer;
    private final ConsumerController consumerController;

    @Autowired
    public RabbitmqConfig(Environment env, CachingConnectionFactory connectionFactory, SimpleRabbitListenerContainerFactoryConfigurer factoryConfigurer, ConsumerController consumerController) {
        this.env = env;
        this.connectionFactory = connectionFactory;
        this.factoryConfigurer = factoryConfigurer;
        this.consumerController = consumerController;
    }

    /**
     * 死信交换机
     * @return
     */
    @Bean
    public DirectExchange registerDelayExchange(){
        return new DirectExchange(env.getProperty("register.delay.exchange.name"));
    }

    /**
     * 死信队列
     * @return
     */
    @Bean
    public Queue registerDelayQueue() {
        Map<String, Object> params = new HashMap<>(16);
        //死信接收交换机(receiveexchange)根据路由键(routekey2)找到绑定自己的死信接收队列(receivequeue)并把消息给它
        params.put("x-dead-letter-exchange", env.getProperty("register.receive.exchange.name"));
        params.put("x-dead-letter-routing-key","receive_key");
        return new Queue(env.getProperty("register.delay.queue.name"), true,false,false,params);
    }

    /**
     * 将死信队列和死信交换机绑定。
     * @return
     */
    @Bean
    public Binding registerDelayBinding(){
        return BindingBuilder.bind(registerDelayQueue()).to(registerDelayExchange()).with("delay_key");
    }

    /**
     * 死信接收交换机
     * @return
     */
    @Bean
    public TopicExchange registerTopicExchange(){
        return new TopicExchange(env.getProperty("register.receive.exchange.name"));
    }

    /**
     * 死信接收队列
     * @return
     */
    @Bean(name = "registerQueue")
    public Queue registerQueue(){
        return new Queue(env.getProperty("register.receive.queue.name"),true);
    }

    @Bean
    public Binding registerBinding(){
        return BindingBuilder.bind(registerQueue()).to(registerTopicExchange()).with("receive_key");
    }

    /**
     * 单一消费者
     * @return
     */
    @Bean(name = "singleListenerContainer")
    public SimpleRabbitListenerContainerFactory listenerContainer(){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factory.setConnectionFactory(connectionFactory);
        factory.setMessageConverter(new Jackson2JsonMessageConverter());
        factory.setConcurrentConsumers(1);
        factory.setMaxConcurrentConsumers(1);
        factory.setPrefetchCount(1);
        factory.setTxSize(1);
        return factory;
    }

    /**
     * 多个消费者
     * @return
     */
    @Bean(name = "multiListenerContainer")
    public SimpleRabbitListenerContainerFactory multiListenerContainer(){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factoryConfigurer.configure(factory,connectionFactory);
        factory.setMessageConverter(new Jackson2JsonMessageConverter());
        factory.setAcknowledgeMode(AcknowledgeMode.NONE);
        return factory;
    }

    @Bean
    public RabbitTemplate rabbitTemplate(){
        connectionFactory.setPublisherConfirms(true);
        connectionFactory.setPublisherReturns(true);
        RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setMandatory(true);
        rabbitTemplate.setConfirmCallback(new RabbitTemplate.ConfirmCallback() {
            @Override
            public void confirm(CorrelationData correlationData, boolean ack, String cause) {
                log.info("消息发送成功:correlationData({}),ack({}),cause({})",correlationData,ack,cause);
            }
        });
        rabbitTemplate.setReturnCallback(new RabbitTemplate.ReturnCallback() {
            @Override
            public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
                log.info("消息丢失:exchange({}),route({}),replyCode({}),replyText({}),message:{}",exchange,routingKey,replyCode,replyText,message);
            }
        });
        return rabbitTemplate;
    }

1.5.4 ProduceController

import com.fasterxml.jackson.databind.ObjectMapper;
import com.softlab.logger.common.RestData;
import com.softlab.logger.common.util.JsonUtil;
import com.softlab.logger.core.model.LogVo;
import com.softlab.logger.core.model.ReceiptVo;
import com.softlab.logger.service.ProducerService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.validation.BindingResult;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*;

import java.util.HashMap;
import java.util.Map;

@CrossOrigin(origins = "*", allowCredentials = "true", allowedHeaders = "*")
@RestController
public class ProducerController {
    private static final Logger log = LoggerFactory.getLogger(ProducerController.class);

    private final ObjectMapper objectMapper;
    private final RabbitTemplate rabbitTemplate;
    private final Environment env;
    private final ProducerService producerService;

    @Autowired
    public ProducerController(ObjectMapper objectMapper, RabbitTemplate rabbitTemplate, Environment env, ProducerService producerService) {
        this.objectMapper = objectMapper;
        this.rabbitTemplate = rabbitTemplate;
        this.env = env;
        this.producerService = producerService;
    }

    @RequestMapping(value = "/add", method = RequestMethod.POST)
    public RestData add(@Validated @RequestBody ReceiptVo receiptVo, BindingResult bindingResult) {
        if (bindingResult.hasErrors()) {
            return new RestData(1, "参数错误");
        }
        try {
            producerService.insert(receiptVo);
        } catch (Exception e) {
            return new RestData(1, e.getLocalizedMessage());
        }
        return new RestData(0, "success");
    }
}

1.5.5 ProduceService && ProduceServiceImpl

import com.fasterxml.jackson.databind.ObjectMapper;
import com.softlab.logger.common.ProducerException;
import com.softlab.logger.core.mapper.OrderTradeRecordMapper;
import com.softlab.logger.core.model.*;
import com.softlab.logger.service.ProducerService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.*;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.AbstractJavaTypeMapper;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Service;

import java.util.Date;
import java.util.HashMap;
import java.util.Map;
@Service
public class ProducerServiceImpl implements ProducerService {
    private static final Logger log = LoggerFactory.getLogger(ProducerServiceImpl.class);

    private final ObjectMapper objectMapper;
    private final RabbitTemplate rabbitTemplate;
    private final Environment env;
    private final OrderTradeRecordMapper orderTradeRecordMapper;

    @Autowired
    public ProducerServiceImpl(ObjectMapper objectMapper, RabbitTemplate rabbitTemplate, Environment env, OrderTradeRecordMapper orderTradeRecordMapper) {
        this.objectMapper = objectMapper;
        this.rabbitTemplate = rabbitTemplate;
        this.env = env;
        this.orderTradeRecordMapper = orderTradeRecordMapper;
    }

    @Override
    public void createTradeRecord(OrderTradeRecordRequest recordRequest) {
        OrderTradeRecord record = new OrderTradeRecord();
        BeanUtils.copyProperties(recordRequest, record);
        record.setCreateTime(new Date());
        record.setStatus(1);
        log.info("插入状态 : " + orderTradeRecordMapper.insertSelective(record));

        final Long ttl = env.getProperty("trade.record.ttl", Long.class);
        rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
        rabbitTemplate.setExchange(env.getProperty("register.delay.exchange.name"));
        rabbitTemplate.setRoutingKey("delay_key");
        log.info("---开始发送消息---");
        rabbitTemplate.convertAndSend(record, new MessagePostProcessor() {
            @Override
            public Message postProcessMessage(Message message) throws AmqpException {
                message.getMessageProperties().setHeader(AbstractJavaTypeMapper.DEFAULT_CONTENT_CLASSID_FIELD_NAME, User.class.getName());
                message.getMessageProperties().setExpiration(ttl + "");
                return message;
            }
        });
        log.info("---生产者结束---");

    }
}

1.5.6 RabbitMQListener

import com.softlab.logger.core.mapper.OrderTradeRecordMapper;
import com.softlab.logger.core.model.OrderTradeRecord;
import com.softlab.logger.core.model.User;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessagePostProcessor;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.AbstractJavaTypeMapper;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.util.Date;
import java.util.Objects;

@Component
public class RabbitMQListener {
    private final static Logger log= LoggerFactory.getLogger(RabbitMQListener.class);

    @Autowired
    private OrderTradeRecordMapper orderTradeRecordMapper;
    @Autowired
    private Environment env;

    @Autowired
    private RabbitTemplate rabbitTemplate;

    /**
     * 如果出现异常，则重新发送消息到正常的消息发送交换机
     * @param record
     * @throws IOException
     */
    @RabbitListener(queues = "${register.receive.queue.name}",containerFactory = "singleListenerContainer")
    public void consumeMessage(@Payload OrderTradeRecord record) throws IOException {
        try {
            log.info("消费者监听交易记录信息： {} ",record);
            //TODO：表示已经到ttl了，却还没付款，则需要处理为失效
            if (Objects.equals(1,record.getStatus())){
                record.setStatus(0);
                record.setUpdateTime(new Date());
                orderTradeRecordMapper.updateByPrimaryKeySelective(record);
            }
        } catch (Exception e) {
            log.error("消息体解析 发生异常； ",e.getLocalizedMessage());

            final Long ttl = env.getProperty("trade.record.ttl", Long.class);
            rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
            rabbitTemplate.setExchange(env.getProperty("register.delay.exchange.name"));
            rabbitTemplate.setRoutingKey("delay_key");
            log.info("---开始重新发送消息---");
            rabbitTemplate.convertAndSend(record, new MessagePostProcessor() {
                @Override
                public Message postProcessMessage(Message message) throws AmqpException {
                    message.getMessageProperties().setHeader(AbstractJavaTypeMapper.DEFAULT_CONTENT_CLASSID_FIELD_NAME, User.class.getName());
                    message.getMessageProperties().setExpiration(ttl + "");
                    return message;
                }
            });

        }
    }
}

1.5.7 RestData

import com.fasterxml.jackson.annotation.JsonInclude;

public class RestData {

    private int code = 0;

    @JsonInclude(JsonInclude.Include.NON_NULL)
    private String message;

    @JsonInclude(JsonInclude.Include.NON_NULL)
    private Object data;


    public RestData(int code, String message) {
        this.code = code;
        this.message = message;
    }

    public RestData(Object data) {
        this.code = 0;
        this.data = data;
    }


    public int getCode() {
        return code;
    }

    public void setCode(int code) {
        this.code = code;
    }

    public String getMessage() {
        return message;
    }

    public void setMessage(String message) {
        this.message = message;
    }

    public Object getData() {
        return data;
    }

    public void setData(Object data) {
        this.data = data;
    }
}

1.5.8 OrderTradeRecordRequest

import javax.validation.constraints.NotNull;
import java.io.Serializable;
import java.math.BigDecimal;
public class OrderTradeRecordRequest implements Serializable {

    @NotNull
    private Integer customerId;
    @NotNull
    private Integer orderId;
    @NotNull
    private BigDecimal price;

    private Integer status=0;


    public Integer getCustomerId() {
        return customerId;
    }

    public void setCustomerId(Integer customerId) {
        this.customerId = customerId;
    }

    public Integer getOrderId() {
        return orderId;
    }

    public void setOrderId(Integer orderId) {
        this.orderId = orderId;
    }

    public BigDecimal getPrice() {
        return price;
    }

    public void setPrice(BigDecimal price) {
        this.price = price;
    }

    public Integer getStatus() {
        return status;
    }

    public void setStatus(Integer status) {
        this.status = status;
    }

    @Override
    public String toString() {
        return "OrderTradeRecordRequest{" +
                "customerId=" + customerId +
                ", orderId=" + orderId +
                ", price=" + price +
                ", status=" + status +
                '}';
    }

}

1.5.9 OrderTradeRecord

```public class OrderTradeRecord {
    private Integer id;

    private Integer customerId;

    private Integer orderId;

    private BigDecimal price;

    private Integer status=1;

    private Date createTime;

    private Date updateTime;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public Integer getCustomerId() {
        return customerId;
    }

    public void setCustomerId(Integer customerId) {
        this.customerId = customerId;
    }

    public Integer getOrderId() {
        return orderId;
    }

    public void setOrderId(Integer orderId) {
        this.orderId = orderId;
    }

    public BigDecimal getPrice() {
        return price;
    }

    public void setPrice(BigDecimal price) {
        this.price = price;
    }

    public Integer getStatus() {
        return status;
    }

    public void setStatus(Integer status) {
        this.status = status;
    }

    public Date getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Date createTime) {
        this.createTime = createTime;
    }

    public Date getUpdateTime() {
        return updateTime;
    }

    public void setUpdateTime(Date updateTime) {
        this.updateTime = updateTime;
    }

    @Override
    public String toString() {
        return "OrderTradeRecord{" +
                "id=" + id +
                ", customerId=" + customerId +
                ", orderId=" + orderId +
                ", price=" + price +
                ", status=" + status +
                ", createTime=" + createTime +
                ", updateTime=" + updateTime +
                '}';
    }
}