SpringBoot 2 集成 RocketMQ 4.9 收发同步、异步以及事务消息

1 摘要

RocketMQ 支持多种推送消息的方式，可根据业务需要使用。本文将介绍SpringBoot 2 集成 RocketMQ 4.9 收发同步、异步以及事务消息

SpringBoot 2 集成 RocketMQ:

SpringBoot 2.x 简易集成 RocketMQ

2 核心 Maven 依赖

./demo-rocketmq-producer/pom.xml
./demo-rocketmq-consumer/pom.xml

        
        <dependency>
            <groupId>org.apache.rocketmqgroupId>
            <artifactId>rocketmq-spring-boot-starterartifactId>
            <version>${rocketmq.spring.version}version>
        dependency>

其中 ${rocketmq.spring.version} 的版本为 2.2.1，对应的 rocketmq-client 版本为 4.9.1

3 核心代码

3.1 同步消息

同步消息使用场景: 对业务数据具有高一致性要求

同步消息程序执行逻辑: 生产者先将消息发送到 RocketMQ (即 Broker) ，Broker 接收到之后再反馈生产者 ACK 码，生产者拿到反馈后，返回消息发送成功。生产者必须等待 Broker 的反馈结果才执行下一步。

发送端(Producer)

./demo-rocketmq-producer/src/main/java/com/ljq/demo/springboot/rocketmq/producer/common/rocketmq/RocketMQProducer.java

    /**
     * 发送同步消息
     *
     * @param message
     * @return
     */
    public SendResult sendSync(String message) {
        log.info("RocketMQ producer, sync message, topic: {}, message: {}", RocketMQConst.TOPIC_SYNC, message);
        return rocketMQTemplate.syncSend(RocketMQConst.TOPIC_SYNC, message);
    }

调用方法

org.apache.rocketmq.spring.core.RocketMQTemplate#syncSend(java.lang.String, java.lang.Object)

接收端(Consumer)

./demo-rocketmq-consumer/src/main/java/com/ljq/springboot/rocketmq/consumer/common/rocketmq/RocketMQSyncConsumer.java

package com.ljq.springboot.rocketmq.consumer.common.rocketmq;

import com.ljq.springboot.rocketmq.consumer.common.constant.RocketMQConst;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Component;

/**
 * @Description: RocketMQ 同步消息消费者
 * @Author: junqiang.lu
 * @Date: 2021/12/2
 */
@Slf4j
@Component
@RocketMQMessageListener(topic = RocketMQConst.TOPIC_SYNC, consumerGroup = RocketMQConst.GROUP_CONSUMER_SYNC)
public class RocketMQSyncConsumer implements RocketMQListener<String> {


    @Override
    public void onMessage(String s) {
        log.info("rocketMQ consumer, topic:{}, message:{}", RocketMQConst.TOPIC_SYNC, s);
    }
}

3.2 异步消息

异步消息使用场景: 数据吞吐量大，保证数据最终一致性

异步消息程序执行逻辑: 生产者将消息发送给 Broker，无需等待 Broker 的反馈结果即可执行下一步程序。Broker 的反馈结果会通过异步的方式通知给生产者。

发送端(Producer)

./demo-rocketmq-producer/src/main/java/com/ljq/demo/springboot/rocketmq/producer/common/rocketmq/RocketMQProducer.java

    /**
     * 发送异步消息
     *
     * @param message
     */
    public void sendAsync(String message) {
        log.info("RocketMQ producer, async message, topic: {}, message: {}", RocketMQConst.TOPIC_ASYNC, message);
        rocketMQTemplate.asyncSend(RocketMQConst.TOPIC_ASYNC, message, new SendCallback() {
            @Override
            public void onSuccess(SendResult sendResult) {
                log.info("async message send success !!! \n topic: {}, message: {}, result: {}",
                        RocketMQConst.TOPIC_ASYNC, message, sendResult);
            }

            @Override
            public void onException(Throwable throwable) {
                log.info("async message send fail ... \n topic: {}, message: {}, error: {}", RocketMQConst.TOPIC_ASYNC,
                        message, throwable.getMessage());
            }
        });
    }

调用方法

org.apache.rocketmq.spring.core.RocketMQTemplate#asyncSend(java.lang.String, java.lang.Object, org.apache.rocketmq.client.producer.SendCallback)

接收端(Consumer)

./demo-rocketmq-consumer/src/main/java/com/ljq/springboot/rocketmq/consumer/common/rocketmq/RocketMQAsyncConsumer.java

package com.ljq.springboot.rocketmq.consumer.common.rocketmq;

import com.ljq.springboot.rocketmq.consumer.common.constant.RocketMQConst;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Component;

/**
 * @Description: RocketMQ 异步消息消费者
 * @Author: junqiang.lu
 * @Date: 2021/12/2
 */
@Slf4j
@Component
@RocketMQMessageListener(topic = RocketMQConst.TOPIC_ASYNC, consumerGroup = RocketMQConst.GROUP_CONSUMER_ASYNC)
public class RocketMQAsyncConsumer implements RocketMQListener<String> {


    @Override
    public void onMessage(String s) {
        log.info("rocketMQ consumer, topic:{}, message:{}", RocketMQConst.TOPIC_ASYNC, s);
    }
}

3.3 事务消息

事务消息使用场景: 适用于需要用到事务的地方，支持对大型事务进行拆分，通过二阶段提交，实现事务最终一致性

事务消息程序执行逻辑: 生产者向 Broker 发送消息，此时消息暂存在 Broker，并未推送给消费者，待生产者端本地事务执行成功之后再向 Broker 发送提交指令，此时 Broker 才会将消息推送给消费者。

重试机制: 当生产者端的事务提交失败后，后间隔一定的时间(默认 60 秒)，对事务状态进行检查，如果此时本地事务提交成功，则会向 Broker 发送提交指令；若本地事务仍然提交失败，则继续间隔一定时间后进行重试，直到达到最大重试次数(默认为 15 次)。

发送端(Producer)

./demo-rocketmq-producer/src/main/java/com/ljq/demo/springboot/rocketmq/producer/common/rocketmq/RocketMQProducer.java

    /**
     * 发送事务消息
     *
     * @param message
     */
    public TransactionSendResult sendTransaction(String message) {
        String transactionId = UUID.randomUUID().toString();
        log.info("{RocketMQ producer, transaction message, topic: {}, message: {}, transactionId: {}}",
                RocketMQConst.TOPIC_TRANSACTION, message, transactionId);
        Message<String> msg = MessageBuilder.withPayload(message).setHeader(RocketMQHeaders.TRANSACTION_ID,
                transactionId).build();
        return rocketMQTemplate.sendMessageInTransaction(RocketMQConst.TOPIC_TRANSACTION, msg, message);
    }

调用方法

org.apache.rocketmq.spring.core.RocketMQTemplate#sendMessageInTransaction

生产者端事务监听

./demo-rocketmq-producer/src/main/java/com/ljq/demo/springboot/rocketmq/producer/common/rocketmq/RocketMQTransactionListenerImpl.java

package com.ljq.demo.springboot.rocketmq.producer.common.rocketmq;

import com.ljq.demo.springboot.rocketmq.producer.common.constant.RocketMQConst;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionListener;
import org.apache.rocketmq.spring.core.RocketMQLocalTransactionState;
import org.apache.rocketmq.spring.support.RocketMQHeaders;
import org.springframework.messaging.Message;

import java.util.Objects;
import java.util.concurrent.ConcurrentHashMap;

/**
 * @Description: RocketMQ 事务监听器
 * @Author: junqiang.lu
 * @Date: 2021/12/6
 */
@Slf4j
@RocketMQTransactionListener()
public class RocketMQTransactionListenerImpl implements RocketMQLocalTransactionListener {

    private ConcurrentHashMap<String, String> localTransaction = new ConcurrentHashMap<>();


    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message message, Object o) {
        String transactionId = String.valueOf(message.getHeaders().get(RocketMQHeaders.TRANSACTION_ID));
        log.info("RocketMQ local transaction execute, message:{},transactionId: {}, param:{}", message.getPayload(),
                transactionId, o);

        // TODO 处理本地事务
        String transactionParam = String.valueOf(o);
        log.info("transactionParam:{}", transactionParam);
        if (transactionParam.contains(RocketMQConst.TRANSACTION_STATUS_SUCCESS)) {
            localTransaction.put(transactionId, RocketMQConst.TRANSACTION_STATUS_SUCCESS);
            log.info("local transaction handle result: {}",RocketMQConst.TRANSACTION_STATUS_SUCCESS);
            return RocketMQLocalTransactionState.COMMIT;
        }
        if (transactionParam.contains(RocketMQConst.TRANSACTION_STATUS_FAIL)) {
            localTransaction.put(transactionId, RocketMQConst.TRANSACTION_STATUS_FAIL);
            log.info("local transaction handle result: {}",RocketMQConst.TRANSACTION_STATUS_FAIL);
            return RocketMQLocalTransactionState.ROLLBACK;
        }
        log.info("transaction status unknown");
        return RocketMQLocalTransactionState.UNKNOWN;
    }

    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message message) {
        String transactionId = String.valueOf(message.getHeaders().get(RocketMQHeaders.TRANSACTION_ID));
        log.info("RocketMQ local transaction check, message:{},transactionId: {}", message.getPayload(), transactionId);

        // TODO 校验本地事务
        String transactionResult = localTransaction.getOrDefault(transactionId, "");
        if (Objects.equals(transactionResult, RocketMQConst.TRANSACTION_STATUS_SUCCESS)) {
            return RocketMQLocalTransactionState.COMMIT;
        }
        if (Objects.equals(transactionResult, RocketMQConst.TRANSACTION_STATUS_FAIL)) {
            return RocketMQLocalTransactionState.ROLLBACK;
        }
        return RocketMQLocalTransactionState.UNKNOWN;
    }
}

接收端(Consumer)

./demo-rocketmq-consumer/src/main/java/com/ljq/springboot/rocketmq/consumer/common/rocketmq/RocketMQTransactionConsumer.java

package com.ljq.springboot.rocketmq.consumer.common.rocketmq;

import com.ljq.springboot.rocketmq.consumer.common.constant.RocketMQConst;
import lombok.extern.slf4j.Slf4j;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.stereotype.Component;

/**
 * @Description: RocketMQ 事务消息消费者
 * @Author: junqiang.lu
 * @Date: 2021/12/6
 */
@Slf4j
@Component
@RocketMQMessageListener(topic = RocketMQConst.TOPIC_TRANSACTION, consumerGroup = RocketMQConst.GROUP_CONSUMER_TRANSACTION)
public class RocketMQTransactionConsumer implements RocketMQListener<String> {

    @Override
    public void onMessage(String s) {
        log.info("rocketMQ consumer, topic:{}, message:{}", RocketMQConst.TOPIC_TRANSACTION, s);
    }
}

4 测试

4.1 同步消息

GET 方式请求接口

http://127.0.0.1:8750/api/rocketmq/producer/send/sync?message=sync1638863656

生产者端日志

2021-12-07 15:54:16 | INFO  | http-nio-8750-exec-1 | com.ljq.demo.springboot.rocketmq.producer.controller.RocketMQProducerController 60| request param: sync1638863656
2021-12-07 15:54:16 | INFO  | http-nio-8750-exec-1 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQProducer 56| RocketMQ producer, sync message, topic: rocketmq_topic_sync, message: sync1638863656
2021-12-07 15:54:16 | DEBUG | http-nio-8750-exec-1 | org.apache.rocketmq.spring.core.RocketMQTemplate 561| send message cost: 209 ms, msgId:7F00000151D818B4AAC2224FD39C0000
2021-12-07 15:54:16 | INFO  | http-nio-8750-exec-1 | com.ljq.demo.springboot.rocketmq.producer.controller.RocketMQProducerController 62| sendResult: SendResult [sendStatus=SEND_OK, msgId=7F00000151D818B4AAC2224FD39C0000, offsetMsgId=B63D13EF00002A9F000000000002B306, messageQueue=MessageQueue [topic=rocketmq_topic_sync, brokerName=broker-a, queueId=0], queueOffset=1]

消费者端日志

2021-12-07 15:54:16 | DEBUG | ConsumeMessageThread_1 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 356| received msg: MessageExt [brokerName=broker-a, queueId=0, storeSize=289, queueOffset=1, sysFlag=0, bornTimestamp=1638863656862, bornHost=/producer_public_ip:25530, storeTimestamp=1638863656093, storeHost=/broker_public_ip:10911, msgId=B63D13EF00002A9F000000000002B306, commitLogOffset=176902, bodyCRC=1374578763, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='rocketmq_topic_sync', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=2, CONSUME_START_TIME=1638863656905, id=72c62515-2696-c417-b471-d0e743b14700, UNIQ_KEY=7F00000151D818B4AAC2224FD39C0000, CLUSTER=DefaultCluster, contentType=text/plain;charset=UTF-8, timestamp=1638863656680}, body=[115, 121, 110, 99, 49, 54, 51, 56, 56, 54, 51, 54, 53, 54], transactionId='null'}]
2021-12-07 15:54:16 | INFO  | ConsumeMessageThread_1 | com.ljq.springboot.rocketmq.consumer.common.rocketmq.RocketMQSyncConsumer 22| rocketMQ consumer, topic:rocketmq_topic_sync, message:sync1638863656
2021-12-07 15:54:17 | DEBUG | ConsumeMessageThread_1 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 361| consume 7F00000151D818B4AAC2224FD39C0000 cost: 0 ms

4.2 异步消息

GET 方式请求接口

http://127.0.0.1:8750/api/rocketmq/producer/send/async?message=async1638863774

生产者端日志

2021-12-07 15:56:13 | INFO  | http-nio-8750-exec-4 | com.ljq.demo.springboot.rocketmq.producer.controller.RocketMQProducerController 74| request param: async1638863774
2021-12-07 15:56:13 | INFO  | http-nio-8750-exec-4 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQProducer 66| RocketMQ producer, async message, topic: rocketmq_topic_async, message: async1638863774
2021-12-07 15:56:13 | INFO  | NettyClientPublicExecutor_2 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQProducer 70| async message send success !!! 
 topic: rocketmq_topic_async, message: async1638863774, result: SendResult [sendStatus=SEND_OK, msgId=7F00000151D818B4AAC222519C4E0002, offsetMsgId=B63D13EF00002A9F000000000002B54A, messageQueue=MessageQueue [topic=rocketmq_topic_async, brokerName=broker-a, queueId=1], queueOffset=3]

消费者端日志

2021-12-07 15:56:13 | DEBUG | ConsumeMessageThread_2 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 356| received msg: MessageExt [brokerName=broker-a, queueId=1, storeSize=291, queueOffset=3, sysFlag=0, bornTimestamp=1638863773774, bornHost=/producer_public_ip:25530, storeTimestamp=1638863773052, storeHost=/broker_public_ip:10911, msgId=B63D13EF00002A9F000000000002B54A, commitLogOffset=177482, bodyCRC=1908008441, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='rocketmq_topic_async', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=4, CONSUME_START_TIME=1638863773850, id=2ba05a08-1e77-2307-3ead-ca4f10b9087e, UNIQ_KEY=7F00000151D818B4AAC222519C4E0002, CLUSTER=DefaultCluster, contentType=text/plain;charset=UTF-8, timestamp=1638863773773}, body=[97, 115, 121, 110, 99, 49, 54, 51, 56, 56, 54, 51, 55, 55, 52], transactionId='null'}]
2021-12-07 15:56:13 | INFO  | ConsumeMessageThread_2 | com.ljq.springboot.rocketmq.consumer.common.rocketmq.RocketMQAsyncConsumer 22| rocketMQ consumer, topic:rocketmq_topic_async, message:async1638863774
2021-12-07 15:56:13 | DEBUG | ConsumeMessageThread_2 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 361| consume 7F00000151D818B4AAC222519C4E0002 cost: 0 ms

4.3 事务消息

GET 方式请求接口

http://127.0.0.1:8750/api/rocketmq/producer/send/transaction?message=transaction_success1638863846

生产者端日志

2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.controller.RocketMQProducerController 87| request param: transaction_success1638863846
2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQProducer 89| {RocketMQ producer, transaction message, topic: rocketmq_topic_transaction, message: transaction_success1638863846, transactionId: bd81c15f-cea8-4301-93ff-2b67d9ebf777}
2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQTransactionListenerImpl 29| RocketMQ local transaction execute, message:[116, 114, 97, 110, 115, 97, 99, 116, 105, 111, 110, 95, 115, 117, 99, 99, 101, 115, 115, 49, 54, 51, 56, 56, 54, 51, 56, 52, 54],transactionId: bd81c15f-cea8-4301-93ff-2b67d9ebf777, param:transaction_success1638863846
2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQTransactionListenerImpl 34| transactionParam:transaction_success1638863846
2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.common.rocketmq.RocketMQTransactionListenerImpl 37| local transaction handle result: success
2021-12-07 15:57:25 | INFO  | http-nio-8750-exec-6 | com.ljq.demo.springboot.rocketmq.producer.controller.RocketMQProducerController 89| transactionSendResult: SendResult [sendStatus=SEND_OK, msgId=7F00000151D818B4AAC22252B5200003, offsetMsgId=null, messageQueue=MessageQueue [topic=rocketmq_topic_transaction, brokerName=broker-a, queueId=3], queueOffset=290]

消费者端日志

2021-12-07 15:57:25 | DEBUG | ConsumeMessageThread_1 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 356| received msg: MessageExt [brokerName=broker-a, queueId=3, storeSize=468, queueOffset=7, sysFlag=8, bornTimestamp=1638863845664, bornHost=/producer_public_ip:25530, storeTimestamp=1638863845020, storeHost=/broker_public_ip:10911, msgId=B63D13EF00002A9F000000000002B83F, commitLogOffset=178239, bodyCRC=537187155, reconsumeTimes=0, preparedTransactionOffset=177773, toString()=Message{topic='rocketmq_topic_transaction', flag=0, properties={TRAN_MSG=true, CONSUME_START_TIME=1638863845813, MIN_OFFSET=0, REAL_TOPIC=rocketmq_topic_transaction, MAX_OFFSET=8, id=cd55c971-6d35-5db1-d124-4ed96bfc5180, UNIQ_KEY=7F00000151D818B4AAC22252B5200003, CLUSTER=DefaultCluster, TRANSACTION_ID=bd81c15f-cea8-4301-93ff-2b67d9ebf777, contentType=text/plain;charset=UTF-8, PGROUP=rocketmq-producer-group, WAIT=false, timestamp=1638863845599, REAL_QID=3}, body=[116, 114, 97, 110, 115, 97, 99, 116, 105, 111, 110, 95, 115, 117, 99, 99, 101, 115, 115, 49, 54, 51, 56, 56, 54, 51, 56, 52, 54], transactionId='7F00000151D818B4AAC22252B5200003'}]
2021-12-07 15:57:25 | INFO  | ConsumeMessageThread_1 | com.ljq.springboot.rocketmq.consumer.common.rocketmq.RocketMQTransactionConsumer 21| rocketMQ consumer, topic:rocketmq_topic_transaction, message:transaction_success1638863846
2021-12-07 15:57:25 | DEBUG | ConsumeMessageThread_1 | org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer 361| consume 7F00000151D818B4AAC22252B5200003 cost: 0 ms

5 注意事项

• 事务消息: RocketMQ 在事务消息处理时，从 executeLocalTransaction 方法中获取到的消息体为二进制数据/byte 数组，并非原始数据，在发送事务消息的时候，也可以通过 param 参数将传递业务数据，org.apache.rocketmq.spring.core.RocketMQTemplate#sendMessageInTransaction 方法的最后一个参数即为 executeLocalTransaction 方法消息之外的入参
• 事务消息参数配置: 当事务消息状态为 unkonwn 时,RocketMQ 会对消息进行事务状态检查,当达到最大检查次数并且事务仍然为unknown 时,消息就会自动回滚

// 事务消息检查最大次数
private int transactionCheckMax = 15;
// 事务消息检查时间间隔
private long transactionCheckInterval = 60 * 1000;

配置文件

./conf/broker.conf

如需改动则需要手动添加配置

6 推荐参考资料

RocketMQ 接口和参数说明-Alibaba

Apache RocketMQ with Spring Boot-Baeldung

RocketMQ-Spring 官方示例

7 Github 源码

Gtihub 源码地址 : https://github.com/Flying9001/springBootDemo

个人公众号:404Code,分享半个互联网人的技术与思考，感兴趣的可以关注.