RocketMQ IPV6集群部署/单机部署
[原创] 转载请注明！
RocketMQ IPV6集群部署/单机部署
-
- 环境
- 问题
- 集群部署RocketMQ
-
- 1. 下载安装包
- 2. 解压
- 3. 修改/覆盖配置文件 [同步复制策略|异步刷盘策略]
- 4. 启动/停止 服务
- 5. 代码测试
-
- 5.1 生产消息
- 5.2 消费消息
IPV6普及程度越来越高，前段时间公司要求全部系统全链路IPV6改造，nginx、keepalived等资料网上还比较多，RockeMQ IPV6改造资料很少，这篇文章主要写部署相关的内容，简单测试的一点代码。
环境
- Centos 7.8 [已配置好IPV4/IPV6地址]，如有您服务器未配置IPV6，请自行百度配置好后再来看此文档。
- JDK1.8
- RocketMQ 4.9.4
问题
RocketMQ官方文档写的4.6.0版本就已经支持IPV6
文档地址：https://rocketmq.apache.org/release_notes/release-notes-4.6.0/
公司之前试用的RocketMQ 4.7.1，配置IPV6后，报错如下：
2022-09-02 18:13:51.388 ERROR 10376 --- [nio-8080-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.rocketmq.client.exception.MQClientException: Send [3] times, still failed, cost [32]ms, Topic: TOPIC_1, BrokersSent: [broker-b, broker-a, broker-b]
See http://rocketmq.apache.org/docs/faq/ for further details.] with root cause
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 1 DESC: java.nio.BufferOverflowException, java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:221)
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at org.apache.rocketmq.client.impl.MQClientAPIImpl.processSendResponse(MQClientAPIImpl.java:665) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:505) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:487) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:431) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:854) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:584) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1289) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:325) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at com.sai.rocketmq.demo.service.MqProducerService.send(MqProducerService.java:45) ~[classes!/:1.0.0]
at com.sai.rocketmq.demo.controller.SuccessController.send(SuccessController.java:30) ~[classes!/:1.0.0]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
......
2022-09-02 18:14:17.474 ERROR 10376 --- [io-8080-exec-10] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/] with root cause
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while
For more information, please visit the url, http://rocketmq.apache.org/docs/faq/
at org.apache.rocketmq.client.impl.MQClientAPIImpl.processSendResponse(MQClientAPIImpl.java:665) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessageSync(MQClientAPIImpl.java:505) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:487) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.MQClientAPIImpl.sendMessage(MQClientAPIImpl.java:431) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendKernelImpl(DefaultMQProducerImpl.java:854) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl(DefaultMQProducerImpl.java:584) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1343) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1289) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:325) ~[rocketmq-client-4.7.1.jar!/:4.7.1]
at com.sai.rocketmq.demo.service.MqProducerService.send(MqProducerService.java:45) ~[classes!/:1.0.0]
at com.sai.rocketmq.demo.controller.SuccessController.send(SuccessController.java:30) ~[classes!/:1.0.0]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-5.3.13.jar!/:5.3.13]
......
解决方案：升级RocketMQ 4.9.4版本，配置IPV6正常使用。
集群部署RocketMQ
双主双备
| hostname | IPV6 | 主broker | 备broker |
|---|---|---|---|
| mq1 | 2409:8a3c:4d9:6001::1001 | broker-a | broker-b-s |
| mq2 | 2409:8a3c:4d9:6001::1002 | broker-b | broker-a-s |
1. 下载安装包
官网：https://rocketmq.apache.org/dowloading/releases/
这里我下载的版本是：Binary: rocketmq-all-4.9.4-bin-release.zip [PGP] [SHA512]
2. 解压
unzip rocketmq-all-4.9.4-bin-release -d /home/xx
3. 修改/覆盖配置文件 [同步复制策略|异步刷盘策略]
# PS：复制策略和刷盘策略可根据需求自行修改
# ----------------------- mq1机器 ------------------------------
# 3.1 mq1机器[2409:8a3c:4d9:6001::1001] nameserver配置 conf/2m-2s-async/namesrv.properties
listenPort=12011
# 3.2 mq1机器[2409:8a3c:4d9:6001::1001] 主broker配置 conf/2m-2s-async/broker-a.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
# nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12012
haListenPort=12013
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-m
storePathCommitLog=/home/xx/rocketmq/store-m/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-m/consumequeue
storePathIndex=/home/xx/rocketmq/store-m/index
storeCheckpoint=/home/xx/rocketmq/store-m/checkpoint
abortFile=/home/xx/rocketmq/store-m/abort
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
# 强制指定本机IP，需要根据每台机器进行修改。官方介绍可为空，系统默认自动识别，但多网卡时IP地址可能读取错误
brokerIP1=2409:8a3c:4d9:6001::1001
# 3.3 mq1机器[2409:8a3c:4d9:6001::1001] 备broker配置 conf/2m-2s-async/broker-b-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12022
haListenPort=12023
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-s
storePathCommitLog=/home/xx/rocketmq/store-s/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-s/consumequeue
storePathIndex=/home/xx/rocketmq/store-s/index
storeCheckpoint=/home/xx/rocketmq/store-s/checkpoint
abortFile=/home/xx/rocketmq/store-s/abort
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#强制指定本机IP，需要根据每台机器进行修改。官方介绍可为空，系统默认自动识别，但多网卡时IP地址可能读取错误
brokerIP1=2409:8a3c:4d9:6001::1001
# ---------------------- mq2机器 -------------------------------
# 3.4 mq2机器[2409:8a3c:4d9:6001::1002] nameserver配置 conf/2m-2s-async/namesrv.properties
listenPort=12011
# 3.5 mq2机器[2409:8a3c:4d9:6001::1002] 主broker配置 conf/2m-2s-async/broker-b.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12012
haListenPort=12013
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/xx/rocketmq/store-m
storePathCommitLog=/home/xx/rocketmq/store-m/commitlog
storePathConsumeQueue=/home/xx/rocketmq/store-m/consumequeue
storePathIndex=/home/xx/rocketmq/store-m/index
storeCheckpoint=/home/xx/rocketmq/store-m/checkpoint
abortFile=/home/xx/rocketmq/store-m/abort
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
#多网卡可以强制指定本机IP，需要根据每台机器进行修改。为空，系统默认自动识别
brokerIP1=2409:8a3c:4d9:6001::1002
# 3.6 mq2机器[2409:8a3c:4d9:6001::1002] 备broker配置 conf/2m-2s-async/broker-a-s.properties
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
#nameServer地址，分号分割
namesrvAddr=[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011
listenPort=12022
haListenPort=12023
deleteWhen=04
fileReservedTime=48
storePathRootDir=/home/np/rocketmq/store-s
storePathCommitLog=/home/np/rocketmq/store-s/commitlog
storePathConsumeQueue=/home/np/rocketmq/store-s/consumequeue
storePathIndex=/home/np/rocketmq/store-s/index
storeCheckpoint=/home/np/rocketmq/store-s/checkpoint
abortFile=/home/np/rocketmq/store-s/abort
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#多网卡可以强制指定本机IP，需要根据每台机器进行修改。为空，系统默认自动识别
brokerIP1=2409:8a3c:4d9:6001::1002
4. 启动/停止 服务
PS 切记！禁止直接kill broker进程！如果想停止mq服务，可以使用命令停止！
# 启动：先启动nameserver集群，再启动master-broker，最后启动slave-broker，具体操作如下：
# 1. 启动mq1和mq2两台虚拟机的 nameserver 服务
# 进入到rocketmq的bin目录
> sh mqnamesrv -c conf/2m-2s-async/namesrv.properties
# 2. 启动mq1和mq2两台虚拟机的 主broker 服务
# 2.1 启动mq1 主broker
> sh mqbroker -c conf/2m-2s-async/broker-a.properties
# 2.2 启动mq2 主broker
> sh mqbroker -c conf/2m-2s-async/broker-b.properties
# 3. 启动mq1和mq2两台虚拟机的 备broker 服务
# 2.1 启动mq1 备broker
> sh mqbroker -c conf/2m-2s-async/broker-b-s.properties
# 2.2 启动mq2 备broker
> sh mqbroker -c conf/2m-2s-async/broker-a-s.properties
# 停止服务，先停 broker服务，再停 nameserver服务
> sh mqshutdown broker
> sh mqshutdown namesrv
5. 代码测试
POM文件引入：
<dependency>
<groupId>org.apache.rocketmqgroupId>
<artifactId>rocketmq-clientartifactId>
<version>4.9.4version>
dependency>
5.1 生产消息
@SneakyThrows
public static void main(String[] args) {
DefaultMQProducer producer = new DefaultMQProducer("PRODUCER_TOPIC_1_GROUP");
producer.setNamesrvAddr("[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011");
producer.start();
Message msg = new Message("TOPIC_1", "Hello RocketMQ!".getBytes());
SendResult result = producer.send(msg);
System.out.println("result = " + result);
TimeUnit.SECONDS.sleep(1);
producer.shutdown();
}
5.2 消费消息
@SneakyThrows
public static void main(String[] args) {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("CONSUMER_TOPIC_1_GROUP");
consumer.setNamesrvAddr("[2409:8a3c:4d9:6001::1001]:12011;[2409:8a3c:4d9:6001::1002]:12011");
consumer.subscribe("TOPIC_1", "*");
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, consumeConcurrentlyContext) -> {
for (MessageExt msg : msgs) {
String str = new String(msg.getBody());
System.out.println(String.format("WANGZZ >>>>>> MSGID:%s | body: %s", msg.getMsgId(), str));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
});
consumer.start();
}