rocketMq 单机及集群搭建，附java连接示例
本篇文章搭建环境
centos7、jdk1.8、rocketmq 4.8.0
rocketmq官网地址：http://rocketmq.apache.org/
一、单机搭建
1.配置host
输入命令查看hostname：hostnamectl
2.vim /etc/hosts 添加配置：192.168.202.221 localhost.localdomain
3.解压rocketMQ包放到指定目录下
例如我的存放目录：/opt/rocketMQ/rocketmq-all-4.8.0-bin-release
4.创建日志存储目录例如：/opt/rocketMQ/store
5.修改启动参数，避免虚拟机中内存不够导致启动失败
/opt/rocketMQ/rocketmq-all-4.8.0-bin-release/bin 目录下的：runbroker.sh、runserver.sh
runserver.sh：
runbroker.sh：
6.启动nameserver，命令如下：
nohup sh rocketmq-all-4.8.0-bin-release/bin/mqnamesrv &
或
nohup sh rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 &
7.启动broker
nohup sh mqbroker -c rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties &
或者
nohup sh rocketmq-all-4.8.0-bin-release/bin/mqbroker -n localhost:9876 &
可以看下：/rocketmq-all-4.8.0-bin-release/logs/rocketmqlogs 目录下的namesrv.log 、broker.log 日志文件
8.使用命令模拟消息消费，可以看出是否可以正常消费
export NAMESRV_ADDR=localhost:9876
sh tools.sh org.apache.rocketmq.example.quickstart.Producer
sh tools.sh org.apache.rocketmq.example.quickstart.Consumer
二、集群搭建
集群搭建采用两主两从：
准备了两个虚拟机每个分别放一主一从
192.168.202.221 a-master、b-slave
192.168.202.222 b-master、a-slave
1.先准备好节点目录，示例：
192.168.202.221：
/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release（a-master）
/opt/rocketmq-cluster/node1/logs
/opt/rocketmq-cluster/node1/store
/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release（b-slave）
/opt/rocketmq-cluster/node1s/logs
/opt/rocketmq-cluster/node1s/store
192.168.202.222：
/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release（b-master）
/opt/rocketmq-cluster/node1/logs
/opt/rocketmq-cluster/node1/store
/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release（a-slave）
/opt/rocketmq-cluster/node1s/logs
/opt/rocketmq-cluster/node1s/store
2.同样和单机搭建一样修改各个节点runbroker.sh、runserver.sh的启动参数，避免因为内存不足而启动失败
修改主从配置文件：
cd /opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf && sed -i 's#${user.home}#/opt/rocketmq-cluster/node1#g' *.xml
cd /opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf && sed -i 's#${user.home}#/opt/rocketmq-cluster/node1s#g' *.xml
3.修改各个节点的配置文件，示例：
/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties（a-master）
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-a
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.221
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1/store/broker-a
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1/store/broker-a/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1/store/broker-a/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1/broker-a/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties（b-slave）
#所属集群名字
brokerClusterName=rocketmq-cluster
brokerName=broker-b
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-b,他的slave也叫broker-b
#0 表示 Master，>0 表示 Slave
brokerId=1
#nameServer地址，分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.221
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1s/store/broker-b-s
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1s/store/broker-b-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1s/store/broker-b-s/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1s/store/broker-b-s/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1s/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1s/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties（b-master）
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.222
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1/store/broker-b
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1/store/broker-b/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1/store/broker-b/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1/store/broker-b/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties（a-slave）
#集群名
brokerClusterName=rocketmq-cluster
brokerName=broker-a
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerId=1
#nameServer地址，分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokeIP1=192.168.202.222
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1s/store/broker-a-s
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1s/store/broker-a-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1s/store/broker-a-s/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1s/store/broker-a-s/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1s/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1s/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
4.启动 mqnamesrv
nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 & （192.168.202.221）
nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 & （192.168.202.222）
5.启动broker
192.168.202.221:
nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1/logs/broker-a.log 2>&1 &
nohup sh node1s/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1s/logs/broker-a.log 2>&1 &
192.168.202.222:
nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1/logs/broker-a.log 2>&1 &
nohup sh node1s/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1s/logs/broker-a.log 2>&1 &
6.可以尝试查看各个启动情况：ps -ef | grep rocketmq
7.查看集群列表：sh node1/rocketmq-all-4.8.0-bin-release/bin/mqadmin clusterList -n localhost:9876
三、用java连接测试
注：先启动Consumer准备消费消息，再启动Producer生产消息测试
Producer
package com.qiaojun.rocketmq;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.Message;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.producer.DefaultMQProducer;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.producer.SendResult;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.remoting.common.RemotingHelper;
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("unique_group_qiaojun_quickstart");
producer.setNamesrvAddr("192.168.202.221:9876;192.168.202.222:9876");
producer.setInstanceName("QuickStartProducer");
producer.start();
for (int i = 0; i < 5; i++) {
try {
Message msg = new Message("TopicTest_qiaojun",// topic
"TagA_qiaojun",// tag
("发送测试消息" + i).getBytes(RemotingHelper.DEFAULT_CHARSET)// body
);
SendResult sendResult = producer.send(msg);
System.out.println("发送结果："+sendResult.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
producer.shutdown();
}
}
执行示例
Consumer
package com.qiaojun.rocketmq;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.DefaultMQPushConsumer;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.Message;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.MessageExt;
import java.util.List;
public class Consumer {
public static void main(String[] args) throws Exception{
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("unique_group_qiaojun_quickstart");
// consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
consumer.setNamesrvAddr("192.168.202.221:9876;192.168.202.222:9876");
consumer.setInstanceName("QuickStartConsumer");
consumer.subscribe("TopicTest_qiaojun", "TagA_qiaojun");
//
consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
System.out.println(Thread.currentThread().getName() + " 接收新消息: " + msgs);
for (Message msg :msgs){
System.out.println("消息内容："+new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
});
// consumer.registerMessageListener((MessageListenerConcurrently) (list, consumeConcurrentlyContext) -> {
// for (MessageExt msg:list){
// System.out.println(msg.getMsgId()+"---"+new String(msg.getBody()));
// }
// return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
// });
consumer.start();
System.out.println("消费者已启动");
}
}
执行示例
