rocketMq 单机及集群搭建,附java连接示例

本篇文章搭建环境

centos7、jdk1.8、rocketmq 4.8.0

rocketmq官网地址:http://rocketmq.apache.org/

一、单机搭建

 

1.配置host

输入命令查看hostname:hostnamectl

rocketMq 单机及集群搭建,附java连接示例_第1张图片

 

2.vim /etc/hosts 添加配置:192.168.202.221 localhost.localdomain

 

3.解压rocketMQ包放到指定目录下

例如我的存放目录:/opt/rocketMQ/rocketmq-all-4.8.0-bin-release

 

4.创建日志存储目录例如:/opt/rocketMQ/store

 

5.修改启动参数,避免虚拟机中内存不够导致启动失败

/opt/rocketMQ/rocketmq-all-4.8.0-bin-release/bin 目录下的:runbroker.shrunserver.sh

runserver.sh:

rocketMq 单机及集群搭建,附java连接示例_第2张图片

runbroker.sh:

rocketMq 单机及集群搭建,附java连接示例_第3张图片

 

6.启动nameserver,命令如下:

nohup sh rocketmq-all-4.8.0-bin-release/bin/mqnamesrv &

nohup sh rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 &

 

7.启动broker

 

nohup sh mqbroker -c rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties &

或者

nohup sh rocketmq-all-4.8.0-bin-release/bin/mqbroker -n localhost:9876 &

可以看下:/rocketmq-all-4.8.0-bin-release/logs/rocketmqlogs 目录下的namesrv.logbroker.log 日志文件

 

8.使用命令模拟消息消费,可以看出是否可以正常消费

export NAMESRV_ADDR=localhost:9876

sh tools.sh org.apache.rocketmq.example.quickstart.Producer  

sh tools.sh org.apache.rocketmq.example.quickstart.Consumer

 

二、集群搭建

集群搭建采用两主两从:

准备了两个虚拟机每个分别放一主一从

192.168.202.221   a-master、b-slave

192.168.202.222   b-master、a-slave

 

1.先准备好节点目录,示例:

192.168.202.221:

/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release(a-master)

/opt/rocketmq-cluster/node1/logs

/opt/rocketmq-cluster/node1/store

/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release(b-slave)

/opt/rocketmq-cluster/node1s/logs

/opt/rocketmq-cluster/node1s/store

192.168.202.222:

/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release(b-master)

/opt/rocketmq-cluster/node1/logs

/opt/rocketmq-cluster/node1/store

/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release(a-slave)

/opt/rocketmq-cluster/node1s/logs

/opt/rocketmq-cluster/node1s/store

 

2.同样和单机搭建一样修改各个节点runbroker.shrunserver.sh的启动参数,避免因为内存不足而启动失败

修改主从配置文件:

cd /opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf && sed -i 's#${user.home}#/opt/rocketmq-cluster/node1#g' *.xml

cd /opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf && sed -i 's#${user.home}#/opt/rocketmq-cluster/node1s#g' *.xml

3.修改各个节点的配置文件,示例:

/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties(a-master)

#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-a
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.221
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1/store/broker-a
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1/store/broker-a/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1/store/broker-a/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1/broker-a/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

 

/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties(b-slave)

#所属集群名字
brokerClusterName=rocketmq-cluster
brokerName=broker-b
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-b,他的slave也叫broker-b
#0 表示 Master,>0 表示 Slave
brokerId=1
#nameServer地址,分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.221
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1s/store/broker-b-s
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1s/store/broker-b-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1s/store/broker-b-s/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1s/store/broker-b-s/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1s/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1s/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

/opt/rocketmq-cluster/node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties(b-master)

#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-b
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokerIP1=192.168.202.222
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1/store/broker-b
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1/store/broker-b/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1/store/broker-b/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1/store/broker-b/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

 

/opt/rocketmq-cluster/node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties(a-slave)

#集群名
brokerClusterName=rocketmq-cluster
brokerName=broker-a
#broker名字,名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerId=1
#nameServer地址,分号分割
namesrvAddr=192.168.202.221:9876;192.168.202.222:9876
brokeIP1=192.168.202.222
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/opt/rocketmq-cluster/node1s/store/broker-a-s
#commitLog 存储路径
storePathCommitLog=/opt/rocketmq-cluster/node1s/store/broker-a-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/opt/rocketmq-cluster/node1s/store/broker-a-s/consumequeue
#消息索引存储路径
storePathIndex=/opt/rocketmq-cluster/node1s/store/broker-a-s/index
#checkpoint 文件存储路径
storeCheckpoint=/opt/rocketmq-cluster/node1s/store/checkpoint
#abort 文件存储路径
abortFile=/opt/rocketmq-cluster/node1s/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

 

4.启动 mqnamesrv

nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 & (192.168.202.221)

nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqnamesrv > /dev/null 2>&1 & (192.168.202.222)

 

5.启动broker

192.168.202.221:

nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1/logs/broker-a.log 2>&1 &

nohup sh node1s/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1s/logs/broker-a.log 2>&1 &

192.168.202.222:

nohup sh node1/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1/logs/broker-a.log 2>&1 &

nohup sh node1s/rocketmq-all-4.8.0-bin-release/bin/mqbroker -c node1s/rocketmq-all-4.8.0-bin-release/conf/2m-2s-async/broker-a.properties >node1s/logs/broker-a.log 2>&1 &

 

6.可以尝试查看各个启动情况:ps -ef | grep rocketmq

rocketMq 单机及集群搭建,附java连接示例_第4张图片

 

7.查看集群列表:sh node1/rocketmq-all-4.8.0-bin-release/bin/mqadmin clusterList -n localhost:9876

三、用java连接测试

注:先启动Consumer准备消费消息,再启动Producer生产消息测试

Producer

package com.qiaojun.rocketmq;

import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.Message;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.producer.DefaultMQProducer;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.producer.SendResult;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.remoting.common.RemotingHelper;


public class Producer {
    public static void main(String[] args) throws Exception {
        DefaultMQProducer producer = new DefaultMQProducer("unique_group_qiaojun_quickstart");
        producer.setNamesrvAddr("192.168.202.221:9876;192.168.202.222:9876");

        producer.setInstanceName("QuickStartProducer");
        producer.start();

        for (int i = 0; i < 5; i++) {
            try {
                Message msg = new Message("TopicTest_qiaojun",// topic
                        "TagA_qiaojun",// tag
                        ("发送测试消息" + i).getBytes(RemotingHelper.DEFAULT_CHARSET)// body
                );
                SendResult sendResult = producer.send(msg);

                System.out.println("发送结果:"+sendResult.toString());
            } catch (Exception e) {
                e.printStackTrace();
            }
        }

        producer.shutdown();

    }
}

执行示例

 rocketMq 单机及集群搭建,附java连接示例_第5张图片

Consumer 

package com.qiaojun.rocketmq;

import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.DefaultMQPushConsumer;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.Message;
import com.aliyun.openservices.shade.com.alibaba.rocketmq.common.message.MessageExt;

import java.util.List;

public class Consumer {
    public static void main(String[] args) throws Exception{
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("unique_group_qiaojun_quickstart");

//        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);

        consumer.setNamesrvAddr("192.168.202.221:9876;192.168.202.222:9876");
        consumer.setInstanceName("QuickStartConsumer");

        consumer.subscribe("TopicTest_qiaojun", "TagA_qiaojun");

//
        consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
            System.out.println(Thread.currentThread().getName() + " 接收新消息: " + msgs);
            for (Message msg :msgs){
                System.out.println("消息内容:"+new String(msg.getBody()));
            }
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
        });

//        consumer.registerMessageListener((MessageListenerConcurrently) (list, consumeConcurrentlyContext) -> {
//            for (MessageExt msg:list){
//                System.out.println(msg.getMsgId()+"---"+new String(msg.getBody()));
//            }
//            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
//        });
        consumer.start();
        System.out.println("消费者已启动");
    }
}

执行示例

rocketMq 单机及集群搭建,附java连接示例_第6张图片

你可能感兴趣的:(java,rocketmq)