分布式消息队列RocketMQ部署与监控
一、RocketMQ简介
==========================================================================================
RocketMQ是一款分布式、队列模型的消息中间件，具有以下特点：
1、支持严格的消息顺序；
2、支持Topic与Queue两种模式；
3、亿级消息堆积能力；
4、比较友好的分布式特性；
5、同时支持Push与Pull方式消费消息；
更多介绍可以参考以下链接：
https://github.com/alibaba/RocketMQ/wiki
http://blog.csdn.net/a19881029/article/details/34446629
http://singlefly.blog.51cto.com/4658189/1368579
本文采用双Master的架构模式。
==========================================================================================
二、服务器分布
==========================================================================================
1、相关说明
| IP地址 |
主机名 |
机型 |
角色 |
架构模式 |
| 10.217.121.123 |
rocketmq-master1 |
128G内存 |
nameserver、brokerserver |
Master1（双Master模式） |
| 10.217.122.31 |
rocketmq-master2 |
128G内存 |
nameserver、brokerserver |
Master2（双Master模式） |
2、hosts信息添加
# vim /etc/hosts
10.217.121.123 mqnameserver1
10.217.122.31 mqnameserver2
10.217.121.123 rocketmq-master1
10.217.122.31 rocketmq-master2
3、系统环境
CentOS 6.3
4、总体架构
==========================================================================================
三、RocketMQ安装与配置
==========================================================================================
1、JDK安装
http://download.oracle.com/otn-pub/java/jdk/7u65-b17/jdk-7u65-linux-x64.tar.gz
# tar xvzf jdk-7u65-linux-x64.gz -C /usr/local
# rm -f jdk-7u65-linux-x64.gz
# cd /usr/local && ln -s jdk1.7.0_65 jdk
# cd /usr/local/bin && ln -s /usr/local/jdk/bin/java
2、RocketMQ安装
https://github.com/alibaba/RocketMQ/releases
# tar xvzf alibaba-rocketmq-3.1.7.tar.gz -C /usr/local
# mv alibaba-rocketmq alibaba-rocketmq-3.1.7
# ln -s alibaba-rocketmq-3.1.7 rocketmq
# rm alibaba-rocketmq-3.1.7.tar.gz
# ll /usr/local
3、环境变量设置
# vim /etc/profile
export PATH=$PATH:/usr/local/bin
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export ROCKETMQ_HOME=/usr/local/rocketmq
export PATH=$PATH::$ROCKETMQ_HOME/bin
# source /etc/profile
4、主机名设置
(1)、Master1服务器
# vim /etc/hosts
10.217.121.123 mqnameserver1
10.217.122.31 mqnameserver2
10.217.121.123 rocketmq-master1
10.217.122.31 rocketmq-master2
# sed -i '/HOSTNAME/d' /etc/sysconfig/network
# echo 'HOSTNAME=rocketmq-master1' >> /etc/sysconfig/network
# hostname rocketmq-master1
(2)、Master2服务器
# vim /etc/hosts
10.217.121.123 mqnameserver1
10.217.122.31 mqnameserver2
10.217.121.123 rocketmq-master1
10.217.122.31 rocketmq-master2
# sed -i '/HOSTNAME/d' /etc/sysconfig/network
# echo 'HOSTNAME=rocketmq-master2' >> /etc/sysconfig/network
# hostname rocketmq-master2
5、RocketMQ配置
(1)、Master1服务器
# vim /usr/local/rocketmq/conf/2m-noslave/broker-a.properties
brokerClusterName=AdpMqCluster
brokerName=broker-a
brokerId=0
namesrvAddr=mqnameserver1:9876;mqnameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
storePathCommitLog=/data/rocketmq/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
(2)、Master2服务器
# vim /usr/local/rocketmq/conf/2m-noslave/broker-b.properties
brokerClusterName=AdpMqCluster
brokerName=broker-b
brokerId=1
namesrvAddr=mqnameserver1:9876;mqnameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=50000000
destroyMapedFileIntervalForcibly=120000
redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
storePathCommitLog=/data/rocketmq/store/commitlog
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
checkTransactionMessageEnable=false
sendMessageThreadPoolNums=128
pullMessageThreadPoolNums=128
(3)、runbroker.sh参数调整
runbroker.sh需要根据内存大小进行适当地调整
JAVA_OPT_1="-server-Xms80g -Xmx80g -Xmn20g -XX:PermSize=1g -XX:MaxPermSize=1g"
6、服务启动
(1)、启动NameServer【master1、master2】
# cd /usr/local/rocketmq/bin
# nohup sh mqnamesrv &
(2)、启动BrokerServer A【master1】
# cd /usr/local/rocketmq/bin
# nohup sh mqbroker -c ../conf/2m-noslave/broker-a.properties &
(3)、启动BrokerServer B【master2】
# cd /usr/local/rocketmq/bin
# nohup sh mqbroker -c ../conf/2m-noslave/broker-b.properties &
# netstat -ntlp
(4). 启动安装启动后台管理界面：
安装tomcat，然后将rocketmq_console(监控mq的管理界面，放tomcat下启动就OK).tar部署到tomcat下启动即可。
==========================================================================================
四、RocketMQ服务健康监控
==========================================================================================
1、依赖组件安装
# yum -y install nmap
2、nameserver监控
# vim /data/scripts/check_nameserver_health.sh
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
#!/bin/sh
PATH=
/sbin
:
/bin
:
/usr/sbin
:
/usr/bin
:
/usr/local/bin
:
/usr/local/sbin
SRV_PORT=
"9876"
## 端口号
SRV_PROT=
"tcp"
## 协议类型
SRV_NAME=
"rocketmq_nameserver"
## 服务名
## 是否已正确扫描
SCAN_FLAG=0
for
ETH_NAME
in
`
/sbin/ifconfig
|
grep
eth1 |
awk
'{print $1}'
`
do
ETH1_ADDR=`
/sbin/ifconfig
${ETH_NAME} |
awk
-F
':'
'/inet addr/{print $2}'
|
sed
's/[a-zA-Z ]//g'
`
[[ -z
"${ETH1_ADDR}"
]] &&
continue
||
break
done
TMP_SRV_PROT=`
echo
${SRV_PROT} |
tr
'[A-Z]'
'[a-z]'
`
if
[[
"${TMP_SRV_PROT}"
==
"tcp"
]];
then
PROT_OPT=
"S"
elif
[[
"${TMP_SRV_PROT}"
==
"udp"
]];
then
PROT_OPT=
"U"
else
echo
"未知的协议类型！"
&& exit1
fi
## 最多扫描3次，成功一次即可，以避免网络抖动而导致误判
for
((i=0; i<3; i++));
do
RETVAL=`
/usr/bin/nmap
-n -s${PROT_OPT} -p ${SRV_PORT} ${ETH1_ADDR} |
grep
open
`
[[ -n
"${RETVAL}"
]] && SCAN_FLAG=1;
break
||
sleep
10
done
if
[[ ${SCAN_FLAG} -
ne
1 ]];
then
[[ -n `
ps
aux |
grep
java |
grep
namesrv` ]] &&
kill
-9 `
ps
aux |
grep
java |
awk
'/namesrv/{print $2}'
`
cd
/usr/local/rocketmq/bin
&&
nohup
sh mqnamesrv &
fi
|
3、brokerserver监控
# vim /data/scripts/check_brokerserver_health.sh
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
#!/bin/sh
PATH=
/sbin
:
/bin
:
/usr/sbin
:
/usr/bin
:
/usr/local/bin
:
/usr/local/sbin
SRV_PORT=
"10911"
## 端口号
SRV_PROT=
"tcp"
## 协议类型
SRV_NAME=
"rocketmq_brokerserver"
## 服务名
## 是否已正确扫描
SCAN_FLAG=0
for
ETH_NAME
in
`
/sbin/ifconfig
|
grep
eth1 |
awk
'{print $1}'
`
do
ETH1_ADDR=`
/sbin/ifconfig
${ETH_NAME} |
awk
-F
':'
'/inet addr/{print $2}'
|
sed
's/[a-zA-Z ]//g'
`
[[ -z
"${ETH1_ADDR}"
]] &&
continue
||
break
done
TMP_SRV_PROT=`
echo
${SRV_PROT} |
tr
'[A-Z]'
'[a-z]'
`
if
[[
"${TMP_SRV_PROT}"
==
"tcp"
]];
then
PROT_OPT=
"S"
elif
[[
"${TMP_SRV_PROT}"
==
"udp"
]];
then
PROT_OPT=
"U"
else
echo
"未知的协议类型！"
&& exit1
fi
## 最多扫描3次，成功一次即可，以避免网络抖动而导致误判
for
((i=0; i<3; i++));
do
RETVAL=`
/usr/bin/nmap
-n -s${PROT_OPT} -p ${SRV_PORT} ${ETH1_ADDR} |
grep
open
`
[[ -n
"${RETVAL}"
]] && SCAN_FLAG=1;
break
||
sleep
10
done
if
[[ ${SCAN_FLAG} -
ne
1 ]];
then
[[ -n `
ps
aux |
grep
java |
grep
broker` ]] &&
kill
-9 `
ps
aux |
grep
java |
awk
'/broker/{print $2}'
`
cd
/usr/local/rocketmq/bin
&&
nohup
sh mqbroker -c ..
/conf/2m-noslave/broker-a
.properties &
fi
|
【注意】
以上红色标注部分，在master2上的名称为“broker-b.properties”
4、crontab信息添加
# touch /var/run/check_rocketmq_nameserver.lock
# touch /var/run/check_rocketmq_brokerserver.lock
# echo 'touch /var/run/check_rocketmq_nameserver.lock' >> /etc/rc.d/rc.local
# echo 'touch /var/run/check_rocketmq_brokerserver.lock' >> /etc/rc.d/rc.local
# crontab -e
*/2 * * * * (flock --timeout=0 /var/run/check_rocketmq_nameserver.lock /data/scripts/check_nameserver_health.sh >/dev/null 2>&1)
*/2 * * * * (flock --timeout=0 /var/run/check_rocketmq_brokerserver.lock /data/scripts/check_brokerserver_health.sh >/dev/null 2>&1)
本文出自 “人生理想在于坚持不懈” 博客，请务必保留此出处http://sofar.blog.51cto.com/353572/1540874