ActiveMQ集群

ActiveMQ有很多种集群方式，包括Queue consumer clusters、Broker clusters、Discovery of brokers、Networks of brokers、Master Slave、Replicated Message Stores。

其中Master Slave集群有四种实现方式，包括Pure Master Slave、Shared File System Master Slave、JDBC Master Slave和Replicated LevelDB Store。

下面详细介绍这些集群方式的实现过程。

一、Queue consumer clusters

一个队列有多个消费者，每个消费者是平等的，他们一起竞争队列中的消息，当一个消费者挂掉时，消息会被其他消费者接收，这样可以实现队列消息的高可用性和负载均衡。

二、Broker clusters

联合多个Broker结点实现集群，使用故障转移协议failover，当集群中的broker结点挂掉时，客户端会自动连接到集群中的其他broker结点。

三、Discovery of brokers

通过静态发送或动态发现，来自动发现集群中的brokers，客户端能自动检测和连接集群中的broker.

四、Networks of brokers

这种集群方式可以支持某个broker中的消费者去接收其他broker中的消息，从而实现负载均衡、提高系统吞吐量。

比如客户端clientA发送消息到brokerA上的队列queueA，客户端clientB连接brokerB上的队列queueA，这时brokerA上队列queueA的消息就会自动路由到brokerB上的队列queueA上，也就是说clientB可以间接读取brokerA的队列queueA上的消息。

下面创建一个包含三个broker的Network of brokers集群。

1.修改三个broker节点的配置文件activemq.xml

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="network_of_brokers_01">
	<networkConnectors>  
		<networkConnector uri="static:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618)" 
		name="bridge01" 
		duplex="true" 
		dynamicOnly="false" 
		conduitSubscriptions="true" 
		decreaseNetworkConsumerPriority="false"
		/>  
	</networkConnectors>  
				
	<transportConnectors>
		<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
	</transportConnectors>
</broker>

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="network_of_brokers_02">
	<networkConnectors>  
		<networkConnector uri="static:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618)" 
		name="bridge01" 
		duplex="true" 
		dynamicOnly="false" 
		conduitSubscriptions="true" 
		decreaseNetworkConsumerPriority="false"
		/>  
	</networkConnectors>  
				
	<transportConnectors>
		<transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>
	</transportConnectors>
</broker>

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="network_of_brokers_01">
	<networkConnectors>  
		<networkConnector uri="static:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618)" 
		name="bridge01" 
		duplex="true" 
		dynamicOnly="false" 
		conduitSubscriptions="true" 
		decreaseNetworkConsumerPriority="false"
		/>  
	</networkConnectors>  
				
	<transportConnectors>
		<transportConnector name="openwire" uri="tcp://0.0.0.0:61618"/>
	</transportConnectors>
</broker>

2.消息接收者

package com.activemq;

import java.util.Date;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageConsumer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 接收消息
 * @author brushli
 * @date 2014-12-05
 */
public class Receiver {
	private static final Logger logger = LoggerFactory.getLogger(Receiver.class);

	public void receiveMessage(){
		Connection connection = null;
		Session session = null;
		try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, "failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)");
    		connection = connectionFactory.createConnection();  
			connection.start();
			
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("cluster_queue");
			MessageConsumer consumer = session.createConsumer(destination);
			logger.info("begin to receive Message! currentTime="+new Date().toLocaleString());
			while (true) {
                //设置接收者接收消息的时间，为了便于测试，这里谁定为100s
				TextMessage message = (TextMessage) consumer.receive();
                if (null != message) {
                    logger.info("receive message[" + message.getText()+"] ");
                }
            }
		} catch (JMSException e) {
			e.printStackTrace();
		}finally{
			try {
				session.close();  
				connection.close();
			} catch (JMSException e) {
				e.printStackTrace();
			}  
		}
		
    }
	
	public static void main(String[] args) {
		Receiver receiver = new Receiver();
		receiver.receiveMessage();
	}

}

3.消息发送者

package com.activemq;

import java.util.Date;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 发送消息
 * @author brushli
 * @date 2014-12-05
 */
public class Sender {
	private static final Logger logger = LoggerFactory.getLogger(Sender.class);
	
    public void sendMessage(){
    	try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, "failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)");
    		Connection connection = connectionFactory.createConnection();
			connection.start();
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("cluster_queue");
			//创建什么什么模式的生产者，可选参数： Queue, TemporaryQueue, TemporaryTopic, Topic
			MessageProducer producer = session.createProducer(destination);
			producer.setDeliveryMode(DeliveryMode.PERSISTENT);
			logger.info("begin to send Message! currentTime="+new Date().toLocaleString());
			for(int i=1; i<=10; i++) {  
				TextMessage message = session.createTextMessage();  
				message.setText("message" + i);
				producer.send(message); 
				Thread.sleep(5000);
				logger.info("send message [" + "message" + i +"]");
			}  
			logger.info("send Message finished! currentTime="+new Date().toLocaleString());
			session.close();  
			connection.close();  
		} catch (JMSException e) {
			e.printStackTrace();
		} catch (InterruptedException e) {
			e.printStackTrace();
		} 
    }
    
	public static void main(String[] args) {
		Sender sender = new Sender();
		sender.sendMessage();
	}
}

4.测试步骤

(1)启动三个消费者

ActiveMQ01 broker01

2014-12-10 10:04:05 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61616

ActiveMQ02 broker02

2014-12-10 10:04:07 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61617

ActiveMQ03 broker03

2014-12-10 10:04:08 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61618

(2)启动发送者

    发送十条消息

2014-12-10 10:04:10 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61618
2014-12-10 10:04:10 INFO Sender.sendMessage: 39 - begin to send Message! currentTime=2014-12-10 10:04:10
2014-12-10 10:04:15 INFO Sender.sendMessage: 45 - send message [message1]
2014-12-10 10:04:20 INFO Sender.sendMessage: 45 - send message [message2]
2014-12-10 10:04:25 INFO Sender.sendMessage: 45 - send message [message3]
2014-12-10 10:04:30 INFO Sender.sendMessage: 45 - send message [message4]
2014-12-10 10:04:35 INFO Sender.sendMessage: 45 - send message [message5]
2014-12-10 10:04:40 INFO Sender.sendMessage: 45 - send message [message6]
2014-12-10 10:04:45 INFO Sender.sendMessage: 45 - send message [message7]
2014-12-10 10:04:50 INFO Sender.sendMessage: 45 - send message [message8]
2014-12-10 10:04:55 INFO Sender.sendMessage: 45 - send message [message9]
2014-12-10 10:05:00 INFO Sender.sendMessage: 45 - send message [message10]
2014-12-10 10:05:00 INFO Sender.sendMessage: 47 - send Message finished! currentTime=2014-12-10 10:05:00

(3)三个消费者接收消息

消费者01，连接ActiveMQ01 broker01

2014-12-10 10:04:05 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61616
2014-12-10 10:04:05 INFO Receiver.receiveMessage: 39 - begin to receive Message! currentTime=2014-12-10 10:04:05
2014-12-10 10:04:10 INFO Receiver.receiveMessage: 44 - receive message[message1] 
2014-12-10 10:04:15 INFO Receiver.receiveMessage: 44 - receive message[message2] 
2014-12-10 10:04:35 INFO Receiver.receiveMessage: 44 - receive message[message6] 
2014-12-10 10:04:40 INFO Receiver.receiveMessage: 44 - receive message[message7] 

消费者02，连接ActiveMQ02 broker02

2014-12-10 10:04:07 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61617
2014-12-10 10:04:07 INFO Receiver.receiveMessage: 39 - begin to receive Message! currentTime=2014-12-10 10:04:07
2014-12-10 10:04:20 INFO Receiver.receiveMessage: 44 - receive message[message3] 
2014-12-10 10:04:25 INFO Receiver.receiveMessage: 44 - receive message[message4] 
2014-12-10 10:04:45 INFO Receiver.receiveMessage: 44 - receive message[message8] 
2014-12-10 10:04:50 INFO Receiver.receiveMessage: 44 - receive message[message9] 

消费者03，连接ActiveMQ03 broker03

2014-12-10 10:04:08 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61618
2014-12-10 10:04:08 INFO Receiver.receiveMessage: 39 - begin to receive Message! currentTime=2014-12-10 10:04:08
2014-12-10 10:04:30 INFO Receiver.receiveMessage: 44 - receive message[message5] 
2014-12-10 10:04:55 INFO Receiver.receiveMessage: 44 - receive message[message10] 

5.总结

发送十条消息到broker01的队列cluster_queue后，三个消费者分别从broker01,broker02,broker03的队列cluster_queue获取到了消息。

五、Master Slave

5.1Pure Master Slave

这是一种比较简单的主从集群方式，一个主点，一个备点，备点的数据只是主点数据的拷贝，当主点挂掉后，会主动启用备点，消息发送到备点，消费者也改从备点获取消息。

这种方式的优点是：实现简单

缺点是：

(1)只能有一个备点

(2)当主点挂掉后，若需要再启动主点，必须停止备点

(3)没有主备数据同步机制，必须手动把备点的数据文件拷贝到主点

由于Pure Master Slave集群方式有这么多缺点，在ActiveMQ5.8的版本及之后的版本，这种集群方式就取消了。

实现方式如下：

(1)Master的配置文件作如下修改

<brokerbrokerName="pure_master">
<kahaDB directory="${activemq.data}/kahadb_pure_master"/>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>

(2)Slave的配置文件作如下修改

<broker brokerName="pure_slave" masterConnectorURI="tcp://0.0.0.0:61616" shutdownOnMasterFailure="false">
<kahaDB directory="${activemq.base}/data/kahadb_pure_slave "/>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>

(3)启动Master

apache-activemq-5.7.0_master\bin\activemq.bat

(4)启动Slave

apache-activemq-5.7.0_Slave\bin\activemq.bat

(5)生产者

package com.activemq;

import java.util.Date;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 发送消息
 * @author brushli
 * @date 2014-12-06
 */
public class Sender {
	private static final Logger logger = LoggerFactory.getLogger(Sender.class);
	
    public void sendMessage(){
    	try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, 
    				"failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617)");
    		Connection connection = connectionFactory.createConnection();
			connection.start();
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("myQueue");
			//创建什么什么模式的生产者，可选参数： Queue, TemporaryQueue, TemporaryTopic, Topic
			MessageProducer producer = session.createProducer(destination);
			producer.setDeliveryMode(DeliveryMode.PERSISTENT);
			logger.info("begin to send Message! currentTime="+new Date().toLocaleString());
			for(int i=1; i<=10; i++) {  
				TextMessage message = session.createTextMessage();  
				try {
					Thread.sleep(3000);
				} catch (InterruptedException e) {
					e.printStackTrace();
				}
				
				message.setText("" + i);
				producer.send(message);
				logger.info("already send "+i);
			}
			logger.info("send Message finished! currentTime="+new Date().toLocaleString());
			session.close();  
			connection.close();  
		} catch (JMSException e) {
			e.printStackTrace();
		} 
    }
    
	public static void main(String[] args) {
		Sender sender = new Sender();
		sender.sendMessage();
	}

}

(6)消费者

package com.activemq;

import java.util.Date;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageConsumer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 接收消息
 * @author brushli
 * @date 2014-12-06
 */
public class Receiver {
	private static final Logger logger = LoggerFactory.getLogger(Sender.class);

	public void receiveMessage(){
		Connection connection = null;
		Session session = null;
		try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, 
			"failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617)");
    		connection = connectionFactory.createConnection();  
			connection.start();
			
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("myQueue");
			MessageConsumer consumer = session.createConsumer(destination);
			logger.info("begin to receive Message! currentTime="+new Date().toLocaleString());
			while (true) {
                //设置接收者接收消息的时间，为了便于测试，这里谁定为100s
				TextMessage message = (TextMessage) consumer.receive();
                if (null != message) {
                    logger.info(message.getText());
                }
            }
			
		} catch (JMSException e) {
			e.printStackTrace();
		}finally{
			try {
				session.close();  
				connection.close();
			} catch (JMSException e) {
				e.printStackTrace();
			}  
		}
		
    }
	
	public static void main(String[] args) {
		Receiver receiver = new Receiver();
		receiver.receiveMessage();
	}

}

(7)测试过程

1).启动生产者，发送消息
2).启动消费者，可看到接收到的消息
3).关闭消费者
4).生产者继续发送消息A
5).停止Master（可看到生产者端显示连接到Slave（tcp://0.0.0.0:61617）了）
6).生产者继续发送消息B
7).启动消费者
8).消费者接收了消息A和消息B，可见Slave接替了Master的工作，而且储存了之前生产者经过Master发送的消息

生产者的日志如下：

14-12-06 22:50:10 INFO  FailoverTransport.doReconnect - Successfully connected to tcp://127.0.0.1:61616
14-12-06 22:50:10 INFO  Sender.sendMessage - begin to send Message! currentTime=2014-12-6 22:50:10
14-12-06 22:50:14 INFO  Sender.sendMessage - already send 1
14-12-06 22:50:17 INFO  Sender.sendMessage - already send 2
14-12-06 22:50:21 INFO  Sender.sendMessage - already send 3
14-12-06 22:50:25 INFO  Sender.sendMessage - already send 4
14-12-06 22:50:28 INFO  Sender.sendMessage - already send 5
14-12-06 22:50:30 WARN  FailoverTransport.handleTransportFailure - Transport (tcp://127.0.0.1:61616) failed, reason:  java.net.SocketException: Connection reset, attempting to automatically reconnect
14-12-06 22:50:31 INFO  FailoverTransport.doReconnect - Successfully reconnected to tcp://127.0.0.1:61617
14-12-06 22:50:31 INFO  Sender.sendMessage - already send 6
14-12-06 22:50:34 INFO  Sender.sendMessage - already send 7
14-12-06 22:50:38 INFO  Sender.sendMessage - already send 8
14-12-06 22:50:41 INFO  Sender.sendMessage - already send 9
14-12-06 22:50:44 INFO  Sender.sendMessage - already send 10
14-12-06 22:50:44 INFO  Sender.sendMessage - send Message finished! currentTime=2014-12-6 22:50:44

消费者的日志如下：

14-12-06 22:54:13 INFO  FailoverTransport.doReconnect - Successfully connected to tcp://127.0.0.1:61617
14-12-06 22:54:13 INFO  Receiver.receiveMessage - begin to receive Message! currentTime=2014-12-6 22:54:13
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 1
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 2
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 3
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 4
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 5
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 6
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 7
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 8
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 9
14-12-06 22:54:13 INFO  Receiver.receiveMessage - 10

可以看出，在主点61616关闭后，生产者和消费者会主动去连接备点61617，这样就实现了消息的双机热备功能。

5.2、Shared File System Master Slave

利用共享文件系统：当多台机器上都部署了AMQ时，指定这些机器的一个共享的文件路径作为存储。
存储默认是基于AMQ的kahaDB(底层是文件系统)实现。
当一个AMQ实例获得了共享文件的锁，这个实例就成为了Master，其它实例即为Slave。如果这时Master挂了，其它AMQ实例会竞争共享文件的锁，获得锁的就成为Master，其它实例还是Slave。部署时Slave没有限制数，而且自动切换Master不需要人工干预。

我们设置三个ActiveMQ，61616,61617,61618

(1)修改配置文件activemq.xml，指一共享的文件路径

1)ActiveMQ1

<persistenceAdapter>
	<kahaDB directory="D:\shareBrokerData"/>
</persistenceAdapter>

<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>

2)ActiveMQ2

<persistenceAdapter>
	<kahaDB directory="D:\shareBrokerData"/>
</persistenceAdapter>

<transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>

3)ActiveMQ3

<persistenceAdapter>
	<kahaDB directory="D:\shareBrokerData"/>
</persistenceAdapter>

<transportConnector name="openwire" uri="tcp://0.0.0.0:61618"/>

(2)修改生产者的连接

failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)

(3)修改消费者的连接

failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)

(4)测试步骤

1.开启ActiveMQ1  61616

2.开启ActiveMQ2  61617

3.开启ActiveMQ3  61618

4.开启生产者

   发送消息1，消息2

5.关闭ActiveMQ1

   发送消息3，消息4

6.关闭ActiveMQ2

   发送消息5，消息6

7.关闭ActiveMQ3

   等待...

8.开启ActiveMQ1  61616

   连接ActiveMQ1,发送消息7、8、9、10

9.开启消费者

   消费消息1、2、3、4、5、6、7、8、9、10

10.生产者的运行日志

14-12-07 23:07:15 INFO  FailoverTransport.doReconnect - Successfully connected to tcp://127.0.0.1:61616
14-12-07 23:07:15 INFO  Sender.sendMessage - begin to send Message! currentTime=2014-12-7 23:07:15
14-12-07 23:07:20 INFO  Sender.sendMessage - already send 1
14-12-07 23:07:25 INFO  Sender.sendMessage - already send 2
14-12-07 23:07:27 WARN  FailoverTransport.handleTransportFailure - Transport (tcp://127.0.0.1:61616) failed, reason:  java.net.SocketException: Connection reset, attempting to automatically reconnect
14-12-07 23:07:34 INFO  FailoverTransport.doReconnect - Successfully reconnected to tcp://127.0.0.1:61617
14-12-07 23:07:34 INFO  Sender.sendMessage - already send 3
14-12-07 23:07:39 INFO  Sender.sendMessage - already send 4
14-12-07 23:07:40 WARN  FailoverTransport.handleTransportFailure - Transport (tcp://127.0.0.1:61617) failed, reason:  java.io.EOFException, attempting to automatically reconnect
14-12-07 23:07:47 INFO  FailoverTransport.doReconnect - Successfully reconnected to tcp://127.0.0.1:61618
14-12-07 23:07:47 INFO  Sender.sendMessage - already send 5
14-12-07 23:07:52 INFO  Sender.sendMessage - already send 6
14-12-07 23:07:54 WARN  FailoverTransport.handleTransportFailure - Transport (tcp://127.0.0.1:61618) failed, reason:  java.net.SocketException: Connection reset, attempting to automatically reconnect
14-12-07 23:08:04 INFO  FailoverTransport.doReconnect - Successfully reconnected to tcp://127.0.0.1:61616
14-12-07 23:08:04 INFO  Sender.sendMessage - already send 7
14-12-07 23:08:09 INFO  Sender.sendMessage - already send 8
14-12-07 23:08:14 INFO  Sender.sendMessage - already send 9
14-12-07 23:08:20 INFO  Sender.sendMessage - already send 10
14-12-07 23:08:20 INFO  Sender.sendMessage - send Message finished! currentTime=2014-12-7 23:08:20

从日志中可以看出，生产者启动的时候是连接ActiveMQ1 61616；

当ActiveMQ1关闭时，生产者自动去连接ActiveMQ2 61617，消息发送至ActiveMQ2；

当ActiveMQ2关闭时，生产者自动去连接ActiveMQ3 61618，消息发送至ActiveMQ3；

当ActiveMQ3关闭时，生产者会一直等待；

当再次启动ActiveMQ1时，生产者自动去连接ActiveMQ1 61616，消息发送至ActiveMQ1；

11.消费者的运行日志

14-12-07 23:15:04 INFO  FailoverTransport.doReconnect - Successfully connected to tcp://127.0.0.1:61616
14-12-07 23:15:04 INFO  Receiver.receiveMessage - begin to receive Message! currentTime=2014-12-7 23:15:04
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 1
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 2
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 3
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 4
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 5
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 6
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 7
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 8
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 9
14-12-07 23:15:04 INFO  Receiver.receiveMessage - 10

启动消费者时，消费者连接ActiveMQ1，并读取所有消息
12.作了集群后的测试

(1). 测试环境
操作系统：Win7 旗舰版64位
CPU：Intel(R) Core(TM) i5-3470 CPU @ 3.2GHz
JAVA：JDK1.6
ActiveMQ：ActiveMQ5.7.0
修改配置文件/conf/activemq.xml

<persistenceAdapter>
<kahaDB directory="D:\shareBrokerData" indexCacheSize="100000" indexWriteBatchSize="1000" enableJournalDiskSyncs="false"  journalMaxFileLength="128mb" concurrentStoreAndDispatchQueues="true" concurrentStoreAndDispatchTopics="true"/>
</persistenceAdapter>

(2)测试步骤

单个生产者向事务/非事务,持久化/非持久化 四个队列分别插入100W条记录,统计所需要的时间.

(3)测试代码

package com.activemq;

import java.util.Date;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MapMessage;
import javax.jms.MessageConsumer;
import javax.jms.MessageProducer;
import javax.jms.Session;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class ActivemqExample {

    private static final Logger logger = LoggerFactory.getLogger(ActivemqExample.class);
    private ConnectionFactory connectionFactory = null;
    private Connection connection = null;

    public void initActiveMQ(){
    	connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    			ActiveMQConnection.DEFAULT_PASSWORD, "failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)");
    }
    
    public void sendTransactedMessage(int persistent){
    	try {
    		connection = connectionFactory.createConnection();
			connection.start();
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.TRUE, Session.AUTO_ACKNOWLEDGE);
			String persistentString = (persistent == DeliveryMode.PERSISTENT ? "persistent" : "non-persistent");
			String queueName="queue-transacted-"+persistentString;
			Destination destination = session.createQueue(queueName);
			//创建什么什么模式的生产者，可选参数： Queue, TemporaryQueue, TemporaryTopic, Topic
			MessageProducer producer = session.createProducer(destination);
			producer.setDeliveryMode(persistent);
			logger.info("begin to send Message! currentTime="+new Date().toLocaleString());
			long beginTime = System.currentTimeMillis();
			
			int totalCount = 1000000;
			for(int i=1; i<=totalCount; i++) {  
				MapMessage message = session.createMapMessage();  
				message.setLong("count", new Date().getTime());  
				//通过消息生产者发出消息  
				producer.send(message); 
				
				if(i%100 == 0){
					session.commit();   //支持事务
					if(i%100000 == 0){
						logger.info("已发送["+i+"]条"+(persistent == DeliveryMode.PERSISTENT ? "persistent" : "non-persistent")+"消息!");
					}
				}
			}  
			long endTime = System.currentTimeMillis();
			logger.info("send Message finished! currentTime="+new Date().toLocaleString());
			logger.info("send Transacted "+persistentString+" Message,100/pertime,totalCount="+totalCount+",cost time="+(endTime-beginTime)/1000+"s,speed="+(totalCount/((endTime-beginTime)/1000))+"/s");
			session.close();  
			connection.close();  
		} catch (JMSException e) {
			e.printStackTrace();
		} 
    }
    
    public void sendNonTransactedMessage(int persistent){
    	try {
			connection = connectionFactory.createConnection();  
			connection.start();  
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			String persistentString = (persistent == DeliveryMode.PERSISTENT ? "persistent" : "non-persistent");
			String queueName="queue-non-transacted-"+persistentString;
			Destination destination = session.createQueue(queueName);
			//创建什么什么模式的生产者，可选参数： Queue, TemporaryQueue, TemporaryTopic, Topic
			MessageProducer producer = session.createProducer(destination);
			producer.setDeliveryMode(persistent);
			logger.info("begin to send Message! currentTime="+new Date().toLocaleString());
			long beginTime = System.currentTimeMillis();
			
			int totalCount = 1000000;
			for(int i=1; i<=totalCount; i++) {  
				MapMessage message = session.createMapMessage();  
				message.setLong("count", new Date().getTime());  
				//通过消息生产者发出消息  
				producer.send(message);
				if(i%100000 == 0){
					logger.info("已发送["+i+"]条"+(persistent == DeliveryMode.PERSISTENT ? "persistent" : "non-persistent")+"消息!");
				}
			}  
			long endTime = System.currentTimeMillis();
			logger.info("send Message finished! currentTime="+new Date().toLocaleString());
			logger.info("send Non Transacted "+persistentString+" Message,1/pertime,totalCount="+totalCount+",cost time="+(endTime-beginTime)/1000+"s,speed="+(totalCount/((endTime-beginTime)/1000))+"/s");
			session.close();  
			connection.close();  
		} catch (JMSException e) {
			e.printStackTrace();
		} 
    }
    
    public void receiveMessage(){
		try {
			connection = connectionFactory.createConnection();  
			connection.start();
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("queue-transacted-persistent");
			MessageConsumer consumer = session.createConsumer(destination);
			logger.info("begin to receive Message! currentTime="+new Date().toLocaleString());
			long beginTime = System.currentTimeMillis();
			int totalCount = 0;
			while (true && totalCount < 1000000) {
                //设置接收者接收消息的时间，为了便于测试，这里谁定为100s
				MapMessage message = (MapMessage) consumer.receive();
                if (null != message) {
                    totalCount++;
                    
                    if(totalCount % 100000 == 0){
                    	System.out.println("已收到["+totalCount+"]条消息");
                    }
                }
            }
			long endTime = System.currentTimeMillis();
			logger.info("receive transacted persistent Message,1/pertime,totalCount="+totalCount+",cost time="+(endTime-beginTime)/1000+"s,speed="+(totalCount/((endTime-beginTime)/1000))+"/s");
		} catch (JMSException e) {
			e.printStackTrace();
		}
		
    }
    
	/**
	 * @param args
	 */
	public static void main(String[] args) {
		ActivemqExample activemqExample = new ActivemqExample();
		activemqExample.initActiveMQ();
		activemqExample.sendTransactedMessage(DeliveryMode.PERSISTENT);
		activemqExample.sendTransactedMessage(DeliveryMode.NON_PERSISTENT);
		activemqExample.sendNonTransactedMessage(DeliveryMode.PERSISTENT);
		activemqExample.sendNonTransactedMessage(DeliveryMode.NON_PERSISTENT);
//		activemqExample.receiveMessage();
	}
}

(4)性能测试报告

主备Shared File System Master Slave集群的测试报告

1)第一次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	106s	9433/s
是	否	100W	62s	16129/s
否	是	100W	106s	9433/s
否	否	100W	22s	45454/s

2)第二次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	119s	8403/s
是	否	100W	57s	17543/s
否	是	100W	119s	8403/s
否	否	100W	24s	41666/s

3)第三次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	123s	8130/s
是	否	100W	62s	16129/s
否	是	100W	132s	7575/s
否	否	100W	20s	50000/s

(5)中断测试

Shared File System Master Slave集群中断测试报告

测试步骤：向各个队列发送100W条记录，中间关闭当前连接的ActiveMQ，重新连接其他ActiveMQ，查看测试结果.

支持事务	支持持久化	结果
是	是	中断前发送的消息有效,中断后重新连接ActiveMQ,剩余的队列数据并不会重新发送. 中断前发送的消息有效,中断后不会重新发送消息.
是	否	中断前发送的消息会回滚,消息全部删除,中断后重新连接ActiveMQ,剩余的队列消息并不会重新发送. 所有消息丢失!
否	是	中断前发送的消息有效,重新连接ActiveMQ后剩余的队列消息会重新发送. 所有消息有效!
否	否	中断前发送的消息会回滚,消息全部删除,重新连接ActiveMQ,剩余的队列消息会重新发送. 中断前发送的消息无效,中断后发送的消息有效!

5.3、JDBC Master Slave

其实与Shared File System一样，只是把共享文件系统换成数据库作为存储。方便实用，但要保证数据库的高可用性。

1、修改三个activemq的配置文件activemq.xml

<broker brokerName="jdbc_broker1" dataDirectory="${activemq.data}" useJmx="true">
  <persistenceAdapter>
      <jdbcPersistenceAdapter dataDirectory="${activemq.base}/data" dataSource="#dataSource" createTablesOnStartup="false" />
  </persistenceAdapter>
</broker>

<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
				<property name="driverClass" value="com.mysql.jdbc.Driver" />
				<property name="jdbcUrl" value="jdbc:mysql://127.0.0.1:3306/test?characterEncoding=utf-8" />
				<property name="user" value="root" />
				<property name="password" value="root" />
				<property name="initialPoolSize" value="10" />
				<property name="minPoolSize" value="10" />
				<property name="maxPoolSize" value="100" />
</bean>

 

<broker brokerName="jdbc_broker2" dataDirectory="${activemq.data}" useJmx="true">
  <persistenceAdapter>
      <jdbcPersistenceAdapter dataDirectory="${activemq.base}/data" dataSource="#dataSource" createTablesOnStartup="false" />
  </persistenceAdapter>
</broker>

<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
				<property name="driverClass" value="com.mysql.jdbc.Driver" />
				<property name="jdbcUrl" value="jdbc:mysql://127.0.0.1:3306/test?characterEncoding=utf-8" />
				<property name="user" value="root" />
				<property name="password" value="root" />
				<property name="initialPoolSize" value="10" />
				<property name="minPoolSize" value="10" />
				<property name="maxPoolSize" value="100" />
</bean>

 

<broker brokerName="jdbc_broker3" dataDirectory="${activemq.data}" useJmx="true">
  <persistenceAdapter>
      <jdbcPersistenceAdapter dataDirectory="${activemq.base}/data" dataSource="#dataSource" createTablesOnStartup="false" />
  </persistenceAdapter>
</broker>

<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
				<property name="driverClass" value="com.mysql.jdbc.Driver" />
				<property name="jdbcUrl" value="jdbc:mysql://127.0.0.1:3306/test?characterEncoding=utf-8" />
				<property name="user" value="root" />
				<property name="password" value="root" />
				<property name="initialPoolSize" value="10" />
				<property name="minPoolSize" value="10" />
				<property name="maxPoolSize" value="100" />
</bean>

注意：jdbc_broker1中的createTablesOnStartup属性，初次启动时要设为true，ActiveMQ会自动到数据库test中创建三个表activemq_acks,activemq_lock,activemq_msgs,之后就可以设置成false.
2、添加JAR包

添加JAR包mysql-connector-java-5.1.22-bin.jar、c3p0-0.9.1.2.jar到ActiveMQ的安装目录的lib文件下，比如：apache-activemq-5.7.0\lib

3、测试

3.1 集群中断测试

每隔5秒向集群发送一条消息，一共发送10消息，期间关闭服务器ActiveMQ1、ActiveMQ2

测试步骤

(1)开启ActiveMQ1 61616

(2)开启ActiveMQ2 61617

(3)开启ActiveMQ3 61618

(4)开启生产者，连接ActiveMQ1 61616

(5)发送消息1、2、3、4、5、6、7

(6)关闭ActiveMQ1 61616

(7)生产者连接ActiveMQ2 61617

(8)发送消息8、9

(9)关闭ActiveMQ2 61617

(10)生产者连接ActiveMQ3 61618

(11)发送消息11

(12)开启消费者

(13)消费消息1、2、3、4、5、6、7、8、9、10

(14)生产者运行日志

2014-12-08 14:59:12 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61616
2014-12-08 14:59:12 INFO Sender.sendMessage: 39 - begin to send Message! currentTime=2014-12-8 14:59:12
2014-12-08 14:59:17 INFO Sender.sendMessage: 45 - send message 1
2014-12-08 14:59:22 INFO Sender.sendMessage: 45 - send message 2
2014-12-08 14:59:27 INFO Sender.sendMessage: 45 - send message 3
2014-12-08 14:59:32 INFO Sender.sendMessage: 45 - send message 4
2014-12-08 14:59:37 INFO Sender.sendMessage: 45 - send message 5
2014-12-08 14:59:42 INFO Sender.sendMessage: 45 - send message 6
2014-12-08 14:59:47 INFO Sender.sendMessage: 45 - send message 7
2014-12-08 14:59:48 WARN FailoverTransport.handleTransportFailure: 255 - Transport (tcp://127.0.0.1:61616) failed, reason:  java.net.SocketException: Connection reset, attempting to automatically reconnect
2014-12-08 14:59:49 INFO FailoverTransport.doReconnect: 1032 - Successfully reconnected to tcp://127.0.0.1:61617
2014-12-08 14:59:52 INFO Sender.sendMessage: 45 - send message 8
2014-12-08 14:59:57 INFO Sender.sendMessage: 45 - send message 9
2014-12-08 14:59:58 WARN FailoverTransport.handleTransportFailure: 255 - Transport (tcp://127.0.0.1:61617) failed, reason:  java.net.SocketException: Connection reset, attempting to automatically reconnect
2014-12-08 14:59:59 INFO FailoverTransport.doReconnect: 1032 - Successfully reconnected to tcp://127.0.0.1:61618
2014-12-08 15:00:02 INFO Sender.sendMessage: 45 - send message 10
2014-12-08 15:00:02 INFO Sender.sendMessage: 47 - send Message finished! currentTime=2014-12-8 15:00:02

(15)消费者运行日志

2014-12-08 15:32:17 INFO FailoverTransport.doReconnect: 1030 - Successfully connected to tcp://127.0.0.1:61618
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 39 - begin to receive Message! currentTime=2014-12-8 15:32:17
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 1
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 2
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 3
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 4
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 5
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 6
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 7
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 8
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 9
2014-12-08 15:32:17 INFO Receiver.receiveMessage: 44 - 10

测试的代码

(1)生产者

package com.activemq;

import java.util.Date;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 发送消息
 * @author brushli
 * @date 2014-12-05
 */
public class Sender {
	private static final Logger logger = LoggerFactory.getLogger(Sender.class);
	
    public void sendMessage(){
    	try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, "failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)");
    		Connection connection = connectionFactory.createConnection();
			connection.start();
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			Session session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("cluster_queue");
			//创建什么什么模式的生产者，可选参数： Queue, TemporaryQueue, TemporaryTopic, Topic
			MessageProducer producer = session.createProducer(destination);
			producer.setDeliveryMode(DeliveryMode.PERSISTENT);
			logger.info("begin to send Message! currentTime="+new Date().toLocaleString());
			for(int i=1; i<=10; i++) {  
				TextMessage message = session.createTextMessage();  
				message.setText(""+i);
				producer.send(message); 
				Thread.sleep(5000);
				logger.info("send message " + i);
			}  
			logger.info("send Message finished! currentTime="+new Date().toLocaleString());
			session.close();  
			connection.close();  
		} catch (JMSException e) {
			e.printStackTrace();
		} catch (InterruptedException e) {
			e.printStackTrace();
		} 
    }
    
	public static void main(String[] args) {
		Sender sender = new Sender();
		sender.sendMessage();
	}

}

(2)消费者

package com.activemq;

import java.util.Date;

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.MessageConsumer;
import javax.jms.Session;
import javax.jms.TextMessage;

import org.apache.activemq.ActiveMQConnection;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * ActiveMQ 接收消息
 * @author brushli
 * @date 2014-12-05
 */
public class Receiver {
	private static final Logger logger = LoggerFactory.getLogger(Receiver.class);

	public void receiveMessage(){
		Connection connection = null;
		Session session = null;
		try {
    		ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(ActiveMQConnection.DEFAULT_USER, 
    				ActiveMQConnection.DEFAULT_PASSWORD, "failover:(tcp://127.0.0.1:61616,tcp://127.0.0.1:61617,tcp://127.0.0.1:61618)");
    		connection = connectionFactory.createConnection();  
			connection.start();
			
			//（parameter a:是否支持事务，parameter b:消息应答模式）
			session = connection.createSession(Boolean.FALSE, Session.AUTO_ACKNOWLEDGE);
			Destination destination = session.createQueue("cluster_queue");
			MessageConsumer consumer = session.createConsumer(destination);
			logger.info("begin to receive Message! currentTime="+new Date().toLocaleString());
			while (true) {
                //设置接收者接收消息的时间，为了便于测试，这里谁定为100s
				TextMessage message = (TextMessage) consumer.receive();
                if (null != message) {
                    logger.info(message.getText());
                }
            }
		} catch (JMSException e) {
			e.printStackTrace();
		}finally{
			try {
				session.close();  
				connection.close();
			} catch (JMSException e) {
				e.printStackTrace();
			}  
		}
		
    }
	
	public static void main(String[] args) {
		Receiver receiver = new Receiver();
		receiver.receiveMessage();
	}

}

3.2 集群性能测试

数据库：MySQL5.5.10 ，存储引擎Innodb

测试报告

(1)第一次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	288s	3472/s
是	否	100W	34s	29411/s
否	是	100W	672s	1488/s
否	否	100W	18s	55555/s

(2)第二次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	270s	3703/s
是	否	100W	32s	31250/s
否	是	100W	700s	1428/s
否	否	100W	17s	58823/s

(3)第三次发送

支持事务	支持持久化	测试数据	花费时间	速度
是	是	100W	264s	3787/s
是	否	100W	35s	28571/s
否	是	100W	733s	1364/s
否	否	100W	20s	50000/s

注意：由于ActiveMQ和数据库MySQL都装在一个主机上，所以速度会慢很多，特别是需要持久化的非事务消息，由于每发送一条消息都需要向数据表activemq_msgs插入一条记录，导致速度很慢。

5.4 Replicated LevelDB Store

ActiveMQ 5.9.0以后，开始支持Replicated LevelDB store方式的主从集群方式。

Replicated LevelDB store使用Zookeeper作为broker节点集群中获取主点的协调中间件，将主点数据保存到LevelDB store，各备点也有各自的LevelDB store，数据会从主点同步到备点。对于Zookeeper的集群，如果集群中有N个节点，那么最多允许(N-1)/2个节点挂掉，集群仍然可用；如果N=3，那么1个节点挂掉集群是可用的，2个节点挂掉，集群就不可用了。关于Zookeeper的集群安装方式请看本人写的另一篇博文： ZooKeeper集群

本人用ActiveMQ连接Zookeeper，一直没成功过，所以该方式的集群待续...

 六、Replicated Message Stores（消息复制存储）

6.1 Use RAID disks

6.2 SAN or shared network drive

6.3 Master/Slave

6.4 Clustered JDBC databases

6.5 Use C-JDBC