RabbitMQ消息可靠性之生产者

消息发布的可靠性

在默认的配置下,生产者发布消息的过程为,首先生产者将消息发送到交换器,然后交换器将消息路由到队列中。在这个过程中可能发生,由于路由键匹配失败,消息无法发送到绑定在交换器上的队列中。这时交换器就会丢掉这条消息(消息进入“黑洞”),但是生产者端是毫无察觉,与发送成功的返回的结果一样,也就是说无法区分消息是否真正到达RabbitMQ的。

 怎么保证我们消息发布的可靠性?有以下常用几种机制。

  • 失败确认
  • 事务
  • 发布者确认
  • 备用交换器

失败确认通知

失败确认就是在发送消息时设置mandatory标志为true,这种方式可以保证,如果消息不可路由,应该将消息返回给发送者并通知失败。

 这里有两点需要注意:

  • RabbitMQ发送失败通知时,如果消息正确路由到队列,则发布者不会受到任何通知。
  • 失败通知消息是无法100%保证一定会返回给生产者端的,因为在失败通知的过程中,如果网络出现问题或服务器宕机,通知失败的消息可能会丢失。

 RabbitMQ消息可靠性之生产者_第1张图片

 Java 生产者代码

  • 发送消息时需要将参数mandatory设置为true
  • 需要在Channel上添加监听器ReturnListener
public class MandatoryProducer {

	public final static String EXCHANGE_NAME = "example_mandatory";
	
	private static String[] logLevels = {"INFO","ERROR"};

	public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
             /*
		 * 以下配置无需设置使用默认即可
		 * port:5672
		 * username:guest
		 * password:guest
		 */
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		Connection connection = connectionFactory.newConnection();

		Channel channel = connection.createChannel();
		channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false);
		//设置失败通知监听器
		channel.addReturnListener(new ReturnListener() {
			@Override
			public void handleReturn(int replyCode, String replyText, String exchange, String routingKey,
					AMQP.BasicProperties properties, byte[] body) throws IOException {

				String message = new String(body);
				System.out.println("["+replyCode+"] ["+replyText+"] ["+exchange+"] ["+routingKey+"] :" +message);
			}
		});
		for (String logLevel : logLevels) {
			String date = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").format(new Date());
			String message = date + " " + logLevel + ": this is log message";
			System.out.println("Sent -> " + message);
			/*
			 * 发送消息,这里第三个参数为mandatory,必须设置为true
			 * basicPublish(String exchange, String routingKey, boolean mandatory, BasicProperties props, byte[] body)
			 */
			channel.basicPublish(EXCHANGE_NAME, logLevel, true, null, message.getBytes());
			TimeUnit.MILLISECONDS.sleep(100);
		}
		channel.close();
		connection.close();
	}

 输出结果如下:

Sent -> 2018-12-12T13:17:45.674 INFO: this is log message
[312] [NO_ROUTE] [example_mandatory] [INFO] :2018-12-12T13:17:45.674 INFO: this is log message
Sent -> 2018-12-12T13:17:45.777 ERROR: this is log message

Java 消费者代码

消费者只消费RoutingKey为ERROR的消息

public class MandatoryConsumer {
	
	public static final String QUEUE_NAME = "mandatory";
	
	public static void main(String[] args) throws IOException, TimeoutException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		
		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		channel.exchangeDeclare(MandatoryProducer.EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
		channel.queueDeclare(QUEUE_NAME, false, false, false, null);
		
		//此处只绑定路由键键为ERROR的消息
		channel.queueBind(QUEUE_NAME, MandatoryProducer.EXCHANGE_NAME, "ERROR");
		//创建队列消费者
		Consumer consumer = new DefaultConsumer(channel){
		       @Override
	            public void handleDelivery(String consumerTag,
	                                       Envelope envelope,
	                                       AMQP.BasicProperties properties,
	                                       byte[] body) throws IOException {
		    	   
	                String message = new String(body, "UTF-8");
	                System.out.println("["+envelope.getExchange()+"] ["+envelope.getRoutingKey() +"] ["+envelope.getDeliveryTag()+"]: "+  message);
	            }
		};
		channel.basicConsume(QUEUE_NAME, true, consumer);
	}
}

消费者输出结果如下:

[example_mandatory] [ERROR] [2]: 2018-12-12T13:17:45.777 ERROR: this is log message

结果总结

 从结果中可以看到只消费了ERROR日志。
 生产者的ReturnListener监听器收到了失败确认通知 “[312] [NO_ROUTE] [example_mandatory] [INFO]”

关于监听器

  • 在Channel关闭时的ShutdownListener监听器
  • 在Connection关闭时的ShutdownListener监听器
//信道关闭时执行
channel.addShutdownListener(new ShutdownListener(){
	@Override
	public void shutdownCompleted(ShutdownSignalException cause) {
         System.out.println("CHANNEL-SHUTDOWN :" + cause.getMessage());
    }
});
		
//连接关闭时执行
connection.addShutdownListener(new ShutdownListener(){
	@Override
    public void shutdownCompleted(ShutdownSignalException cause) {
         System.out.println("CONNECTION-SHUTDOWN :" + cause.getMessage());
    }
});

事务

事务主要是针对信道(Channel)的设置,与JDBC事务类似,包括:启动事务、提交事务、回滚事务三步操作。

事务过程如下:

  1. 客户端发送给服务器Tx.Select(开启事务模式)
  2. 服务器端返回Tx.Select-Ok(开启事务模式ok)
  3. ·推送消息
  4. ·客户端发送给事务提交Tx.Commit
  5. 服务器端返回Tx.Commit-Ok

如果其中任意一个环节出现问题,就会抛出IoException异常,这样用户就可以拦截异常进行事务回滚。

注意:虽然AMQP协议层面为我们提供了事务机制,但是RabbitMQ事务本身有严重的性能问题。

在RabbitMQ中事务会降低2~10倍的性能。

所以,建议使用发布者确认方式来代替事务。

由于事务在实际的开发过程中基本不会使用,所以此处就不做代码展示了。

发布者确认

基于事务的性能问题,RabbitMQ团队为我们拿出了更好的方案,即采用发送方确认模式,该模式比事务更轻量,性能影响几乎可以忽略不计。

与事务一样,confirm模式也是建立在信道(Channel)上的。

  • 实现原理

生产者客户端发送的每一条消息都会指派一个唯一的序列号(在delivery-tag域中),如果RabbitMQ将消息成功的路由到队列,

那么就会返回给生产者客户端一条带有确认序列号的消息。

  • 路由成功的消息

生产者应用程序发布一条消息,需要等待RabbitMQ确认,确认成功后就会返回一条ACK的确认消息,如果RabbitMQ因为自身内部错误导致消息丢失,就会返回一条NACK确认消息。

注意:无论返回的确认消息是ACK还是NACK,只有生产者收到确认消息后,才能继续的发布下一条消息。

 RabbitMQ消息可靠性之生产者_第2张图片

  • 路由失败的消息

即使消息没有路由成功,RabbitMQ也会返回确认成功。所以需要结合失败确认共同完成可靠的消息处理。
注意:生产者发布的消息是否被确认成功,与消费者没有任何关系,只与RabbitMQ本身有关系。

RabbitMQ消息可靠性之生产者_第3张图片

Confirm模式中,有三种确认方式:

  • 同步普通确认

channel.waitForConfirms(),普通发送方确认模式,消息到达交换器,就会返回true。

  • 同步批量确认

channel.waitForConfirmsOrDie()批量确认模式;使用同步方式等所有的消息发送之后才会执行后面代码,只要有一个消息未到达交换器就会抛出IOException异常。

  • 异步确认

channel.addConfirmListener() 异步确认模式,其中包含handleAck和handleNack两个方法。
客户端无法控制消息是否批量确认,由RabbitMQ内部进行处理。可以通过参数multiple进行判断
无论是哪一种确认模式只有生产者收到确认消息后,才能继续的发布下一条消息而异步确认模式优点在于,在等待确认的过程中,可以处理其他的业务逻辑。

普通确认模式

public class ProducerConfirm {
	public final static String EXCHANGE_NAME = "exchange_confirm";
	private static String[] logLevels = {"INFO","ERROR"};
	
	public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");

		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
		
		//设置失败通知监听器
		channel.addReturnListener(new ReturnListener() {
			@Override
			public void handleReturn(int replyCode, String replyText, String exchange, String routingKey,
					AMQP.BasicProperties properties, byte[] body) throws IOException {
				String message = new String(body);
				System.out.println("["+replyCode+"] ["+replyText+"] ["+exchange+"] ["+routingKey+"] :" +message);
			}
		});
		
		//启用发送者确认模式
		channel.confirmSelect();
		
		for (String logLevel : logLevels) {
			String date = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").format(new Date());
			String message = date + " " + logLevel + ": this is log message";
			//发送消息,开启失败通知设置mandatory=true
			channel.basicPublish(EXCHANGE_NAME, logLevel, true, null, message.getBytes());
                   //注意此处会抛出InterruptedException
			if(channel.waitForConfirms()){
				System.out.println("Sent Success -> " + message);
			}else{
				System.out.println("Sent Failure -> " + message);
			}
		}
		Thread.sleep(1000);
		channel.close();
		connection.close();
	}
}

输出结果:

[312] [NO_ROUTE] [example_message_confirm] [INFO] :2018-12-12T19:07:02.366 INFO: this is log message
Sent Success -> 2018-12-12T19:07:02.366 INFO: this is log message
Sent Success -> 2018-12-12T19:07:02.370 ERROR: this is log message

从结果可以看出发送的两条消息,都被交换器确认成功。路由键为INFO的这条消息因为路由失败,而发送了失败通知。

批量确认模式

public class ProducerBatchConfirm {

	public static void main(String[] args) throws Exception {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		connectionFactory.setPort(5672);

		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		channel.exchangeDeclare(ProducerConfirm.EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
		
		//启用发送者确认模式
		channel.confirmSelect();

		for (String logLevel : ProducerConfirm.logLevels) {
			for(int i=0;i<3;i++){
				String date = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").format(new Date());
				String message = date + " " + logLevel + ": this is log message";
                           //执行批量确认模式,如果确认失败,此处会抛出IOException
				channel.basicPublish(ProducerConfirm.EXCHANGE_NAME, logLevel, null, message.getBytes());
			}
		}
		//执行批量确认模式
		channel.waitForConfirmsOrDie();
		channel.close();
		connection.close();
	}
}

输出结果如下:

Sent -> 2018-12-12T19:26:49.064 INFO: this is log message
Sent -> 2018-12-12T19:26:49.068 INFO: this is log message
Sent -> 2018-12-12T19:26:49.068 INFO: this is log message
Sent -> 2018-12-12T19:26:49.068 ERROR: this is log message
Sent -> 2018-12-12T19:26:49.069 ERROR: this is log message
Sent -> 2018-12-12T19:26:49.069 ERROR: this is log message

异步确认模式

  public class ProducerConfirmAsync {
	public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		connectionFactory.setPort(5672);
		
		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		channel.exchangeDeclare(ProducerConfirm.EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
		// 设置失败通知监听器
		channel.addReturnListener(new ReturnListener() {
			@Override
			public void handleReturn(int replyCode, String replyText, String exchange, String routingKey,
					AMQP.BasicProperties properties, byte[] body) throws IOException {
				String message = new String(body);
				System.out.println("["+replyCode+"] ["+replyText+"] ["+exchange+"] ["+routingKey+"] :" +message);
			}
		});
		// 设置异步消息确认监听器
		channel.addConfirmListener(new ConfirmListener() {
			@Override
			public void handleAck(long deliveryTag, boolean multiple) throws IOException {
				System.out.println("ACK : deliveryTag=" + deliveryTag + ",multiple=" + multiple);
			}

			@Override
			public void handleNack(long deliveryTag, boolean multiple) throws IOException {
				System.out.println("NACK : deliveryTag=" + deliveryTag + ",multiple=" + multiple);
			}
		});
		// 启用发送者确认模式
		channel.confirmSelect();
		for (String logLevel : ProducerConfirm.logLevels) {
			for(int i=0;i<3;i++){
				String date = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").format(new Date());
				String message = date + " " + logLevel + ": this is log message";
				System.out.println("Sent -> " + message);
				// 发送消息
				channel.basicPublish(ProducerConfirm.EXCHANGE_NAME, logLevel, true, null, message.getBytes());
			}
		}
		Thread.sleep(3000);
		channel.close();
		connection.close();
	}

 输出的结果:

Sent -> 2018-12-12T19:38:37.745 INFO: this is log message
Sent -> 2018-12-12T19:38:37.747 INFO: this is log message
Sent -> 2018-12-12T19:38:37.747 INFO: this is log message
Sent -> 2018-12-12T19:38:37.748 ERROR: this is log message
Sent -> 2018-12-12T19:38:37.748 ERROR: this is log message
Sent -> 2018-12-12T19:38:37.749 ERROR: this is log message
[312] [NO_ROUTE] [example_message_confirm] [INFO] :2018-12-12T19:38:37.745 INFO: this is log message
ACK : deliveryTag=1,multiple=false
[312] [NO_ROUTE] [example_message_confirm] [INFO] :2018-12-12T19:38:37.747 INFO: this is log message
ACK : deliveryTag=2,multiple=false
[312] [NO_ROUTE] [example_message_confirm] [INFO] :2018-12-12T19:38:37.747 INFO: this is log message
ACK : deliveryTag=3,multiple=false
ACK : deliveryTag=4,multiple=false
ACK : deliveryTag=6,multiple=true

结果总结

从结果中可以看出,一共发送了6条消息,但只收到了5条ACK确认,是因为最后一条multiple=true批量确认了两条消息。
还有3条没有路由成功的消息。

备用交换器

备用交换器可以代替失败确认通知模式,当生产者发送的消息无法路由时,那么消息将被路由到备用交换器。
同时启动备用交换器和设置失败确认(mandatory=true) 时,由于消息发送到备用交换器上,所以不会发送失败确认通知。

声明备用交换器的流程:

  1. 首先需要先声明主交换器,通过交换器参数alternate-exchange,将备用交换器和主交换器进行关联。
  2. 备用交换器与普通的交换器一样,需通过绑定队列对无法路由的消息进行消费。
  3. 建立设置为fanout类型,Queue绑定时的路由键设置为“#”。

生产者代码

public class BackupExchangeProducer {
	//主交换器名称
	public final static String MAIN_EXCHANGE_NAME = "exchange_main";
	//备用交换器名称
	public final static String BACKUP_EXCHANGE_NAME = "exchange_backup";
	public static String[] logLevels = {"INFO","ERROR"};
	public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();

		HashMap exchangeArguments = new HashMap<>();
		//alternate-exchange 这个名字是固定写法。
		exchangeArguments.put("alternate-exchange", BACKUP_EXCHANGE_NAME);
		//声明主交换器,需要将交换器参数传入
		channel.exchangeDeclare(MAIN_EXCHANGE_NAME, BuiltinExchangeType.DIRECT, false, false, exchangeArguments);
		//声明备用交换器,建议用Fanout类型
		channel.exchangeDeclare(BACKUP_EXCHANGE_NAME, BuiltinExchangeType.FANOUT, true ,false ,null);
		
		for (String logLevel : logLevels) {
			String date = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").format(new Date());
			String message = date + " " + logLevel + ": this is log message";
			System.out.println(message);
			channel.basicPublish(MAIN_EXCHANGE_NAME, logLevel, null, message.getBytes());
		}
		channel.close();
		connection.close();
	}
}

输出的结果:

2018-12-13T23:04:15.455 INFO: this is log message
2018-12-13T23:04:15.458 ERROR: this is log message

主交换器上的消费者

public class MainExchangeConsumer {
	
	public static final String QUEUE_NAME = "q_main_exchange";

	public static void main(String[] args) throws IOException, TimeoutException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");
		
		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		
		channel.queueDeclare(QUEUE_NAME, false, false, false, null);
		//只消费RoutingKey为ERROR的的消息
		channel.queueBind(QUEUE_NAME, BackupExchangeProducer.MAIN_EXCHANGE_NAME, "ERROR");
		
		Consumer consumer = new DefaultConsumer(channel) {
			@Override
			public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
					byte[] body) throws IOException {
				String message = new String(body, "UTF-8");
				System.out.println("[" + envelope.getExchange() + "] [" + envelope.getRoutingKey()
						+ "] [" + envelope.getDeliveryTag() + "] Received : " + message);
			}
		};

		channel.basicConsume(QUEUE_NAME, true, consumer);
	}
}

输出结果:

[exchange_main] [INFO] [3] Received : 2018-12-13T23:04:15.455 INFO: this is log message

备用交换器上的消费者

public class BackupExchangeConsumer {
	
	public static final String QUEUE_NAME = "q_backup_exchange";

	public static void main(String[] args) throws IOException, TimeoutException {
		ConnectionFactory connectionFactory = new ConnectionFactory();
		connectionFactory.setHost("127.0.0.1");

		Connection connection = connectionFactory.newConnection();
		Channel channel = connection.createChannel();
		channel.exchangeDeclare(BackupExchangeProducer.BACKUP_EXCHANGE_NAME, BuiltinExchangeType.FANOUT ,true);
		channel.queueDeclare(QUEUE_NAME, false, false, false, null);
		channel.queueBind(QUEUE_NAME, BackupExchangeProducer.BACKUP_EXCHANGE_NAME, "#");

		Consumer consumer = new DefaultConsumer(channel) {
			@Override
			public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
					byte[] body) throws IOException {
				String message = new String(body, "UTF-8");
				//此处打印的是主交换器的名字
				System.out.println("[" + envelope.getExchange() + "] [" + envelope.getRoutingKey()
						+ "] [" + envelope.getDeliveryTag() + "] Received : " + message);
			}
		};

		channel.basicConsume(QUEUE_NAME, true, consumer);
	}
}

输出的结果:

[exchange_main] [ERROR] [1] Received : 2018-12-13T23:04:15.458 ERROR: this is log message

结果总结

  1. 生产者发送两条消息,第一条消息的RoutingKey=INFO,第二条消息的RoutingKey=ERROR
  2. 主交换器上队列的消费者只消费RoutingKey=ERROR的消息
  3. RoutingKey=INFO由于消息路由失败而发送到备用交换器,被备用交换器上队列的消费者读取。

你可能感兴趣的:(消息中间件,RabbitMQ,mandatory,rabbitmq,发布确认,RabbitMQ事务,消息的可靠性)