Rabbitmq可靠性、幂等性、及集群高可用

前言

Rabbimq的作用,解耦,消峰,异步通信等。

一、Rabbitmq可靠性

1.生产者可靠性保证

a. rabbitmq发送消息,如果不进行特殊设置(设置为事务模式或者确认模型),则生产者可靠性无法保证,默认rabbitmq不会返回任何信息给生产者。
b.事务模式的缺点,事务模式本身为同步机制,性能上比确认模式要差,确认模式为异步模式(与mqserver连接的信道也为双向信道)。

1.mandatory参数与备份交换机

1.mandatory参数设置为true
package com.rabbitmq.test;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.util.concurrent.TimeoutException;


public class ProducerV5 {
    static String EXCHANGE_NAME = "exchange_name";
    static String QUEUE_NAME = "queue_name";
    static String ROUTING_KEY = "root_key";
    public static void main(String[] args) throws Exception{
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("****");
        factory.setPassword("****");
        factory.setVirtualHost("/");
        factory.setHost("**.**.**.**");
        factory.setPort(5672);
        Connection conn = null;
        final Channel channel;
        try {
            conn = factory.newConnection();
            //注意conn不关闭时,程序不停止。
            channel = conn.createChannel();
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, null);
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            //当rabbimq无法将消息正确路由到队列时,触发。
            channel.addReturnListener(new ReturnListener() {
                public void handleReturn(int replyCode, String replyText, String exchange, String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
                    System.out.println("replyCode:"+replyCode);
                    System.out.println("replyText:"+replyText);
                    System.out.println("exchange:"+exchange);
                    System.out.println("routingKey:"+routingKey);
                    String string = new String(body);
                    System.out.println("basic return"+string);
                }
            });
            int i=1;
            while (i<20){
                String string = "hello world"+i;
                //rabbitmq无法找到EXCHANGE_NAME时,会触发异常。无法找到queue时,mq会发送Basic.Return命令给生产者,addReturnListener进行监听。
                channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY+"uuuu", true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                        string.getBytes());
                i++;
                System.out.println(i+"ddd");
            }
            System.out.println("dddfdafdf");
            //channel.addReturnListener();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }finally {
            conn.close();
        }
        System.out.println("end");
    }
}
2.备份交换机
package com.rabbitmq.test;
import com.rabbitmq.client.*;
import org.omg.CosNaming.NamingContextExtPackage.StringNameHelper;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;


public class ProducerV5 {
    static String EXCHANGE_NAME = "exchange_name_v2";
    static String BEI_FEN_EXCHANGE_NAME = "bei_fen_exchange_name";
    static String UN_ROUTED_QUEUE_NAME = "un_routed_exchange_name";
    static String QUEUE_NAME = "queue_name";
    static String ROUTING_KEY = "root_key";
    public static void main(String[] args) throws Exception{
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("***");
        factory.setPassword("***");
        factory.setVirtualHost("/");
        factory.setHost("**.**.**.**");
        factory.setPort(5672);
        Connection conn = null;
        final Channel channel;
        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
            Map<String,Object> argsProperties = new HashMap<String, Object>();
            argsProperties.put("alternate-exchange", BEI_FEN_EXCHANGE_NAME);
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, argsProperties);
            channel.exchangeDeclare(BEI_FEN_EXCHANGE_NAME,"fanout", true, false, null);
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueDeclare(UN_ROUTED_QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            channel.queueBind(UN_ROUTED_QUEUE_NAME, BEI_FEN_EXCHANGE_NAME, "");
            int i=1;
            while (i<20){
                String string = "hello world"+i;
                //此时路由不到队列时,消息发送到备份交换机。
                channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY+"uuuufff", true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                        string.getBytes());
                i++;
                System.out.println(i+"ddd");
            }
            //channel.addReturnListener();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }finally {
            conn.close();
        }
        System.out.println("end");
    }
}

注意:备份交换机与普通交换机没有区别,发送到备份交换机的路由key与正常交换机一样。所以备份交换机建议设置为fanout。
在这里插入图片描述

3.confirm异步模式
package com.rabbitmq.test;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.TreeMap;
import java.util.concurrent.TimeoutException;

public class ProducerV3 {
    static String EXCHANGE_NAME = "exchange_name";
    static String QUEUE_NAME = "queue_name";
    static String ROUTING_KEY = "root_key";
    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("***");
        factory.setPassword("****");
        factory.setVirtualHost("/");
        factory.setHost("10.12.24.104");
        factory.setPort(5672);
        Connection conn = null;
        final Channel channel;
        final TreeMap<Long, String> map = new TreeMap<Long, String>();
        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, null);
            channel.exchangeDeclare(EXCHANGE_NAME+"dddd","direct", true, false, null);

            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            //设置mq为认证模式。
            channel.confirmSelect();
            channel.addReturnListener(new ReturnListener() {
                public void handleReturn(int replyCode, String replyText, String exchange, String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
                    System.out.println("replyCode:"+replyCode);
                    System.out.println("replyText:"+replyText);
                    System.out.println("exchange:"+exchange);
                    System.out.println("routingKey:"+routingKey);
                    String string = new String(body);
                    System.out.println("basic return"+string);

                }
            });
            //rabbitmq发送Basic.Ack.(这里测试与书上说的不一样,并不是正确的投递到队列,才会发送Basic.Ack,自己测试得到的结果消息到达了交换机就会发送Ack,所以发送认证机制最好跟备份交换机或者madatory参数共同保证可靠性。Nack理解可能是rabbitmq因为自身错误,无法接收消息时,才发送Nack。)
            channel.addConfirmListener(new ConfirmListener() {
                public void handleAck(long deliveryTag, boolean multiple) throws IOException {
                   System.out.println("handleAck"+deliveryTag);
                   map.remove(deliveryTag);
                }
                //mutiple为之前的消息是否正确投递
                public void handleNack(long deliveryTag, boolean multiple) throws IOException {
                    System.out.println("handleNack"+deliveryTag);
                    // 重新发送一条相同的消息。注意信道认证模式时,没发送到信道的消息都有唯一的deliveryTag.
                    String string = map.get(deliveryTag);
                    channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY, true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                            string.getBytes());
                    map.clear(deliveryTag);
                }
            });
            int i=1;
            while (i<20){
                String string = "hello world"+i;
                long nextNo = channel.getNextPublishSeqNo();
                //这里无法路由到正确的队里,仍然会触发ack.
                channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY+"gggggg", true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                        string.getBytes());
                i++;
                map.put(nextNo, string);
            }
            System.out.println("dddfdafdf");
            //channel.addReturnListener();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }
    }
}
4.confirm批量认证模式
package com.rabbitmq.test;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.ArrayList;
import java.util.concurrent.TimeoutException;

public class ProducerV4 {
    static String EXCHANGE_NAME = "exchange_name";
    static String QUEUE_NAME = "queue_name";
    static String ROUTING_KEY = "root_key";
    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("kreditplus_dev");
        factory.setPassword("kreditplus_dev");
        factory.setVirtualHost("/");
        factory.setHost("10.12.24.104");
        factory.setPort(5672);
        Connection conn = null;
        Channel channel = null;
        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, null);
            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            channel.confirmSelect();
            ArrayList<String> arraylist = new ArrayList<String>();
            channel.addReturnListener(new ReturnListener() {
                public void handleReturn(int replyCode, String replyText, String exchange, String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
                    System.out.println("replyCode:" + replyCode);
                    System.out.println("replyText:" + replyText);
                    System.out.println("exchange:" + exchange);
                    System.out.println("routingKey:" + routingKey);
                    String string = new String(body);
                    System.out.println("basic return" + string);
                }
            });
            int i=1;
            int mod = 0;
            while (i<40) {
                String string = "hello world" + i;
                long nextNo = channel.getNextPublishSeqNo();
                System.out.println("nextNo"+ nextNo);
                channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                        string.getBytes());
                i++;
                arraylist.add(string);
                if (++mod >= 20) {
                    mod = 0;
                    try {
                        //批量认证模式
                        boolean value = channel.waitForConfirms();
                        if (!value) {
                            //重复发送
                            for (String string_v2 : arraylist) {
                                channel.basicPublish(EXCHANGE_NAME, "dadfa", true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                                        string_v2.getBytes());
                            }
                            System.out.println();
                        } else {
                            arraylist.clear();
                            System.out.println("success" + string);
                        }
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }
    }
}

2.rabbitmq持久化可靠性保证

1.交换机,队列,消息持久化。

交换机持久化:保证重启时,交换机元数据不丢失。不持久化,则mq重启后,交换机无法继续使用。 不影响消息是否丢失。
队列持久化:队列不持久化时,重启后消息丢失。
消息持久化:发送消息时,deliveryMode设置为2,将消息持久化,如果只有队列持久化,没有消息持久化,重启后,消息会丢失。

2.镜像队列

消息持久化到磁盘时,可能需要在操作系统缓存保存一段时间,才能够存到物理磁盘中,这段时间如果rabbitmq挂掉,可能会丢失数据,可通过镜像队里进行保证。

3.消息的保存时间

消息保存时间有两个限制:
1.消息本身的的保存时间TTL和队列的保存时间TTL,消息的过期时间,取两者中小的那个,不设置TTL,则消息会等到消费后,才会从rabbitmq删除。

3.消费者可靠性保证

1.消费者可靠性保证主要利用手动确认机制。
2.如果多个消费者订阅同一队列,rabbitmq采用轮训方式推送消息到消费者。
3. channel.basicQos(2),控制了单个消费者,在信道上未确认的个数,channel.basixQos(10, true), global为true整体信道上的消费者都需要遵从这个限定值,(加和限定和单个限定有待实验)。

package com.rabbitmq.test;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class ConsumerTestV2 {
    static String QUEUE_NAME = "queue_name";

    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("10.12.24.104");
        factory.setPort(5672);
        factory.setUsername("kreditplus_dev");
        factory.setPassword("kreditplus_dev");
        //factory.setVirtualHost("/");
        Connection connection = null;

        try {
            connection = factory.newConnection();
            final Channel channel = connection.createChannel();
            //设置客户端最多接收未被ack的消息的个数, 起到了一个滑动窗口的作用,channel.basicQos还有针对一个消费者和多个消费者的区别。
            channel.basicQos(2);
            System.out.println("aaaaaa");
            boolean autoAck = false;
            channel.basicConsume(QUEUE_NAME, autoAck, "myConsumerTagV2",
                    new DefaultConsumer(channel) {
                        @Override
                        public void handleDelivery(String consumerTag,
                                                   Envelope envelope,
                                                   AMQP.BasicProperties properties,
                                                   byte[] body)
                                throws IOException
                        {
                            String routingKey = envelope.getRoutingKey();
                            String contentType = properties.getContentType();
                            long deliveryTag = envelope.getDeliveryTag();
                            // (process the message components here ...)
                            System.out.print("aaaaaavdadfadsa"+new String(body)+deliveryTag);
                            try {
                                Thread.sleep(1000);
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                            channel.basicAck(deliveryTag, false);
                        }
                    });
//            channel.close();
//            connection.close();

       } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }finally {
        }
    }
}

4.死信队列

消息在一个队列中变成死信,能够过重新被发送到一个交换机DLX,
绑定DLX的队列就称之为死信队列。
消息变成死信的情况:
1.消息被拒绝
2.消息过期
3.队列达到最大长度。

生产者

package com.rabbitmq.test;
import com.rabbitmq.client.*;
import com.rabbitmq.client.AMQP.BasicProperties;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeoutException;

public class Producer {
    static String EXCHANGE_NAME = "exchange_name";
    static String QUEUE_NAME = "queue_name_v2";
    static String ROUTING_KEY = "root_key";
    static String DEAD_DLX_EXCHANGE = "dlx_exchange";
    static String DEAD_QUEUE = "dlx_queue";
    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("kreditplus_dev");
        factory.setPassword("kreditplus_dev");
        factory.setVirtualHost("/");
        factory.setHost("10.12.24.104");
        factory.setPort(5672);
        Connection conn = null;
        Channel channel = null;
        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, null);
            //dead exchange
            channel.exchangeDeclare(DEAD_DLX_EXCHANGE,"direct", true, false, null);
            Map<String,Object> ars= new HashMap<String,Object>();
            ars.put("x-dead-letter-exchange", DEAD_DLX_EXCHANGE);
            //也可以为这个DLX指定路由键,如果没有特殊指定,则使用原队列的路由键
            ars.put("x-dead-letter-routing-key","dlx-routing-key"); 
            //给正常队列绑定死信交换机
            channel.queueDeclare(QUEUE_NAME, true, false, false, ars);
            channel.queueDeclare(DEAD_QUEUE, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            //死信交换机、死信队列、路由key绑定
            channel.queueBind(DEAD_QUEUE, DEAD_DLX_EXCHANGE, "dlx-routing-key");
            int i=10;
            while (i<20){
                String string = "hello world"+i;
                channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY, new BasicProperties.Builder().deliveryMode(2).build(),  //设置为持久化
                        string.getBytes());
                i++;
            }

            channel.close();
            conn.close();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }
        System.out.println("aaaaaa");
    }

}

消费者

package com.rabbitmq.test;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;


public class ConsumerTest {
    static String QUEUE_NAME = "queue_name";

    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("10.12.24.104");
        factory.setPort(5672);
        factory.setUsername("kreditplus_dev");
        factory.setPassword("kreditplus_dev");
        //factory.setVirtualHost("/");
        Connection connection = null;
        try {
            connection = factory.newConnection();
            final Channel channel = connection.createChannel();
            //设置客户端最多接收未被ack的消息的个数
            channel.basicQos(3);
            System.out.println("aaaaaa");
            boolean autoAck = false;
            channel.basicConsume(QUEUE_NAME, autoAck, "myConsumerTag",
                    new DefaultConsumer(channel) {
                        @Override
                        public void handleDelivery(String consumerTag,
                                                   Envelope envelope,
                                                   AMQP.BasicProperties properties,
                                                   byte[] body)
                                throws IOException
                        {
                            String routingKey = envelope.getRoutingKey();
                            String contentType = properties.getContentType();
                            long deliveryTag = envelope.getDeliveryTag();
                            // (process the message components here ...)
                            System.out.println("aaaaaavdadfadsa"+new String(body));
                            System.out.println(deliveryTag);
                            try {
                                Thread.sleep(10000);
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                            //重点是这里最后一个参数决定了是否让消息重新回到队里,还是转发到死信队列中。
                            channel.basicNack(deliveryTag, false, false);
                            //channel.basicAck(deliveryTag, false);
                        }
                    });
//            channel.close();
//            connection.close();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }finally {

        }
    }
}

5.延迟队列

延迟队列是在死信队列的基础上实现的,可通过消息的TTL来控制,生产者消息生产时,设置死信队列和消息TTL时间,消息过期进入到死信队列。 消费者消费死信队列保证了延迟功能。

二、Rabbitmq的幂等性

生产者

package com.rabbitmq.test;

import com.rabbitmq.client.*;
import com.rabbitmq.redis.RedisPool;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;

import java.io.IOException;
import java.util.TreeMap;
import java.util.UUID;
import java.util.concurrent.TimeoutException;

public class ProducerV6 {
    static String EXCHANGE_NAME = "exchange_name";
    static String QUEUE_NAME = "queue_name";
    static String ROUTING_KEY = "root_key";
    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setUsername("admin");
        factory.setPassword("admin");
        factory.setVirtualHost("/");
        factory.setHost("192.168.0.17");
        factory.setPort(5672);
        Connection conn = null;
        final Channel channel;
        final TreeMap<Long, String> map = new TreeMap<Long, String>();
        try {
            conn = factory.newConnection();
            channel = conn.createChannel();
            channel.exchangeDeclare(EXCHANGE_NAME,"direct", true, false, null);
            channel.exchangeDeclare(EXCHANGE_NAME+"dddd","direct", true, false, null);

            channel.queueDeclare(QUEUE_NAME, true, false, false, null);
            channel.queueBind(QUEUE_NAME, EXCHANGE_NAME, ROUTING_KEY);
            channel.confirmSelect();
            channel.addReturnListener(new ReturnListener() {
                public void handleReturn(int replyCode, String replyText, String exchange, String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
                    System.out.println("replyCode:"+replyCode);
                    System.out.println("replyText:"+replyText);
                    System.out.println("exchange:"+exchange);
                    System.out.println("routingKey:"+routingKey);
                    String string = new String(body);
                    System.out.println("basic return"+string);

                }
            });
            channel.addConfirmListener(new ConfirmListener() {
                public void handleAck(long deliveryTag, boolean multiple) throws IOException {
                    System.out.println("handleAck"+deliveryTag);
                    map.remove(deliveryTag);
                }
                public void handleNack(long deliveryTag, boolean multiple) throws IOException {
                    System.out.println("handleNAck"+deliveryTag);
                    map.remove(deliveryTag);
                    // string need has primary key,
                    // repeat send
                    String string = map.get(deliveryTag);
                    channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY, true, MessageProperties.PERSISTENT_TEXT_PLAIN,
                            string.getBytes());
                }
            });
            //这里为了做实验保证幂等性,实际corrId每条消息唯一。
            String corrId = "10389234451254177779900";
            AMQP.BasicProperties props = new AMQP.BasicProperties.Builder()
                    .deliveryMode(2)
                    .correlationId(corrId)
                    .build();
            int i=1;
            while (i<20){
                String string = "hello world"+i;

                long nextNo = channel.getNextPublishSeqNo();
                channel.basicPublish(EXCHANGE_NAME,  ROUTING_KEY, false, props,
                        string.getBytes());
                i++;
                map.put(nextNo, string);
            }
            System.out.println("dddfdafdf");
            //channel.addReturnListener();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }
    }
}


消费者
消息重复发送的原因有,重试,生产者重复发送等,需要自己在消费者端保证消息的唯一性。
方法有
1.利用redis的setnx命令判断是否已经消费过,缓存时间的设置,要看内存的大小,及消息重复发送的时间间隔进行评估。
2.利用mysql的唯一健,但是高并发的情况下,对是数据库压力较大。

package com.rabbitmq.test;

import com.rabbitmq.client.*;
import com.rabbitmq.redis.RedisPool;
import redis.clients.jedis.Jedis;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class ConsumerTestV3 {
    static String QUEUE_NAME = "queue_name";
    static Jedis jedis = RedisPool.getJedis();

    public static void main(String[] args) {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("192.168.0.17");
        factory.setPort(5672);
        factory.setUsername("admin");
        factory.setPassword("admin");
        //factory.setVirtualHost("/");
        Connection connection = null;

        try {
            connection = factory.newConnection();
            final Channel channel = connection.createChannel();
            //设置客户端最多接收未被ack的消息的个数
            channel.basicQos(3);
            System.out.println("aaaaaa");
            boolean autoAck = false;
            channel.basicConsume(QUEUE_NAME, autoAck, "myConsumerTag",
                    new DefaultConsumer(channel) {
                        @Override
                        public void handleDelivery(String consumerTag,
                                                   Envelope envelope,
                                                   AMQP.BasicProperties properties,
                                                   byte[] body)
                                throws IOException
                        {
                            String routingKey = envelope.getRoutingKey();
                            String contentType = properties.getContentType();
                            long deliveryTag = envelope.getDeliveryTag();
                            String uuid = properties.getCorrelationId();
                            System.out.println(uuid);
                            //缓存超时时间需要根据业务指定
                            if(jedis.set(uuid,"success","NX","PX",30000)!=null){
                                // (process the message components here ...)
                                System.out.println(new String(body));
                                try {
                                    Thread.sleep(10000);
                                } catch (InterruptedException e) {
                                    e.printStackTrace();
                                }
                            }
                            channel.basicNack(deliveryTag, false, false);
                            //channel.basicAck(deliveryTag, false);
                        }
                    });
//            channel.close();
//            connection.close();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (TimeoutException e) {
            e.printStackTrace();
        }finally {

        }
    }
}

三、Rabbitmq的顺序性

rabbitmq的顺序性实现的话核心原理
1.按顺序的进入队列
2.按顺序的处理消息
3.重试异常一起的消息重试如何保证顺序。

1.按顺序进入队列需要保证,生产者唯一,发送消息时保证顺序,比如数据1,数据2,数据3,按顺序发送到同一队列。

2.消费者处理消息时,处理数据1,数据2,数据3时,核心是保证一个消费者线程处理数据1,数据2,数据3.

3.发送超时重试等现象数据被破坏,个人理解可通过状态机来保证。存储每个数据的三种状态。

四、Rabbitmq的集群及镜像队列

这篇文章:https://blog.csdn.net/weixin_40816738/article/details/105704335

1.rabbitmq集群原理

1.集群同步4中元数据

a. 队列元数据:队列名称和它的属性

b. 交换器元数据:交换器名称、类型和属性

c. 绑定元数据:一张简单的表格展示了如何将消息路由到队列

d. vhost元数据:为vhost内的队列、交换器和绑定提供命名空间和安全属性
Rabbitmq可靠性、幂等性、及集群高可用_第1张图片

2.rabbitmq集群采用原数据同步的原因

第一,存储空间。如果每个集群节点都拥有所有Queue的完全数据拷贝,那么每个节点的存储空间会非常大,集群的消息积压能力会非常弱(无法通过集群节点的扩容提高消息积压能力);

第二,性能。消息的发布者需要将消息复制到每一个集群节点,对于持久化消息,网络和磁盘同步复制的开销都会明显增加。

2.客户端连接队列所在节点

a. 如果有一个消息生产者或者消息消费者通过amqp-client的客户端连接至节点1进行消息的发布或者订阅,那么此时的集群中的消息收发只与节点1相关。
b.客户端连接的是非队列数据所在节点
非数据所在节点只会起到一个路由转发的作用,最终负责存储和消费的消息还是会在队列数据真正所在的节点上。

3.集群节点类型

磁盘节点:持久化存储原数据,如果是单点部署,那么一定是磁盘节点,如果磁盘节点挂掉,则不能创建队列,交换器等。
内存节点:将配置信息和原数据信息保存在内存中,通常负责整个集群与客户端的连接。

4.负载均衡模式

下图未将磁盘节点和内存节点进行区分,可理解为节点均需持久化原数据。
Rabbitmq可靠性、幂等性、及集群高可用_第2张图片
keepalived机制利用(虚拟路由冗余协议),已软件的形式实现服务的热备功能,(通常是两台linux服务器组成热备组,master和backup,同一时间只有master对外服务,master会虚拟出一个ip地址,简称VIP,这个VIP只存在于master上对外服务。如果keepalived检测到master故障,备份服务器backup自动接管VIP并成为master. keepalived将原master移除,当源master恢复后,会自动加入到热备组,默认再抢占,成为master。

1.rabbitmq镜像队列

镜像队列是基于普通的集群模式的,然后再添加一些策略,所以还是得先配置普通集群,然后才能设置镜像队列。镜像队列存在于多个节点。要实现镜像模式,需要先搭建一个普通集群模式,在这个模式的基础上再配置镜像模式以实现高可用。
镜像队列结构如下:
Rabbitmq可靠性、幂等性、及集群高可用_第3张图片
所有对mirror_queue_master的操作,会通过可靠组播GM的方式同步到各slave节点。
GM负责消息的广播,mirror_queue_slave负责回调处理,而master上的回调处理是由coordinator负责完成。mirror_queue_slave中包含了普通的BackingQueue进行消息的存储,master节点中BackingQueue包含在mirror_queue_master中由AMQQueue进行调用。
整体流程个人理解:mirror_queue_master当收到操作时,触发master对应的GM,GM收到消息并进行回调Coordinator. 同时AMQQueue也会调用BackingQueue进行master消息的持久化,
master的GM通过链表的方式传递消息,当slave的GM收到消息时,回调mirror_queue_slave进行处理,将消息持久化到或镜像队列所在BackingQueue。 此时可保证master节点在持久化数据时,数据还未刷到磁盘中,挂掉了,可通过镜像队列保证可靠性。

1.GM

GM模块实现的一种可靠的组播通讯协议,该协议能够保证组播消息的原子性,即保证组中活着的节点要么都收到消息要么都收不到。

它的实现大致如下:

将所有的节点形成一个循环链表,每个节点都会监控位于自己左右两边的节点,当有节点新增时,相邻的节点保证当前广播的消息会复制到新的节点上;当有节点失效时,相邻的节点会接管保证本次广播的消息会复制到所有的节点。在master节点和slave节点上的这些gm形成一个group,group(gm_group)的信息会记录在mnesia中。不同的镜像队列形成不同的group。消息从master节点对于的gm发出后,顺着链表依次传送到所有的节点,由于所有节点组成一个循环链表,master节点对应的gm最终会收到自己发送的消息,这个时候master节点就知道消息已经复制到所有的slave节点了。
新增节点:

Rabbitmq可靠性、幂等性、及集群高可用_第4张图片
每当一个节点加入或者重新加入(例如从网络分区中恢复过来)镜像队列,之前保存的队列内容会被清空。

1.节点的失效

slave节点失效,系统处理做些记录外几乎啥都不做。
master节点失效:
1.与客户端连接全部断开
2.选取最老的slave节点作为master, 如果此时slave为同步master数据,则数据丢失。
3.新的master重新入队列所有为没有ack的消息,此时客户端可能会有重复消息。

你可能感兴趣的:(分布式,rabbitmq)