【开源项目】消息队列XXL-MQ源码解析

消息队列XXL-MQ源码解析

项目介绍

XXL-MQ是一款轻量级分布式消息队列,拥有 “水平扩展、高可用、海量数据堆积、单机TPS过10万、毫秒级投递” 等特性, 支持 “并发消息、串行消息、广播消息、延迟消息、事务消费、失败重试、超时控制” 等消息特性。现已开放源代码,开箱即用。

源码地址

https://gitee.com/xuxueli0323/xxl-mq
【开源项目】消息队列XXL-MQ源码解析_第1张图片

源码解析

Admin启动

  1. 服务端启动。XxlMqBrokerImpl实现了InitializingBean,重写了afterPropertiesSet方法。
    @Override
    public void afterPropertiesSet() throws Exception {
        // init server
        initServer();

        // init thread
        initThead();
    }

  1. 初始化服务器,XxlMqBrokerImpl#initServer。添加providerFactory.addService(IXxlMqBroker.class.getName(), null, this);。将IXxlMqBroker的类名作为key,将this对象作为value存到serviceMap中。
    public void initServer() throws Exception {

        // address, static registry
        ip = (ip!=null&&ip.trim().length()>0)?ip:IpUtil.getIp();
        String address = IpUtil.getIpPort(ip, port);

        XxlCommonRegistryData xxlCommonRegistryData = new XxlCommonRegistryData();
        xxlCommonRegistryData.setKey(IXxlMqBroker.class.getName());
        xxlCommonRegistryData.setValue(address);
        XxlCommonRegistryServiceImpl.staticRegistryData = xxlCommonRegistryData;


        // init server
        providerFactory = new XxlRpcProviderFactory();
        providerFactory.initConfig(NetEnum.NETTY, Serializer.SerializeEnum.HESSIAN.getSerializer(), ip, port, null, null, null);

        // add server
        providerFactory.addService(IXxlMqBroker.class.getName(), null, this);

        // start server
        providerFactory.start();
    }
	public void addService(String iface, String version, Object serviceBean){
		String serviceKey = makeServiceKey(iface, version);
		serviceData.put(serviceKey, serviceBean);

		logger.info(">>>>>>>>>>> xxl-rpc, provider factory add service success. serviceKey = {}, serviceBean = {}", serviceKey, serviceBean.getClass());
	}
  1. 因为网络类型是NetEnum.NETTY,所以会获取到NettyServerserver = netType.serverClass.newInstance();。执行NettyServer#start。该服务器使用NettyServerHandler作为消息的处理器。当有消息过来,调用xxlRpcProviderFactory.invokeService(xxlRpcRequest),后面细讲。
    @Override
    public void channelRead0(final ChannelHandlerContext ctx, final XxlRpcRequest xxlRpcRequest) throws Exception {

        try {
            // do invoke
            serverHandlerPool.execute(new Runnable() {
                @Override
                public void run() {
                    // invoke + response
                    XxlRpcResponse xxlRpcResponse = xxlRpcProviderFactory.invokeService(xxlRpcRequest);

                    ctx.writeAndFlush(xxlRpcResponse);
                }
            });
        } catch (Exception e) {
            // catch error
            XxlRpcResponse xxlRpcResponse = new XxlRpcResponse();
            xxlRpcResponse.setRequestId(xxlRpcRequest.getRequestId());
            xxlRpcResponse.setErrorMsg(ThrowableUtil.toString(e));

            ctx.writeAndFlush(xxlRpcResponse);
        }
    }
  1. 服务器启动,还会初始化线程,XxlMqBrokerImpl#initThead,主要是将newMessageQueue中的消息存储到数据库中。
try {
                            XxlMqMessage message = newMessageQueue.take();
                            if (message != null) {
                                // load
                                List<XxlMqMessage> messageList = new ArrayList<>();
                                messageList.add(message);

                                List<XxlMqMessage> otherMessageList = new ArrayList<>();
                                int drainToNum = newMessageQueue.drainTo(otherMessageList, 100);
                                if (drainToNum > 0) {
                                    messageList.addAll(otherMessageList);
                                }

                                // save
                                xxlMqMessageDao.save(messageList);
                            }
                        } catch (Exception e) {
                            if (!executorStoped) {
                                logger.error(e.getMessage(), e);
                            }
                        }

客户端启动

  1. XxlMqSpringClientFactory实现了ApplicationContextAware,重写方法setApplicationContext。找到带有@MqConsumer注解的Bean,存储到serviceMap;另外,初始化xxlMqClientFactory
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {

        // load consumer from spring
        List<IMqConsumer> consumerList = new ArrayList<>();

        Map<String, Object> serviceMap = applicationContext.getBeansWithAnnotation(MqConsumer.class);
        if (serviceMap!=null && serviceMap.size()>0) {
            for (Object serviceBean : serviceMap.values()) {
                if (serviceBean instanceof IMqConsumer) {
                    consumerList.add((IMqConsumer) serviceBean);
                }
            }
        }

        // init
        xxlMqClientFactory = new XxlMqClientFactory();

        xxlMqClientFactory.setAdminAddress(adminAddress);
        xxlMqClientFactory.setAccessToken(accessToken);
        xxlMqClientFactory.setConsumerList(consumerList);

        xxlMqClientFactory.init();
    }
  1. XxlMqSpringClientFactory初始化。
    public void init() {
        // pre : valid consumer
        validConsumer();

        // start BrokerService
        startBrokerService();

        // start consumer
        startConsumer();
    }
  1. XxlMqClientFactory#validConsumer,校验消费者是否合法。如果是合法的,会封装成ConsumerThread,加入consumerRespository
    private void validConsumer(){
        // valid
        if (consumerList==null || consumerList.size()==0) {
            logger.warn(">>>>>>>>>>> xxl-mq, MqConsumer not found.");
            return;
        }

        // make ConsumerThread
        for (IMqConsumer consumer : consumerList) {

            // valid annotation
            MqConsumer annotation = consumer.getClass().getAnnotation(MqConsumer.class);
            if (annotation == null) {
                throw new RuntimeException("xxl-mq, MqConsumer("+ consumer.getClass() +"), annotation is not exists.");
            }

            // valid group
            if (annotation.group()==null || annotation.group().trim().length()==0) {
                // empty group means consume broadcase message, will replace by uuid
                try {
                    // annotation memberValues
                    InvocationHandler invocationHandler = Proxy.getInvocationHandler(annotation);
                    Field mValField = invocationHandler.getClass().getDeclaredField("memberValues");
                    mValField.setAccessible(true);
                    Map memberValues = (Map) mValField.get(invocationHandler);

                    // set data for "group"
                    String randomGroup = UUID.randomUUID().toString().replaceAll("-", "");
                    memberValues.put("group", randomGroup);
                } catch (Exception e) {
                    throw new RuntimeException("xxl-mq, MqConsumer("+ consumer.getClass() +"), group empty and genereta error.");
                }

            }
            if (annotation.group()==null || annotation.group().trim().length()==0) {
                throw new RuntimeException("xxl-mq, MqConsumer("+ consumer.getClass() +"),group is empty.");
            }

            // valid topic
            if (annotation.topic()==null || annotation.topic().trim().length()==0) {
                throw new RuntimeException("xxl-mq, MqConsumer("+ consumer.getClass() +"), topic is empty.");
            }

            // consumer map
            consumerRespository.add(new ConsumerThread(consumer));
        }
    }
  1. XxlMqClientFactory#startBrokerServiceXxlRpcInvokerFactory#start,设置了xxlMqBroker,开启了多个线程。
    public void startBrokerService() {
        // init XxlRpcInvokerFactory
        xxlRpcInvokerFactory = new XxlRpcInvokerFactory(XxlRegistryServiceRegistry.class, new HashMap<String, String>(){{
            put(XxlRegistryServiceRegistry.XXL_REGISTRY_ADDRESS, adminAddress);
            put(XxlRegistryServiceRegistry.ACCESS_TOKEN, accessToken);
        }});
        try {
            xxlRpcInvokerFactory.start();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

        // init ConsumerRegistryHelper
        XxlRegistryServiceRegistry commonServiceRegistry = (XxlRegistryServiceRegistry) xxlRpcInvokerFactory.getServiceRegistry();
        consumerRegistryHelper = new ConsumerRegistryHelper(commonServiceRegistry);

        // init IXxlMqBroker
        xxlMqBroker = (IXxlMqBroker) new XxlRpcReferenceBean(
                NetEnum.NETTY,
                Serializer.SerializeEnum.HESSIAN.getSerializer(),
                CallType.SYNC,
                LoadBalance.ROUND,
                IXxlMqBroker.class,
                null,
                10000,
                null,
                null,
                null,
                xxlRpcInvokerFactory).getObject();
        // ...添加线程处理
    }
  1. XxlRpcInvokerFactory#start,主要初始化了XxlRegistryClient。服务注册调用的是"/api/registry",另外开启发现进程,调用接口/api/monitor,如果获取到true,会调用"/api/discovery"接口,会将主题的数据存储到discoveryData

  2. xxlMqBroker = (IXxlMqBroker) new XxlRpcReferenceBean(...).getObject();。构造方法中初始化了NettyClient,当需要服务发现的时候,用netty进行通信。用了动态代理。

  3. XxlMqClientFactory#startConsumer,注册消费者,开启线程。

    private void startConsumer() {

        // valid
        if (consumerRespository ==null || consumerRespository.size()==0) {
            return;
        }

        // registry consumer
        getConsumerRegistryHelper().registerConsumer(consumerRespository);

        // execute thread
        for (ConsumerThread item: consumerRespository) {
            clientFactoryThreadPool.execute(item);
            logger.info(">>>>>>>>>>> xxl-mq, consumer init success, , topic:{}, group:{}", item.getMqConsumer().topic(), item.getMqConsumer().group());
        }

    }

发消息

  1. 发送消息。会调用XxlMqClientFactory#addMessages
		String topic = "topic_1";
		String data = "时间戳:" + System.currentTimeMillis();
		XxlMqProducer.produce(new XxlMqMessage(topic, data));
  1. XxlMqClientFactory#addMessages,没有指定则是异步,添加到newMessageQueue
    public static void addMessages(XxlMqMessage mqMessage, boolean async){
        if (async) {
            // async queue, mult send
            newMessageQueue.add(mqMessage);
        } else {
            // sync rpc, one send
            xxlMqBroker.addMessages(Arrays.asList(mqMessage));
        }

    }
  1. 客户端消费队列,还是调用xxlMqBroker.addMessages(messageList);xxlMqBroker是代理对象,会封装成XxlRpcRequest发送给服务端。
try {
                            XxlMqMessage message = newMessageQueue.take();
                            if (message != null) {
                                // load
                                List<XxlMqMessage> messageList = new ArrayList<>();
                                messageList.add(message);

                                List<XxlMqMessage> otherMessageList = new ArrayList<>();
                                int drainToNum = newMessageQueue.drainTo(otherMessageList, 100);
                                if (drainToNum > 0) {
                                    messageList.addAll(otherMessageList);
                                }

                                // save
                                xxlMqBroker.addMessages(messageList);
                            }
                        } catch (Exception e) {
                            if (!XxlMqClientFactory.clientFactoryPoolStoped) {
                                logger.error(e.getMessage(), e);
                            }
                        }
  1. 服务端xxlRpcProviderFactory.invokeService(xxlRpcRequest),根据xxlRpcRequest的方法名和类名,获取到对应的bean,反射调用方法。即执行XxlMqBrokerImpl#addMessages
    @Override
    public int addMessages(List<XxlMqMessage> messages) {
        newMessageQueue.addAll(messages);
        return messages.size();
    }
  1. 服务端消费队列,存储到数据库中。
                        try {
                            XxlMqMessage message = newMessageQueue.take();
                            if (message != null) {
                                // load
                                List<XxlMqMessage> messageList = new ArrayList<>();
                                messageList.add(message);

                                List<XxlMqMessage> otherMessageList = new ArrayList<>();
                                int drainToNum = newMessageQueue.drainTo(otherMessageList, 100);
                                if (drainToNum > 0) {
                                    messageList.addAll(otherMessageList);
                                }

                                // save
                                xxlMqMessageDao.save(messageList);
                            }
                        } catch (Exception e) {
                            if (!executorStoped) {
                                logger.error(e.getMessage(), e);
                            }
                        }

监听消息

  1. 核心就是ConsumerThread。判断当前线程是否活跃,ConsumerRegistryHelper.ActiveInfo activeInfo = XxlMqClientFactory.getConsumerRegistryHelper().isActice(this);,用于判断是否有新内容。
  2. ConsumerRegistryHelper#isActice方法中,会执行TreeSet onlineConsumerSet = serviceRegistry.discovery(registryKey);,主要用于获取缓存discoveryDatadiscoveryData获取前,会判断主题。
  3. 拉取消息列表,List messageList = XxlMqClientFactory.getXxlMqBroker().pullNewMessage(mqConsumer.topic(), mqConsumer.group(), activeInfo.rank, activeInfo.total, 100);。同样,XxlMqClientFactory.getXxlMqBroker()获取到的是代理类,会调用netty发送远程请求,服务端执行XxlMqBrokerImpl#pullNewMessage。服务端是从数据库中获取消息列表。
    @Override
    public List<XxlMqMessage> pullNewMessage(String topic, String group, int consumerRank, int consumerTotal, int pagesize) {
        List<XxlMqMessage> list = xxlMqMessageDao.pullNewMessage(XxlMqMessageStatus.NEW.name(), topic, group, consumerRank, consumerTotal, pagesize);
        return list;
    }
		SELECT <include refid="Base_Column_List" />
		FROM xxl_mq_message AS t
		WHERE 	t.topic = #{topic}
			AND t.group = #{group}
			AND t.status = #{newStatus}
			AND t.effectTime  NOW()
    `

服务发现

  1. 客户端执行XxlRegistryBaseClient#discovery
  2. admin服务端提供发现接口,ApiController#discovery,主要是读取文件, String fileName = registryDataFilePath.concat(File.separator).concat(key).concat(".properties");,本地路径:D:\data\applogs\xxl-mq\registrydata

服务注册

  1. ApiController#registry,主要是往XxlCommonRegistryServiceImpl#registryQueue里面添加信息。
  2. XxlCommonRegistryServiceImpl实现了InitializingBean,重写XxlCommonRegistryServiceImpl#afterPropertiesSet中启动线程处理registryQueue,保存注册信息入库。
                    try {
                        XxlCommonRegistryData xxlCommonRegistryData = registryQueue.take();
                        if (xxlCommonRegistryData !=null) {

                            // refresh or add
                            int ret = xxlCommonRegistryDataDao.refresh(xxlCommonRegistryData);
                            if (ret == 0) {
                                xxlCommonRegistryDataDao.add(xxlCommonRegistryData);
                            }

                            // valid file status
                            XxlCommonRegistry fileXxlCommonRegistry = getFileRegistryData(xxlCommonRegistryData);
                            if (fileXxlCommonRegistry!=null && fileXxlCommonRegistry.getDataList().contains(xxlCommonRegistryData.getValue())) {
                                continue;     // "Repeated limited."
                            }

                            // checkRegistryDataAndSendMessage
                            checkRegistryDataAndSendMessage(xxlCommonRegistryData);
                        }
                    } catch (Exception e) {
                        if (!executorStoped) {
                            logger.error(e.getMessage(), e);
                        }
                    }
  1. 处理注册信息,保存到文件中。
                    try {
                        // new message, filter readed
                        List<XxlCommonRegistryMessage> messageList = xxlCommonRegistryMessageDao.findMessage(readedMessageIds);
                        if (messageList!=null && messageList.size()>0) {
                            for (XxlCommonRegistryMessage message: messageList) {
                                readedMessageIds.add(message.getId());

                                // from registry、add、update、deelete,ne need sync from db, only write
                                XxlCommonRegistry xxlCommonRegistry = JacksonUtil.readValue(message.getData(), XxlCommonRegistry.class);

                                // default, sync from db (aready sync before message, only write)

                                // sync file
                                setFileRegistryData(xxlCommonRegistry);
                            }
                        }

                        // clean old message;
                        if (System.currentTimeMillis() % registryBeatTime ==0) {
                            xxlCommonRegistryMessageDao.cleanMessage(10);
                            readedMessageIds.clear();
                        }
                    } catch (Exception e) {
                        if (!executorStoped) {
                            logger.error(e.getMessage(), e);
                        }
                    }

你可能感兴趣的:(讲解开源项目,开源,java,开发语言)