OpenStack-RPC-server的构建(四)

        本章我们分析通过oslo_messaging层向下创建consumer的代码流程，在创建consumer之前，首先利用上篇文章所说的创建的connection作为参数创建一个AMQPListener对象，将该对象赋给创建consumer的callback参数，所以当有消息到达时，调用AMQPListener类的__call__方法。这里需要注意的是conn =self._get_connection(rpc_amqp.PURPOSE_LISTEN)，该对象是ConnectionContext对象，ConnectionContext类其实相当于所有oslo_messaging层的connection的委托代理类。

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py
def listen(self, target):
        conn = self._get_connection(rpc_amqp.PURPOSE_LISTEN)

        listener = AMQPListener(self, conn)

        conn.declare_topic_consumer(exchange_name=self._get_exchange(target),
                                    topic=target.topic,
                                    callback=listener)
        conn.declare_topic_consumer(exchange_name=self._get_exchange(target),
                                    topic='%s.%s' % (target.topic,
                                                     target.server),
                                    callback=listener)
        conn.declare_fanout_consumer(target.topic, listener)

    return listener
class AMQPListener(base.Listener):

    def __init__(self, driver, conn):
        super(AMQPListener, self).__init__(driver)
        self.conn = conn
        self.msg_id_cache = rpc_amqp._MsgIdCache()
        self.incoming = []
        self._stopped = threading.Event()
    #usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:ConnectionContext
    def __getattr__(self, key):
        """Proxy all other calls to the Connection instance."""
        if self.connection:
            return getattr(self.connection, key)
        else:
            raise rpc_common.InvalidRPCConnectionReuse()

        根据上述的说明，我们知道ConnectionContext类其实相当于所有oslo_messaging层的connection的委托代理类。在利用ConnectionContext对象访问一个并不存在的属性时(如conn.declare_topic_consumer(…))，将调用ConnectionContext类的__getattr__方法，__getattr__方法实际上是一个回滚(fallback)方法，它只会在某个属性没有找到的时候才会被调用。

        因此当我们执行conn.declare_topic_consumer(…)方法时，由于ConnectionContext类中并没有declare_topic_consumer方法，所以将会执行__getattr__方法。根据我们前面的分析self.connection是/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py中的Connection对象，所以将返回Connection对象的declare_topic_consumer地址(getattr(self.connection,key),就拿这里的例子来说，key在这里是declare_topic_consumer)，然后执行conn.declare_topic_consumer(…)时，将把相应的参数传递进去去执行相应的操作。

    #/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py
    def declare_topic_consumer(self, exchange_name, topic, callback=None,
                               queue_name=None):
        """Create a 'topic' consumer."""
        self.declare_consumer(functools.partial(TopicConsumer,
                                                name=queue_name,
                                                exchange_name=exchange_name,
                                                ),
                              topic, callback)

    def declare_fanout_consumer(self, topic, callback):
        """Create a 'fanout' consumer."""
        self.declare_consumer(FanoutConsumer, topic, callback)

    def declare_consumer(self, consumer_cls, topic, callback):
        """Create a Consumer using the class that was passed in and
        add it to our list of consumers
        """

        def _connect_error(exc):
            log_info = {'topic': topic, 'err_str': exc}
            LOG.error(_("Failed to declare consumer for topic '%(topic)s': "
                      "%(err_str)s"), log_info)

        def _declare_consumer():
            #这里self.consumer_num=count(1)，它是一个迭代器对象，当第一次执行next时，
            #将返回1，执行多次next时，将会无限累加。其作用是用于为同一个channel上的
            #多个consumer打上不同的tag。比如在Nova-scheduler组件中，同一个channel上有
            #三个consumer，其tag分别为1,2和3。
            consumer = consumer_cls(self.driver_conf, self.channel, topic,
                                    callback, six.next(self.consumer_num))
            self.consumers.append(consumer)
            return consumer

        with self._connection_lock:
            return self.ensure(_declare_consumer,
                               error_callback=_connect_error)

        首先解释functools.partial()的用法，函数partial()允许我们给一个或多个参数指定固定的值，以此减少需要提供给之后调用的参数数量，并返回一个全新的可调用对象。所以declare_topic_consumer方法传递给declare_consumer方法的consumer_cls为TopicConsumer方法名，且该方法的name和exchange_name参数利用partial()函数已经提前设置完成。因此在_declare_consumer()方法中，构造TopicConsumer对象时，我们不需指定name和exchange_name参数。顺便提一句，在declare_topic_consumer方法的exchange_name=self._get_exchange(target)

#/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py
def _get_exchange(self, target):
        return target.exchange or self._default_exchange

        由于我们在oslo_messaging层创建RPC-server的Target时，只指定了topic和server参数，并未指定exchange参数(exchange参数在构造Target可选)，因此这里取的是self._default_exchange的值，该值是在构造RabbitDriver对象传递下来的，该exchange的值为：’nova’。

        我们继续回到declare_topic_consumer和declare_fanout_consumer方法，这里listen方法中调用declare_topic_consumer两次，且callback参数为AMQPListener对象，拿Nova-scheduler组件而言，两个consumer的topic分别为scheduler和scheduler.jun(target.topic=scheduler，target.server=jun)。然后declare_topic_consumer和declare_fanout_consumer方法都会调用declare_consumer方法，且declare_consumer方法将调用oslo_messaging层的ensure方法，对于ensure方法的代码流程请看上一篇文章，因为我们的connection和channel已经创建完成，所以这里不会再次创建，同时，通过钩子函数execute_method(oslo_messaging层)回调去执行_declare_consumer方法，在_declare_consumer方法去将去构造相应的consumer对象。由于构造topic和fanout的consumer对象的代码流程相似，所以我们主要分析topic方式的代码流程。

#/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py
class TopicConsumer(ConsumerBase):
    """Consumer class for 'topic'."""

    def __init__(self, conf, channel, topic, callback, tag, exchange_name,
                 name=None, **kwargs):
        """Init a 'topic' queue.

        :param channel: the amqp channel to use
        :param topic: the topic to listen on
        :paramtype topic: str
        :param callback: the callback to call when messages are received
        :param tag: a unique ID for the consumer on the channel
        :param exchange_name: the exchange name to use
        :param name: optional queue name, defaults to topic
        :paramtype name: str

        Other kombu options may be passed as keyword arguments
        """
        # Default options
        options = {'durable': conf.amqp_durable_queues,
                   'queue_arguments': _get_queue_arguments(conf),
                   'auto_delete': conf.amqp_auto_delete,
                   'exclusive': False}
        options.update(kwargs)
        exchange = kombu.entity.Exchange(name=exchange_name,
                                         type='topic',
                                         durable=options['durable'],
                                         auto_delete=options['auto_delete'])
        super(TopicConsumer, self).__init__(channel,
                                            callback,
                                            tag,
                                            name=name or topic,
                                            exchange=exchange,
                                            routing_key=topic,
                                            **options)

Topic方式的consumer创建的代码流程主要分为3部分：

a. kombu层的exchange对象的创建

b. kombu层的queue对象的创建.

c. 通过kombu层的exchange和queue对象的信息，创建amqp层的exchange和queue对象，且绑定amqp层的exchange和queue对象。

#/usr/lib/python2.7/site-packages/kombu/entity.py
class Exchange(MaybeChannelBound):
    """An Exchange declaration.

    :keyword name: See :attr:`name`.
    :keyword type: See :attr:`type`.
    :keyword channel: See :attr:`channel`.
    :keyword durable: See :attr:`durable`.
    :keyword auto_delete: See :attr:`auto_delete`.
    :keyword delivery_mode: See :attr:`delivery_mode`.
    :keyword arguments: See :attr:`arguments`.

    .. attribute:: name

        Name of the exchange. Default is no name (the default exchange).

    .. attribute:: type

        AMQP defines four default exchange types (routing algorithms) that
        covers most of the common messaging use cases. An AMQP broker can
        also define additional exchange types, so see your broker
        manual for more information about available exchange types.

            * `direct` (*default*)

                Direct match between the routing key in the message, and the
                routing criteria used when a queue is bound to this exchange.

            * `topic`

                Wildcard match between the routing key and the routing pattern
                specified in the exchange/queue binding. The routing key is
                treated as zero or more words delimited by `"."` and
                supports special wildcard characters. `"*"` matches a
                single word and `"#"` matches zero or more words.

            * `fanout`

                Queues are bound to this exchange with no arguments. Hence any
                message sent to this exchange will be forwarded to all queues
                bound to this exchange.

            * `headers`

                Queues are bound to this exchange with a table of arguments
                containing headers and values (optional). A special argument
                named "x-match" determines the matching algorithm, where
                `"all"` implies an `AND` (all pairs must match) and
                `"any"` implies `OR` (at least one pair must match).

                :attr:`arguments` is used to specify the arguments.

            This description of AMQP exchange types was shamelessly stolen
            from the blog post `AMQP in 10 minutes: Part 4`_ by
            Rajith Attapattu. This article is recommended reading.

            .. _`AMQP in 10 minutes: Part 4`:
                http://bit.ly/amqp-exchange-types

    .. attribute:: channel

        The channel the exchange is bound to (if bound).

    .. attribute:: durable

        Durable exchanges remain active when a server restarts. Non-durable
        exchanges (transient exchanges) are purged when a server restarts.
        Default is :const:`True`.

    .. attribute:: auto_delete

        If set, the exchange is deleted when all queues have finished
        using it. Default is :const:`False`.

    .. attribute:: delivery_mode

        The default delivery mode used for messages. The value is an integer,
        or alias string.

            * 1 or `"transient"`

                The message is transient. Which means it is stored in
                memory only, and is lost if the server dies or restarts.

            * 2 or "persistent" (*default*)
                The message is persistent. Which means the message is
                stored both in-memory, and on disk, and therefore
                preserved if the server dies or restarts.

        The default value is 2 (persistent).

    .. attribute:: arguments

        Additional arguments to specify when the exchange is declared.

    """
    TRANSIENT_DELIVERY_MODE = TRANSIENT_DELIVERY_MODE
    PERSISTENT_DELIVERY_MODE = PERSISTENT_DELIVERY_MODE

    name = ''
    type = 'direct'
    durable = True
    auto_delete = False
    passive = False
    delivery_mode = PERSISTENT_DELIVERY_MODE

    attrs = (
        ('name', None),
        ('type', None),
        ('arguments', None),
        ('durable', bool),
        ('passive', bool),
        ('auto_delete', bool),
        ('delivery_mode', lambda m: DELIVERY_MODES.get(m) or m),
    )

    def __init__(self, name='', type='', channel=None, **kwargs):
        super(Exchange, self).__init__(**kwargs)
        self.name = name or self.name
        self.type = type or self.type
        self.maybe_bind(channel)

#/usr/lib/python2.7/site-packages/kombu/abstract.py
def maybe_bind(self, channel):
        """Bind instance to channel if not already bound."""
        if not self.is_bound and channel:
            self._channel = maybe_channel(channel)
            self.when_bound()
            self._is_bound = True
        return self

        上述代码是创建kombu层的exchange对象的代码，其中对于Nova-scheduler组件而言，name分别为scheduler和scheduler.jun(因为有两个topic方式的consumer)，这里channel参数并未传递，所以在这里channel采用默认值None，所以直接返回self。但是当创建kombu层的queue将返回amqp层的channel，后面我们将分析。

        在创建了kombu层的exchange后，TopicConsumer类将调用父类的初始化代码(__init__)，这段代码内包括创建kombu层的queue对象，amqp层的exchange和queue对象的创建和绑定。

#/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py
class ConsumerBase(object):
    """Consumer base class."""

    def __init__(self, channel, callback, tag, **kwargs):
        """Declare a queue on an amqp channel.

        'channel' is the amqp channel to use
        'callback' is the callback to call when messages are received
        'tag' is a unique ID for the consumer on the channel

        queue name, exchange name, and other kombu options are
        passed in here as a dictionary.
        """
        self.callback = callback
        self.tag = six.text_type(tag)
        self.kwargs = kwargs
        self.queue = None
        self.reconnect(channel)

    def reconnect(self, channel):
        """Re-declare the queue after a rabbit reconnect."""
        self.channel = channel
        self.kwargs['channel'] = channel
        self.queue = kombu.entity.Queue(**self.kwargs)
        try:
            self.queue.declare()
        except Exception as e:
            # NOTE: This exception may be triggered by a race condition.
            # Simply retrying will solve the error most of the time and
            # should work well enough as a workaround until the race condition
            # itself can be fixed.
            # TODO(jrosenboom): In order to be able to match the Exception
            # more specifically, we have to refactor ConsumerBase to use
            # 'channel_errors' of the kombu connection object that
            # has created the channel.
            # See https://bugs.launchpad.net/neutron/+bug/1318721 for details.
            LOG.error(_("Declaring queue failed with (%s), retrying"), e)
            self.queue.declare()

        首先创建kombu层的queue对象。代码流程如下。

#/usr/lib/python2.7/site-packages/kombu/entity.py
class Queue(MaybeChannelBound):
    """A Queue declaration.

    :keyword name: See :attr:`name`.
    :keyword exchange: See :attr:`exchange`.
    :keyword routing_key: See :attr:`routing_key`.
    :keyword channel: See :attr:`channel`.
    :keyword durable: See :attr:`durable`.
    :keyword exclusive: See :attr:`exclusive`.
    :keyword auto_delete: See :attr:`auto_delete`.
    :keyword queue_arguments: See :attr:`queue_arguments`.
    :keyword binding_arguments: See :attr:`binding_arguments`.
    :keyword on_declared: See :attr:`on_declared`

    .. attribute:: name

        Name of the queue. Default is no name (default queue destination).

    .. attribute:: exchange

        The :class:`Exchange` the queue binds to.

    .. attribute:: routing_key

        The routing key (if any), also called *binding key*.

        The interpretation of the routing key depends on
        the :attr:`Exchange.type`.

            * direct exchange

                Matches if the routing key property of the message and
                the :attr:`routing_key` attribute are identical.

            * fanout exchange

                Always matches, even if the binding does not have a key.

            * topic exchange

                Matches the routing key property of the message by a primitive
                pattern matching scheme. The message routing key then consists
                of words separated by dots (`"."`, like domain names), and
                two special characters are available; star (`"*"`) and hash
                (`"#"`). The star matches any word, and the hash matches
                zero or more words. For example `"*.stock.#"` matches the
                routing keys `"usd.stock"` and `"eur.stock.db"` but not
                `"stock.nasdaq"`.

    .. attribute:: channel

        The channel the Queue is bound to (if bound).

    .. attribute:: durable

        Durable queues remain active when a server restarts.
        Non-durable queues (transient queues) are purged if/when
        a server restarts.
        Note that durable queues do not necessarily hold persistent
        messages, although it does not make sense to send
        persistent messages to a transient queue.

        Default is :const:`True`.

    .. attribute:: exclusive

        Exclusive queues may only be consumed from by the
        current connection. Setting the 'exclusive' flag
        always implies 'auto-delete'.

        Default is :const:`False`.

    .. attribute:: auto_delete

        If set, the queue is deleted when all consumers have
        finished using it. Last consumer can be cancelled
        either explicitly or because its channel is closed. If
        there was no consumer ever on the queue, it won't be
        deleted.

    .. attribute:: queue_arguments

        Additional arguments used when declaring the queue.

    .. attribute:: binding_arguments

        Additional arguments used when binding the queue.

    .. attribute:: alias

        Unused in Kombu, but applications can take advantage of this.
        For example to give alternate names to queues with automatically
        generated queue names.

    .. attribute:: on_declared

        Optional callback to be applied when the queue has been
        declared (the ``queue_declare`` method returns).
        This must be function with a signature that accepts at least 3
        positional arguments: ``(name, messages, consumers)``.

    """
    name = ''
    exchange = Exchange('')
    routing_key = ''

    durable = True
    exclusive = False
    auto_delete = False
    no_ack = False

    attrs = (
        ('name', None),
        ('exchange', None),
        ('routing_key', None),
        ('queue_arguments', None),
        ('binding_arguments', None),
        ('durable', bool),
        ('exclusive', bool),
        ('auto_delete', bool),
        ('no_ack', None),
        ('alias', None),
        ('bindings', list),
    )

    def __init__(self, name='', exchange=None, routing_key='',
                 channel=None, bindings=None, on_declared=None,
                 **kwargs):
        super(Queue, self).__init__(**kwargs)
        self.name = name or self.name
        self.exchange = exchange or self.exchange
        self.routing_key = routing_key or self.routing_key
        self.bindings = set(bindings or [])
        self.on_declared = on_declared

        # allows Queue('name', [binding(...), binding(...), ...])
        if isinstance(exchange, (list, tuple, set)):
            self.bindings |= set(exchange)
        if self.bindings:
            self.exchange = None

        # exclusive implies auto-delete.
        if self.exclusive:
            self.auto_delete = True
        self.maybe_bind(channel)

#/usr/lib/python2.7/site-packages/kombu/abstract.py
def maybe_bind(self, channel):
        """Bind instance to channel if not already bound."""
        if not self.is_bound and channel:
            self._channel = maybe_channel(channel)
            self.when_bound()
            self._is_bound = True
        return self
#/usr/lib/python2.7/site-packages/kombu/connection.py
def maybe_channel(channel):
    """Returns channel, or returns the default_channel if it's a
    connection."""
    if isinstance(channel, Connection):
        return channel.default_channel
    return channel

        由于在创建kombu层的queue对象时，从oslo_messaging层传递了channel下来，且对Nova-scheduler组件而言，channel对象不是kombu层的Connection对象，它是amqp层的channel对象，所以不会创建default_channel对象。因此self._channel的值为amqp层的channel对象。后面绑定exchange和queue会利用self._channel。

        此时kombu层的queue对象创建完成，下面解释amqp层的exchange和queue对象的创建以及绑定。其代码为self.queue.declare()(在TopicConsumer类的父类的初始化代码中)触发的。

#/usr/lib/python2.7/site-packages/kombu/entity.py:Queue
def declare(self, nowait=False):
        """Declares the queue, the exchange and binds the queue to
        the exchange."""
        # - declare main binding.
        if self.exchange:
            self.exchange.declare(nowait)
        self.queue_declare(nowait, passive=False)

        if self.exchange and self.exchange.name:
            self.queue_bind(nowait)

        # - declare extra/multi-bindings.
        for B in self.bindings:
            B.declare(self.channel)
            B.bind(self, nowait=nowait)
        return self.name

    #/usr/lib/python2.7/site-packages/kombu/entity.py:Exchange
    def declare(self, nowait=False, passive=None):
        """Declare the exchange.

        Creates the exchange on the broker.

        :keyword nowait: If set the server will not respond, and a
            response will not be waited for. Default is :const:`False`.

        """
        passive = self.passive if passive is None else passive
        if self.name:
            return self.channel.exchange_declare(
                exchange=self.name, type=self.type, durable=self.durable,
                auto_delete=self.auto_delete, arguments=self.arguments,
                nowait=nowait, passive=passive,
            )

    #/usr/lib/python2.7/site-packages/kombu/entity.py:Queue
    def queue_declare(self, nowait=False, passive=False):
        """Declare queue on the server.

        :keyword nowait: Do not wait for a reply.
        :keyword passive: If set, the server will not create the queue.
            The client can use this to check whether a queue exists
            without modifying the server state.

        """
        ret = self.channel.queue_declare(queue=self.name,
                                         passive=passive,
                                         durable=self.durable,
                                         exclusive=self.exclusive,
                                         auto_delete=self.auto_delete,
                                         arguments=self.queue_arguments,
                                         nowait=nowait)
        if not self.name:
            self.name = ret[0]
        if self.on_declared:
            self.on_declared(*ret)
        return ret

    #/usr/lib/python2.7/site-packages/kombu/entity.py:Queue
    def queue_bind(self, nowait=False):
        """Create the queue binding on the server."""
        return self.bind_to(self.exchange, self.routing_key,
                            self.binding_arguments, nowait=nowait)

    def bind_to(self, exchange='', routing_key='',
                arguments=None, nowait=False):
        if isinstance(exchange, Exchange):
            exchange = exchange.name
        return self.channel.queue_bind(queue=self.name,
                                       exchange=exchange,
                                       routing_key=routing_key,
                                       arguments=arguments,
                                       nowait=nowait)

    #上述的self.channel是下面的含有@property的channel方法，该方法是exchange和queue的
    #相同的父类MaybeChannelBound的方法。返回的是amqp层的channel对象。即self._channel
    #/usr/lib/python2.7/site-packages/kombu/abstract.py
    @property
    def channel(self):
        """Current channel if the object is bound."""
        channel = self._channel
        if channel is None:
            raise NotBoundError(
                "Can't call method on %s not bound to a channel" % (
                    self.__class__.__name__))
        if isinstance(channel, ChannelPromise):
            channel = self._channel = channel()
        return channel

        上述便是通过kombu层向下创建amqp层的exchange和queue对象以及它们的绑定代码流程，在这里我们就不贴amqp层的代码了。需要注意的是，两个topic方式的consumer的exchange名称都为nova，那么是否amqp层的exchange会创建两个nova名称的exchange？答案是No，因为第二次创建nova名称的exchange时，amqp层会检测保护机制。所以只会有一个nova名称的exchange。然后两个topic方法的queue将被绑定到该exchange上面。

        上述便是topic方式的代码流程，对于fanout方式的代码流程与topic方式的相同，不过fanout方式创建的exchange和queue与topic方式的不同，fanout方式的exchange在Nova-scheduler组件的名称不是nova，而是scheduler_fanout，且queue名称则是scheduler_fanout_xxxx(xxxx表示随机序列)。

总结：本文主要通过创建topic方式的consumer来讲解consumer的创建流程，其主要分为3部分：

a. kombu层的exchange对象的创建。

b. kombu层的queue对象的创建。

c. 通过kombu层的exchange和queue对象的信息，创建amqp层的exchange和queue对象，且绑定amqp层的exchange和queue对象。

# usr/lib/python2.7/site-packages/oslo_messaging/server.py
def start(self):
        """Start handling incoming messages.

        This method causes the server to begin polling the transport for
        incoming messages and passing them to the dispatcher. Message
        processing will continue until the stop() method is called.

        The executor controls how the server integrates with the applications
        I/O handling strategy - it may choose to poll for messages in a new
        process, thread or co-operatively scheduled coroutine or simply by
        registering a callback with an event loop. Similarly, the executor may
        choose to dispatch messages in a new thread, coroutine or simply the
        current thread.
        """
        if self._executor is not None:
            return
        try:
            listener = self.dispatcher._listen(self.transport)
        except driver_base.TransportDriverError as ex:
            raise ServerListenError(self.target, ex)

        self._executor = self._executor_cls(self.conf, listener,
                                            self.dispatcher)
        self._executor.start()

        目前，我们把usr/lib/python2.7/site-packages/oslo_messaging/server.py中的start方法中的listener = self.dispatcher._listen(self.transport)代码流程分析完成了。总结下来包括两部分：

1. 通过oslo_messaging层和kombu层向下创建amqp层的connection和channel对象。

2. 在connection和channel对象基础上，创建consumer(两个topic和一个fanout方式)。内部包括exchange和queue对象创建以及它们之间的binding。

        下一篇文章我们将解释剩下的两条语句的代码流程。