为了方便工作中使用,对 RabbitMQ 的【Shovel】文档进行了翻译,鉴于自己水平有限,翻译中难免有纰漏产生,如果疑问,欢迎指出探讨。此文以中英对照方式呈现。
官方原文:http://www.rabbitmq.com/shovel.html
转载请注明:http://my.oschina.net/u/617889/blog/96716
========================================
The Shovel plugin allows you to configure a number of shovels, which start automatically when the broker starts.
Shovel 插件 允许你 同时配置多个 shovel 以便在 broker 启动的时候可以随之自动启动。The high level goal of a shovel is to reliably and continually take messages from a queue (a source) in one broker and publish them to exchanges in another broker (a destination).
shovel 的高层次的 目标是, 保证可靠连续地将 message 从位于某个 broker 上的 queue (作为源端)中取出,再将其 publish 到另外一个 broker 中的相应 exchange 上(作为目的端)。The source queue and destination exchanges can be on the same broker or distinct brokers.
作为源的 queue 和作为目的的 exchange 可以同时在一个 broker 上 也可以在不同的 broker 上。A shovel behaves like a well-written client application, which connects to its source and destination, reads and writes messages, and copes with connection failures.
shovel 的行为就像编写良好的客户端应用程序,其 负责连接源和目的、负责 message 的读和写,以及负责处理连接失败问题。The primary advantages of a shovel are:
shovel 的主要 优势在于:A shovel can move messages between brokers (or clusters) in different administrative domains:
shovel 可以将位于不同管理域中的 broker (或者 cluster)上的 message 进行移动。WAN 友好
The Shovel plugin uses AMQP to communicate between brokers, and is designed to tolerate intermittent connectivity without message loss.
Shovel 插件利用 AMQP 协议在 broker 间相互通信, 并被设计成可以容忍时断时续的连通性并且保证 message 不会丢失。To define and start a shovel, you do not have to reconfigure either the source or the destination resources. You do not even have to restart the broker(s) upon which they reside: the shovels can run in a separate broker altogether.
为了使用 shovel 功能,你 不必重新配置任何源或者目的资源。你甚至 不需要重新启动这些资源所在的 broker :因为 shovel 可以运行于完全 不相关的 broker 上面。When a shovel connects (either to the source or the destination) it can be configured to perform any number of explicit methods. For example, the source queue need not exist initially, and can be declared on connect.
当 shovel 成功连接后(连接上源或者目的),可以对其进行相应配置以执行任意数量的相关方法。例如,源 queue 在最初不需要存在,可以在连接建立后声明。A comparison between clustering, federated exchanges and shovels is given on the Distributed Messaging page.
关于 clustering、federated exchange 和 shovel 的比较请查看 [ Distributed Messaging] 。
The Shovel plugin defines (and runs) an Erlang client application for each shovel defined in its configuration.
Shovel 插件会定义(和运行) Erlang 客户端应用程序,只要在相应的配置中定义了 shovel 相关信息。In essence, a shovel is a simple pump. Each shovel:
在本质上,shovel 可以类比为一个简单的水泵。每一个 shovel 都会:
The shovel configuration allows each of these processes to be tailored.
允许上述各项在 shovel 配置中随意定制。After connection to a source or a destination broker a series of configured AMQP declarations can be issued. Queues, exchanges and bindings can be declared.
在成功连接源或目的 broker 之后,就可以执行一系列相应的 AMQP 配置声明了。Queue 、exchange 以及 binding 均可以进行声明。A shovel will attempt to reconnect to a broker if a failure occurs and multiple brokers can be specified for the source and destination so that another broker may be selected (at random) to reconnect to. A reconnection delay can be specified to avoid flooding the network with reconnection attempts, or to prevent reconnection on failure altogether.
shovel 方式会保证在(当前 shovel broker)失效后尝试重新连接到(其他的 shovel) broker 上 - 可以为源端 broker 和目的端 broker 同时指定多个 shovel broker,以便从中(随机)选择要连接的 broker 。 可以设置 reconnection delay 以避免由于重连行为导致的网络泛洪,或者可以在重连失败后直接停止重连。All configured declarations (for that source or destination) are re-issued upon re-connect.
所有配置声明(针对相应的源和目的)会在重连成功后被重新发送。The shovel's consumer can acknowledge messages automatically on receipt, after (re-)publication, or after confirmation of its publication.
在 Shovel 模型中的,consumer 收到 message 并自动进行 acknowledge 动作的时机有以下几种:【re-publishes】
Both the publish method and the message properties can be modified with explicit parameter values.
可以显式地对 publish 方法和 message 属性进行修改。Full details are given in the configuration section, below.
具体细节在下面的 [ configuration] 段中给出。
The Shovel plugin is included in the RabbitMQ distribution. To enable it, use rabbitmq-plugins:
Shovel 插件会在 RabbitMQ 的发布包中包含,可以通过 [ rabbitmq-plugins] 使能:rabbitmq-plugins enable rabbitmq_shovel
You may also wish to enable the rabbitmq_shovel_management plugin (see below).
你可能同样希望使能 rabbitmq_shovel_management 插件。
There are two ways of discovering the status of configured shovels.
可以通过两种方式 查询已配置 shovel 的状态信息。
Shovel status can be reported on the Management plugin user interface by enabling the rabbitmq_shovel_management plugin wherever you have the management plugin enabled.
当你在安装了管理插件的机器上使能 rabbitmq_shovel_management 插件之后,就可以在 [ Management plugin] UI 上查询 Shovel 的状态信息。Information about configured shovels will automatically appear in the management API and UI.
已配置的 shovel 的相关信息将自动出现在管理 API 和 UI 中。
Shovel status can be obtained by direct query of the Shovel plugin app. Issue the following rabbitmqctl command:
可以通过对 Shovel 插件 app 的直接查询获取 shovel 的状态信息。可以执行如下 rabbitmqctl 命令:$ rabbitmqctl eval 'rabbit_shovel_status:status().'
This calls the status method in a module of the rabbitmq_shovel plugin, which will return an Erlang list, with one element for each configured shovel.
该命令会调用 rabbitmq_shovel 插件模块中的 status 方法,该方法将返回一个 Erlang 列表,其中每一个元素对应一个已配置好的 shovel 。Each element of the list is a tuple with three fields: {Name, Status, Timestamp}.
列表中的每一个元素都以三元组的形式构成:{Name, Status, Timestamp} 。Timestamp is a local calendar time of the form {{YYYY, MM, DD}, {HH, MM, SS}}.
Timestamp 的格式是具有 {{YYYY, MM, DD}, {HH, MM, SS}} 格式的本地时间。Status takes one of three forms:
Status 取值下面三种:The shovel is starting up, connecting and creating resources:
当 shovel 处于启动、连接和创建资源的时候状态为:
'starting'
The shovel is running normally:
当 shovel 正常运行时状态为:
{ 'running' | 'blocked', {'source', Source},{'destination', Destination}}:
where Source and Destination terms give the respective connection parameters.当 shovel 终止时:
{'terminated', Reason}
where Reason is an Erlang term that indicates the reason for the termination.
其中 Reason 是 Erlang 项式,用以表明何种原因导致的终止。The first term of a normal status can be 'running' or 'blocked':
正常状态时,第一个项式可能是 'running' 或 'blocked' :When 'blocked', the shovel will raise channel.flow to the source, asking the source to stop sending further messages to the shovel. Any messages that are received by the shovel before the source observes the channel.flow are correctly buffered and maintained in order, and are published to the destination as soon as the destination drops the channel.flow block.
当处于 'blocked' 状态时,shovel 将通过 channel.flow 对源端进行流控,要求源端停止进一步向当前 shovel 发送 message 。在源端发现自己被“流控”前,其 shovel 接收到的 message(即尚未 publist 到目的端的哪些 message)都会被可靠正确有序的缓存起来,然后当目的端不再通过 channel.flow 进行流控时,将缓存的 message 发送过去。
The configuration for the Shovel plugin in the broker configuration file is an Erlang term (as usual) and consists of a single shovels clause:
在 [ broker configuration file] 中针对 Shovel 插件的配置信息(通常情况下)属于 Erlang 项式,由单条 shovel 条目构成:
{rabbitmq_shovel, [ {shovels, [ {shovel_name, [ ... ]}, ... ]} ]}
A (deliberately verbose) example configuration is given below.
如下是一个(故意弄的繁琐一点的)[ example configuration] 。Each element of the list in the shovels clause is a named shovel. The shovel_names in the list must be distinct.
列表中的每一个 shovel 条目对应的元素都是一个命名 shovel 。列表中的 shovel_names 必须是独一无二的。Each shovel definition looks like this:
每一个 shovel 的定义都像如下这样:where shovel_name is the name of the shovel (an Erlang atom) and the clauses for sources, destinations and queue are mandatory. All the other clauses are optional.
其中 shovel_name 标明 shovel 的名字(属于 Erlang 的 atom 类型); sources 、 destinations 和 queue 这三个条目是 必须的;其他条目是 可选的。Each clause is fully described below.
下面详细描述每一个条目的信息。Both of these clauses are mandatory. They take the form:
这两个条目都是强制要求的,其常见配置形式如下:(or {destinations, ...}). The brokers clause is mandatory and the declarations clause optional.
(上面的 sources 可以直接替换为 destinations)。上面 brokers 条目是强制的,declarations 条目是可选的。This clause (or its variant broker clause, see note below) is mandatory. In
该条目(或者是其变体 broker 条目,看下面的说明)是强制的。
{brokers, broker_list}
broker_list is a list of URI broker connections (for the basic syntax, see AMQP URI), for example:broker_list 是以 URI 形式表示的 broker 连接的列表(其基本的构成语法,参阅 [ AMQP URI]),例如:
[ "amqp://fred:[email protected]/my_vhost"]
If the host is omitted (not valid in a general AMQP URI), the shovel uses a direct connection to the broker in which it is running. This avoids using the network stack.
如果上面的 URI 中的 host 部分被省略了(在常规的 AMQP URI 中是不合法的),shovel 将会与自己所运行的那台机器上的 broker 进行直连。这样可以避免使用网络协议栈。The syntax is extended to include a query part to permit the configuration of additional connection parameters. heartbeat, channel_max, and frame_max can be specified, in any order. Omitted fields assume default values. For example:
上述语法被扩展后可以包含一个 query part 以允许对额外的连接参数进行配置。heartbeat 、channel_max 和 frame_max 可以按任意顺序指定。未指定的域都假定使用默认值,例如:
"amqp://myhost?heartbeat=5&frame_max=8192"
specifies a (non-encrypted) network connection to the host myhost, using default username, password, port, vhost and channel_max. The heartbeat interval is set to 5 seconds, and the maximum frame size to 8192 bytes.
指定了一个到名为 myhost 的 host 的(非加密的)网路连接,其中使用了默认的 username 、password 、port 、vhost 和 channel_max 值。heartbeat 间隔设置为 5 秒,最大帧长度 frame_max 设置为 8192 字节。同样可以配置一个加密后的 SSL 连接,一般形式如下:
"amqps://username:password@host :port/vhost?&fail_if_no_peer_cert=failOption"
(This URI has been split across several lines only for readbility. There must be no white-space in the URI.)
(该 URI 被分割成多行形式仅仅是为了可读性,在 URI 中是禁止出现 white-space 的)All five parameters (3 paths: cacertfile, certfile and keyfile; and 2 options: verify, fail_if_no_peer_cert) must be specified as well as the amqps scheme. See the SSL guide for details of SSL in RabbitMQ in general and specifically the Erlang client section.
以上 5 个参数(其中 3 个是路径:cacertfile 、certfile 和 keyfile ;2 个是选项:verify 和 fail_if_no_peer_cert)必须指定具体的值,同时必须是使用 amqps 协议方式。参阅 [ SSL guide] 了解 SSL 在 RabbitMQ 中的一般用法,可以特别关注下 [ Erlang client] 段。Note: If the broker list consists of a single connection URI, the variant form:
注:如果 broker 列表仅由一条连接 URI 构成,其相应的变体形式如下:
{broker, amqp_uri_string}
is equivalent to a brokers clause with a single-element list.
其定同于在 brokers 条目中的列表仅包含一个元素的形式。declaration_list
This clause is optional. In
该条目可选。
{declarations, declaration_list}
the declaration_list is a list of AMQP methods (in the style of the Erlang client) which can be sent to the broker after connection and before shovelling.
在上述定义中, declaration_list 指明了可以使用的 AMQP 方法列表(以 Erlang 客户端的样式呈现),这些方法将用于“连接建立之后和开始进行 shovel 之前”的时间段内发送。This allows any resources that may need to be set up to be configured, including the source queue and the destination exchanges. For example:
可以对任何待建立的资源进行配置,其中包括源端 queue 和目的端 exchange ,例如:
{declarations, [ 'queue.declare']}
will first declare an anonymous queue, and then bind it to the exchange called "my_exchange". (The queue parameter <<>> on queue.bind means 'use the queue last declared on this channel'.)
上述配置将首先声明一个 匿名 queue ,然后将其绑定到名字为 "my_exchange" 的 exchange 上。(在 queue.bind 中的 queue 参数 <<>> 含义是 "使用在当前 channel 上声明的最后一个 queue" 。)Each element of the list is either an atom, being the name of an AMQP method, or a tuple with first element the method atom, and second element a property-list of parameter settings.
列表中的每一个元素或者是仅仅由 atom 类型构成的 AMQP 方法名,或者是以“首元素为 atom 类型的方法,次元素是参数设置属性列表”方式构成的 元组。If just the AMQP method atom is supplied all the parameters take their defaults (as illustrated with 'queue.declare' above).
如果在定义中仅提供了类型为 atom 的 AMQP 方法(未指定其对应设置值),那么所有项取其默认值(正如上面的参数 'queue.declare' 的情况)。
If a tuple and property-list is supplied, then the properties in the list specify some or all of the parameters explicitly.
如果给出了元组和其对应的属性列表,则在列表中给出的属性就是显式设置的那些属性。Here is another example:
下面是另外一个例子:
{'exchange.declare', [ {exchange, <<"my_exchange">>}]}
will declare a durable, direct exchange called "my_exchange".For full details, consult the Erlang Client documentation.
该例子将 声明一个持久的、direct 类型的、名字为 "my_exchange" 的 exchange 。This clause is mandatory. In
queue 条目是强制要求的。
{queue, queue_name}
queue_name is the name of the queue (as a binary string) to shovel messages from. For example:
queue_name 是(二进制字串形式的)queue 的名字,其代表 message 从哪里被 shovel 。例如:
{queue, <<"my_work_queue">>}
This queue must exist. Use the resource declarations to create the queue (or ensure it exists) first. If queue_name is <<>> (the empty binary string) the most recently declared queue in declarations is used. This allows anonymous queues to be declared and used.
该 queue 必须是存在的。可以使用资源声明的方式先创建相应的 queue(或者通过某种方式确认其存在)。如果 queue_name 的值为 <<>> (即空的二进制字符串),则在 declarations 条目中最近声明的 queue 将被使用。这种方式允许匿名 queue 被声明和使用。该条目可选。
{prefetch_count, count}
count is the maximum number of unacknowledged messages the shovel may hold at a time (a non-negative integer). For example:
count 的值指定 shovel 一次可以缓存的未确认的 message 的最大值(非负整数)。例如:
{prefetch_count, 1}
If this number is zero (the default), there is no limit.
如果这个值取 0 (默认值),则等同于无限。This clause is optional. In
该条目可选。
{ack_mode, a_mode}
a_mode is one of 'no_ack', 'on_publish' or 'on_confirm'.
a_mode 的值可以是 'no_ack' 、'on_publish' 或者 'on_confirm' 。indicates that no message acknowledgements are to be generated by the shovel (the broker automatically acknowledges all delivered messages);
表明 shovel 不会产生任何 message 确认(即 broker 会自动地 acknowledge 全部投递的 message);indicates that a message acknowledgement is to be sent (to the source broker) after each message is re-published to the destination;
表明 shovel 会在把每一条 message re-publish 到目的端之后(向源 broker)发送 message 确认;indicates that publish confirmations are sought and that a message acknowledgement is to be sent (to the source broker) after each message publication is confirmed by the destination broker.
表明 shovel 会对 publish confirmation 执行 seek 动作,表明 shovel 会在收到来自目的端 broker 的,对应于每一条 message 的确认之后,向远端 broker 发送 message 确认。The default is 'on_confirm', which is highly recommended. If other options are chosen performance may improve slightly, but messages are more likely to be lost in the event of failures.
a_mode 的默认值是 'on_confirm' ,并强烈建议使用该值。如果你选择使用其他的值,整体性能虽然会有略微提升,但是发生(各种)失效问题的情况时, message 将更可能会丢失。该条目可选,配置形式如下:
{publish_properties, property_list}
where the properties in the list are set on the basic.properties of each message before it is re-published.
其中,通过在 property_list 列表中指定的属性值,会在每一条 message 被 re-publish 前,设置进 basic.properties 方法中。For example:
举例:{publish_properties, [ {delivery_mode, 2} ]}
would mark all re-published messages persistent.
该设置将所有被 re-publish 的 message 设置为持久的。By default the properties of the message are preserved, but this clause can be used to change, or set any property, including content_type, content_encoding, headers, delivery_mode, priority, correlation_id, reply_to, expiration, message_id, timestamp, type, user_id, app_id and cluster_id.
默认情况下,(被转发的)message 的各项属性都是被保留的,但是该条目可以用来对任何属性进行改变或者设置,包括:content_type 、content_encoding 、headers 、delivery_mode 、priority 、correlation_id 、reply_to 、expiration 、message_id 、timestamp 、type 、user_id 、app_id 和 cluster_id 。This clause is optional. It takes the form:
该条目可选,配置形式如下:
{publish_fields, property_list}
where the properties in the list are used to set the fields on the basic.publish method used to re-publish messages.
其中在 property_list 列表中指定的属性值会在每一条 message 被 re-publish 前设置进 basic.publish 方法中。By default the messages are re-published using the original exchange name and routing key, for example. By specifying:
默认情况下,message 会使用原始的 exchange 名和 routing key 进行 re-publish ,例如:This clause is optional. In
该条目可选。reconn_delay is the number of seconds to wait before reconnecting in the event of connection failure (a non-negative number). For example:
reconn_delay 指定在连接失效的情况下,重新进行连接前需要等待的秒数(非负值)。例如:would delay for one and a half seconds before reconnecting after failure.
上述设置表明在连接失效后,重新连接前会等待 1.5 秒。If reconn_delay is 0, then no reconnections occur: the shovel will stop after the first failure.
如果 reconn_delay 的值为 0 ,则不会进行重连动作:即 shovel 将会在首次连接失效时停止工作。The default reconn_delay is 5 (seconds).
默认的 reconn_delay 值为 5 秒。
A verbose shovel configuration might look like this:
一份详细 shovel 配置文件看起来像下面这样:
{rabbitmq_shovel, [ {shovels, [ {my_first_shovel, [ {sources, [ {brokers, [ "amqp://fred:[email protected]/my_vhost" , "amqp://john:[email protected]/my_vhost" ]} , {declarations, [ {'exchange.declare', [ {exchange, <<"my_fanout">>} , {type, <<"fanout">>} , durable ]} , {'queue.declare', [{arguments, [{<<"x-message-ttl">>, long, 60000}]}]} , {'queue.bind', [ {exchange, <<"my_direct">>} , {queue, <<>>} ]} ]} ]} , {destinations, [ {broker, "amqp://"} , {declarations, [ {'exchange.declare', [ {exchange, <<"my_direct">>} , {type, <<"direct">>} , durable ]} ]} ]} , {queue, <<>>} , {prefetch_count, 10} , {ack_mode, on_confirm} , {publish_properties, [ {delivery_mode, 2} ]} , {publish_fields, [ {exchange, <<"my_direct">>} , {routing_key, <<"from_shovel">>} ]} , {reconnect_delay, 5} ]} ]} ]}
The configuration above defines a single shovel called 'my_first_shovel'.
上面的配置信息定义一个名字为 'my_first_shovel' 单独的 shovel 。'my_first_shovel' will connect to a broker on either host1 or host2 (as source), and directly to the local broker (as destination). It will reconnect to the other source broker on failure, after a delay of 5 seconds.
名字为 'my_first_shovel' 的 shovel 将会连接到(作为源端的)host1 或者 host2 上的 broker ,同时会直接连接到(作为目的端的)本地 broker 上。其会在当前作为源端的 broker 失效时重新连接到其他的源 broker 上,当然,是在 5 秒钟的延迟等待后。When connected to the source it will declare a direct, fanout exchange called "my_fanout", an anonymous queue with a per-queue message ttl, and bind the queue to the exchange.
当成功连接到源端时,其会:
a) 声明一个名字为 "my_fanout" 的 direct + fanout 类型 exchange ;When connected to the destination (the local broker) it will declare a durable, direct exchange called "my_direct".
当成功连接到目的端(本地 broker)时,其会声明一个名字为 "my_direct" 持久的,direct 类型的 exchange 。This shovel will re-publish messages sent to the anonymous queue on the source to the local exchange with the fixed routing key "from_shovel". The messages will be persistent and only acknowledged after receiving a publish confirm from the local broker.
源端匿名 queue 上收到的 message 会,以值为 "from_shovel" 的 routing_key ,被 shovel re-publish 到作为目的端的本地 exchange 上。该 message 将是持久的,并且只有在收到来自本地 broker 的 publish confirm 后再向源端进行 acknowledge 。The shovel consumer will not be allowed to hold more than ten unacknowledged messages at a time.
shovel 的 consumer 不允许一次性持有超过 10 条未确认的 message 。