The Idempotent Consumer from the EIP patterns is used to filter out duplicate messages.
This pattern is implemented using the IdempotentConsumer class. This uses an Expression to calculate a unique message ID string for a given message exchange; this ID can then be looked up in the IdempotentRepository to see if it has been seen before; if it has the message is consumed; if its not then the message is processed and the ID is added to the repository.
The Idempotent Consumer essentially acts like a Message Filter to filter out duplicates.
Camel will add the message id eagerly to the repository to detect duplication also for Exchanges currently in progress.
On completion Camel will remove the message id from the repository if the Exchange failed, otherwise it stays there.
The Idempotent Consumer has the following options:
Option | Default | Description |
---|---|---|
eager | true | Camel 2.0: Eager controls whether Camel adds the message to the repository before or after the exchange has been processed. If enabled before then Camel will be able to detect duplicate messages even when messages are currently in progress. By disabling Camel will only detect duplicates when a message has successfully been processed. |
messageIdRepositoryRef | null | A reference to a IdempotentRepository to lookup in the registry. This option is mandatory when using XML DSL. |
The following example will use the header myMessageId to filter out duplicates
RouteBuilder builder = new
RouteBuilder() {
public
void configure() {
errorHandler(deadLetterChannel("mock:error"
));
from("seda:a"
) .idempotentConsumer(header("myMessageId"
),
MemoryIdempotentRepository.memoryIdempotentRepository(200))
.to("seda:b"
);
}
};
The above example will use an in-memory based MessageIdRepository which can easily run out of memory and doesn't work in a clustered environment. So you might prefer to use the JPA based implementation which uses a database to store the message IDs which have been processed
from("direct:start"
).idempotentConsumer(
header("messageId"
),
jpaMessageIdRepository(lookup(JpaTemplate.class), PROCESSOR_NAME)
).to("mock:result"
);
In the above example we are using the header messageId to filter out duplicates and using the collection myProcessorName to indicate the Message ID Repository to use. This name is important as you could process the same message by many different processors; so each may require its own logical Message ID Repository.
For further examples of this pattern in use you could look at the junit test case
The following example will use the header myMessageId to filter out duplicates
<camelContext xmlns="http://camel.apache.org/schema/spring"
>
<route>
<from uri="direct:start"
/>
<idempotentConsumer messageIdRepositoryRef="myRepo"
>
<!-- use the messageId header as key for identifying duplicate messages -->
<header>
messageId</header>
<!-- if not a duplicate send it to this mock endpoint -->
<to uri="mock:result"
/>
</idempotentConsumer>
</route>
</camelContext>
If you would like to use this EIP Pattern then please read the Getting Started , you may also find the Architecture useful particularly the description of Endpoint and URIs . Then you could try out some of the Examples first before trying this pattern out.
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
二、
Parent Directory |
Revision Log
CAMEL-2576: Renamed redeliverDelay to redeliveryDelay.
1 | /** |
2 | * Licensed to the Apache Software Foundation (ASF) under one or more |
3 | * contributor license agreements. See the NOTICE file distributed with |
4 | * this work for additional information regarding copyright ownership. |
5 | * The ASF licenses this file to You under the Apache License, Version 2.0 |
6 | * (the "License"); you may not use this file except in compliance with |
7 | * the License. You may obtain a copy of the License at |
8 | * |
9 | * http://www.apache.org/licenses/LICENSE-2.0 |
10 | * |
11 | * Unless required by applicable law or agreed to in writing, software |
12 | * distributed under the License is distributed on an "AS IS" BASIS, |
13 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
14 | * See the License for the specific language governing permissions and |
15 | * limitations under the License. |
16 | */ |
17 | package org.apache.camel.processor; |
18 | |
19 | import org.apache.camel.ContextTestSupport; |
20 | import org.apache.camel.Endpoint; |
21 | import org.apache.camel.Exchange; |
22 | import org.apache.camel.Message; |
23 | import org.apache.camel.Processor; |
24 | import org.apache.camel.builder.RouteBuilder; |
25 | import org.apache.camel.component.mock.MockEndpoint; |
26 | import org.apache.camel.processor.idempotent.MemoryIdempotentRepository; |
27 | |
28 | /** |
29 | * @version $Revision$ |
30 | */ |
31 | public class IdempotentConsumerTest extends ContextTestSupport { |
32 | protected Endpoint startEndpoint; |
33 | protected MockEndpoint resultEndpoint; |
34 | |
35 | @Override |
36 | public boolean isUseRouteBuilder() { |
37 | return false; |
38 | } |
39 | |
40 | public void testDuplicateMessagesAreFilteredOut() throws Exception { |
41 | context.addRoutes(new RouteBuilder() { |
42 | @Override |
43 | public void configure() throws Exception { |
44 | from("direct:start").idempotentConsumer( |
45 | header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200) |
46 | ).to("mock:result"); |
47 | } |
48 | }); |
49 | context.start(); |
50 | |
51 | resultEndpoint.expectedBodiesReceived("one", "two", "three"); |
52 | |
53 | sendMessage("1", "one"); |
54 | sendMessage("2", "two"); |
55 | sendMessage("1", "one"); |
56 | sendMessage("2", "two"); |
57 | sendMessage("1", "one"); |
58 | sendMessage("3", "three"); |
59 | |
60 | assertMockEndpointsSatisfied(); |
61 | } |
62 | |
63 | public void testFailedExchangesNotAddedDeadLetterChannel() throws Exception { |
64 | context.addRoutes(new RouteBuilder() { |
65 | @Override |
66 | public void configure() throws Exception { |
67 | errorHandler(deadLetterChannel("mock:error").maximumRedeliveries(2).redeliveryDelay(0).logStackTrace(false)); |
68 | |
69 | from("direct:start").idempotentConsumer( |
70 | header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200) |
71 | ).process(new Processor() { |
72 | public void process(Exchange exchange) throws Exception { |
73 | String id = exchange.getIn().getHeader("messageId", String.class); |
74 | if (id.equals("2")) { |
75 | throw new IllegalArgumentException("Damm I cannot handle id 2"); |
76 | } |
77 | } |
78 | }).to("mock:result"); |
79 | } |
80 | }); |
81 | context.start(); |
82 | |
83 | // we send in 2 messages with id 2 that fails |
84 | getMockEndpoint("mock:error").expectedMessageCount(2); |
85 | resultEndpoint.expectedBodiesReceived("one", "three"); |
86 | |
87 | sendMessage("1", "one"); |
88 | sendMessage("2", "two"); |
89 | sendMessage("1", "one"); |
90 | sendMessage("2", "two"); |
91 | sendMessage("1", "one"); |
92 | sendMessage("3", "three"); |
93 | |
94 | assertMockEndpointsSatisfied(); |
95 | } |
96 | |
97 | public void testFailedExchangesNotAddedDeadLetterChannelNotHandled() throws Exception { |
98 | context.addRoutes(new RouteBuilder() { |
99 | @Override |
100 | public void configure() throws Exception { |
101 | errorHandler(deadLetterChannel("mock:error").handled(false).maximumRedeliveries(2).redeliveryDelay(0).logStackTrace(false)); |
102 | |
103 | from("direct:start").idempotentConsumer( |
104 | header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200) |
105 | ).process(new Processor() { |
106 | public void process(Exchange exchange) throws Exception { |
107 | String id = exchange.getIn().getHeader("messageId", String.class); |
108 | if (id.equals("2")) { |
109 | throw new IllegalArgumentException("Damm I cannot handle id 2"); |
110 | } |
111 | } |
112 | }).to("mock:result"); |
113 | } |
114 | }); |
115 | context.start(); |
116 | |
117 | // we send in 2 messages with id 2 that fails |
118 | getMockEndpoint("mock:error").expectedMessageCount(2); |
119 | resultEndpoint.expectedBodiesReceived("one", "three"); |
120 | |
121 | sendMessage("1", "one"); |
122 | sendMessage("2", "two"); |
123 | sendMessage("1", "one"); |
124 | sendMessage("2", "two"); |
125 | sendMessage("1", "one"); |
126 | sendMessage("3", "three"); |
127 | |
128 | assertMockEndpointsSatisfied(); |
129 | } |
130 | |
131 | public void testFailedExchangesNotAdded() throws Exception { |
132 | context.addRoutes(new RouteBuilder() { |
133 | @Override |
134 | public void configure() throws Exception { |
135 | // use default error handler |
136 | errorHandler(defaultErrorHandler()); |
137 | |
138 | from("direct:start").idempotentConsumer( |
139 | header("messageId"), MemoryIdempotentRepository.memoryIdempotentRepository(200) |
140 | ).process(new Processor() { |
141 | public void process(Exchange exchange) throws Exception { |
142 | String id = exchange.getIn().getHeader("messageId", String.class); |
143 | if (id.equals("2")) { |
144 | throw new IllegalArgumentException("Damm I cannot handle id 2"); |
145 | } |
146 | } |
147 | }).to("mock:result"); |
148 | } |
149 | }); |
150 | context.start(); |
151 | |
152 | resultEndpoint.expectedBodiesReceived("one", "three"); |
153 | |
154 | sendMessage("1", "one"); |
155 | sendMessage("2", "two"); |
156 | sendMessage("1", "one"); |
157 | sendMessage("2", "two"); |
158 | sendMessage("1", "one"); |
159 | sendMessage("3", "three"); |
160 | |
161 | assertMockEndpointsSatisfied(); |
162 | } |
163 | |
164 | protected void sendMessage(final Object messageId, final Object body) { |
165 | template.send(startEndpoint, new Processor() { |
166 | public void process(Exchange exchange) { |
167 | // now lets fire in a message |
168 | Message in = exchange.getIn(); |
169 | in.setBody(body); |
170 | in.setHeader("messageId", messageId); |
171 | } |
172 | }); |
173 | } |
174 | |
175 | @Override |
176 | protected void setUp() throws Exception { |
177 | super.setUp(); |
178 | |
179 | startEndpoint = resolveMandatoryEndpoint("direct:start"); |
180 | resultEndpoint = getMockEndpoint("mock:result"); |
181 | } |
182 | |
183 | } |
二、
java.lang.Object
org.apache.camel.impl.ServiceSupport
org.apache.camel.processor.idempotent.IdempotentConsumer
public class IdempotentConsumer
An implementation of the Idempotent Consumer pattern.
Constructor Summary | |
---|---|
IdempotentConsumer (Expression messageIdExpression, IdempotentRepository <String > idempotentRepository, boolean eager, Processor processor) |
Method Summary | |
---|---|
protected void |
doStart () |
protected void |
doStop () |
IdempotentRepository <String > |
getIdempotentRepository () |
Expression |
getMessageIdExpression () |
Processor |
getProcessor () |
boolean |
hasNext () Are there more outputs? |
List <Processor > |
next () Next group of outputs |
protected void |
onDuplicateMessage (Exchange exchange, String messageId) A strategy method to allow derived classes to overload the behaviour of processing a duplicate message |
void |
process (Exchange exchange) Processes the message exchange |
String |
toString () |
Methods inherited from class org.apache.camel.impl.ServiceSupport |
---|
addChildService , getStatus , getVersion , isRunAllowed , isStarted , isStarting , isStopped , isStopping , removeChildService , start , stop |
Methods inherited from class java.lang.Object |
---|
clone , equals , finalize , getClass , hashCode , notify , notifyAll , wait , wait , wait |
Constructor Detail |
---|
public IdempotentConsumer
(Expression
messageIdExpression,
IdempotentRepository
<String
> idempotentRepository,
boolean eager,
Processor
processor)
Method Detail |
---|
public String
toString
()
toString
in class
Object
public void process
(Exchange
exchange)
throws Exception
Processor
process
in interface
Processor
exchange
- the message exchange
Exception
- if an internal processing error has occurred.
public List
<Processor
> next
()
Navigate
next
in interface
Navigate <Processor >
public boolean hasNext
()
Navigate
hasNext
in interface
Navigate <Processor >
public Expression
getMessageIdExpression
()
public IdempotentRepository
<String
> getIdempotentRepository
()
public Processor
getProcessor
()
protected void doStart
()
throws Exception
doStart
in class
ServiceSupport
Exception
protected void doStop
()
throws Exception
doStop
in class
ServiceSupport
Exception
protected void onDuplicateMessage
(Exchange
exchange,
String
messageId)
exchange
- the exchange
messageId
- the message ID of this exchange
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
三、rg.apache.camel.spi
public interface IdempotentRepository<E>
Access to a repository of Message IDs to implement the Idempotent Consumer pattern.
The add and contains methods is operating according to the Set
contract.
Method Summary | |
---|---|
boolean |
add (E key) Adds the key to the repository. |
boolean |
confirm (E key) Confirms the key, after the exchange has been processed sucesfully. |
boolean |
contains (E key) Returns true if this repository contains the specified element. |
boolean |
remove (E key) Removes the key from the repository. |
Method Detail |
---|
boolean add
(E
key)
key
- the key of the message for duplicate test
boolean contains
(E
key)
key
- the key of the message
boolean remove
(E
key)
Is usually invoked if the exchange failed.
key
- the key of the message for duplicate test
boolean confirm
(E
key) Confirms the key, after the exchange has been processed sucesfully.
Parameters:
key
- the key of the message for duplicate test
Returns:
true
if the key was confirmed
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
四、Fluent Builders
Camel provides fluent builders for creating routing and mediation rules using a type-safe IDE friendly way which provides smart completion and is refactoring safe.