nternet transactions, business-to-business systems, peer-to-peer processes, and real-time workflows are too dynamic and too complex to be modeled by traditional sequential-processing methods. Therefore, the need for more sophisticated asynchronous processing techniques is quickly becoming apparent. To address these unpredictable environments, the current trend in systems architecture is service-oriented design and event-driven programming.
A service-oriented architecture (SOA) presents a dynamic runtime environment, where loose couplings between service providers and/or service consumers enable powerful and flexible component interactions. Building a communication model to exploit this power and flexibility is a high priority for competitive software development. An event-driven communication model is able to respond better to real-time changes and stimuli than conventional request/reply mechanisms.
Service-oriented and event-driven architectures are natural fits for distributed systems since they share many of the same characteristics, such as modularity, loose-couplings, and adaptability.
In this article, I discuss the details of designing an effective event-driven and service-oriented platform using Mule, a lightweight event-messaging framework designed around an enterprise service bus (ESB) pattern. Components and applications can use Mule to communicate through a common messaging framework implemented transparently using Java Message Service (JMS) or another messaging technology.
Overview of service-oriented architecture
The term "service-oriented" has evolved to define an architecture where a service is a software component that embodies a core piece of an enterprise's business logic and features the following characteristics:
* Loosely coupled: Services are not fundamentally bound with other components
* Protocol-independent: Multiple protocols can transparently access a given service
* Location-agnostic: A given service typically performs a composite form of business logic and returns the result in a single call
* Coarse-grained: Services can be accessed in the same manner no matter their location
* Maintains no user state
Services typically focus exclusively on solving business-domain problems.
Generally, service clients rely on configuration data, registries, and software factories to determine the location, protocol, and public interface for each service.
Applications are typically described by what they do, not necessarily by what they are or what they contain. For this reason, it's much more straightforward to describe an application publicly using verbs (services) as opposed to nouns (objects). Since objects define a thing and not an action, an impedance mismatch can occur when attempting to encapsulate what a component does as opposed to what a component is. In SOA, an application is described naturally, since each logical business operation of the application that can be verbalized is a likely candidate for a service. Therefore, SOA solves the impedance mismatch by allowing applications and components to access the functionality of services based on what the services do, i.e., what actions they perform. In turn, application developers can more easily match their needs with the appropriate services, since the interfaces for the services describe more completely the problems they solve.
Overview of event-driven architecture
An event-driven architecture (EDA) defines a methodology for designing and implementing applications and systems in which events transmit between loosely coupled software components and services. An event-driven system is typically comprised of event consumers and event producers. Event consumers subscribe to an intermediary event manager, and event producers publish to this manager. When the event manager receives an event from a producer, the manager forwards the event to the consumer. If the consumer is unavailable, the manager can store the event and try to forward it later. This method of event transmission is referred to in message-based systems as store and forward.
Building applications and systems around an event-driven architecture allows these applications and systems to be constructed in a manner that facilitates more responsiveness, since event-driven systems are, by design, more normalized to unpredictable and asynchronous environments.
Benefits of event-driven design and development
Event-driven design and development provide the following benefits:
* Allows easier development and maintenance of large-scale, distributed applications and services involving unpredictable and/or asynchronous occurrences
* Allows new and existing applications and services to be assembled, reassembled, and reconfigured easily and inexpensively
* Promotes component and service reuse, therefore enabling a more agile and bug-free development environment
* Short-term benefits: Allows easier customization because the design is more responsive to dynamic processes
* Long-term benefits: Allows system and organizational health to become more accurate and synchronized closer to real-time changes
EDA and SOA together
Unlike a request/reply system, where callers must explicitly request information, an event-driven architecture (EDA) provides a mechanism for systems to respond dynamically as events occur. In an EDA, events are published by event producers, and event consumers receive events as they happen.
Business systems benefit from the features of both an SOA and an EDA, since an EDA can trigger event consumers as events happen and loosely coupled services can be quickly accessed and queried from those same consumers.
For systems to be most responsive, they must be able to quickly determine the necessary actions when events are triggered. To this end, events should be published and consumed across all boundaries of the SOA, including the layers of the architectural stack and across physical tiers.
Figure 1 illustrates possible events that can be triggered across layers of an architectural stack:
Figure 1: Event flow across architecture stack. Click on thumbnail to view full-sized image.
In the context of Figure 1, an event can be defined as any published change in a system, platform, component, business, or application process. Events can be high-level and business-oriented, or low-level and technical in character. Because events can be transmitted and received, event-aware applications and services can respond to the underlying changes as needed.
Event taxonomies and causality
The secret to understanding a given event is to know its cause at the time the event occurred, knowledge often referred to as event causality. Event causality is typically divided into two basic categories:
* Horizontal causality: Both the event's source and cause reside on the same conceptual layer in the architectural stack
* Vertical causality: Both the event's source and cause reside on different conceptual layers in the architectural stack
Vertical causality implies an event taxonomy that remains somewhat constant across different layers of a system, as illustrated by the following list:
* Lifecycle events: Signify changes in an entity's lifecycle, such as stopping or starting a process
* Execution events: Signify runtime occurrences, such as service or component invocations
* Management events: Signify when thresholds have exceeded defined limits or ranges
Horizontal causality implies an event taxonomy that also remains somewhat constant across different layers of a system, as illustrated by the following list:
* System-layer events: Signify system-level activities, such as the creation of a file or the closing of a port
* Platform-layer events: Signify platform-level activities, such as the modification of a datasource or the addition of a new service
* Component-layer events: Signify component-level activities, such as the transformation of a view object or a state-machine transition
* Business-layer events: Signify business-level activities, such as the creation of a user or the removal of an account
* Application-layer events: Signify application-level activities, such as a premium increase or a quote submission
The benefits of event-driven communication within an SOA are currently being realized by a number of ESB frameworks and platforms. One of the most promising of these within the Java development realm is Mule.
Introducing Mule
Mule is an open source ESB-messaging framework and message broker, loosely based on the staged event-driven architecture (SEDA). SEDA defines a highly concurrent enterprise platform in terms of stages (self-contained application components) connected by queues. Mule uses concepts from SEDA to increase the performance of processing events.
Mule provides support for asynchronous, synchronous, and request-response event processing using disparate technologies and transports such as JMS, HTTP, email, and XML-based Remote Procedure Call. Mule can be easily embedded into any application framework and explicitly supports the Spring framework. Mule also supports dynamic, declarative, content-based, and rule-based message routing. Mule facilitates declarative and programmatic transaction support, including XA transaction support. Mule provides a representational state transfer (REST) API to provide Web-based access to events.
The Mule ESB model drives all services in a system over a decoupled, message-communication backbone. Services registered with the bus have no knowledge of other registered services; therefore, each service is concerned with processing only the events it receives. Mule also decouples container, transport, and transformation details from the services, allowing any kind of object to be registered as a service on the bus.
I use the Mule framework to demonstrate the concepts and ideas discussed in this article.
The Mule architecture
The Mule architecture consists primarily of the following elements:
The Universal Message Object (UMO) API
The UMO API defines the services and interactions of objects to be managed by Mule.
UMO components
UMO components can be any component in the Mule system that receives, processes, and sends data as event messages.
Mule server
The Mule server component is a server application launched to bootstrap the Mule environment.
Descriptors
The descriptor components describe a Mule UMO's attributes. New Mule UMOs can be initialized as needed from their associated descriptor. A descriptor consists of:
* The UMO component name
* The UMO component version
* The UMO component implementation class
* An exception strategy
* Inbound and outbound providers
* Inbound and outbound routers
* Interceptors
* Receive and send endpoints
* Inbound and outbound transformers
* Miscellaneous properties
Connectors
Connectors are components that provide the implementation for connecting to an external system or protocol and managing the session for that system or protocol. A connector is responsible for sending data to an external message receiver and for managing the registration and deregistration of message receivers.
Providers
Providers are components that manage the sending, receiving, and transformation of event data to and from external systems. They enable connections to external systems or other components in Mule. A provider acts as a bridge from the external system into Mule and vice versa. It is, in fact, a composite of a set of objects used to connect to and communicate with the underlying system. The elements of a provider are:
* Connector: Responsible for connecting to the underlying system
* Message receiver: Used to receive events from the system
* Connector dispatchers: Pass data to the system
* Transformers: Convert data received from the system and data being sent to the system
* Endpoint: Used as the channel address through which a connection is made
* Transaction configuration: Used to define the connection's transactional properties
Endpoint resolvers
Endpoint resolvers determine what method to invoke on a UMO component when the component receives an event.
Transformers
Transformer components transform message or event payloads to and from different data formats. Transformers can be chained together to perform successive transforms on an event before an object receives it.
Message adapters
Message adapters provide a common manner in which to read disparate data from external systems.
Message receivers
Message receivers are listener-endpoint threads that receive data from an external system.
Message dispatchers
Message dispatchers send (synchronous) or dispatch (asynchronous) events to the underlying technology.
Message routers
Message routers are components that can be configured for a UMO component to route a message to deferent providers based on the message or other configuration.
Agents
Agents are components that bind to external services such as Java Management Extension servers.
Mule model
A Mule model encapsulates and manages the runtime behavior of a Mule server instance. A model consists of:
* Descriptors
* UMO components
* An endpoint resolver
* A lifecycle-adapter factory
* A component resolver
* A pool factory
* An exception strategy
Mule manager
The Mule manager maintains and provides the following services:
* Agents
* Providers
* Connectors
* Endpoints
* Transformers
* The interceptor stack
* A Mule model
* A Mule server
* The transaction manager
* Application properties
* The Mule configuration
The diagram in Figure 2 illustrates a high-level view of the message flow for the Mule architecture.
Figure 2: Mule high-level architecture. Click on thumbnail to view full-sized image.
Mule events
Mule events contain event data and properties examined and manipulated by event-aware components. The properties are arbitrary and can be set at any time after an event is created.
The org.mule.umo.UMOEvent class represents an event occurring in the Mule environment. All data sent or received within the Mule environment is passed between components as an instance of UMOEvent. The data in a Mule event can be accessed in its original format or in a transformed format. A Mule event uses a transformer associated with the provider that received the event to transform the event's payload into a format the receiving component understands.
The payload for a Mule event is contained within an instance of the org.mule.umo.UMOMessage interface. A UMOMessage instance is composed of the payload itself and its associated properties. This interface also acts as a common abstraction of different message implementations provided by different underlying technologies.
The org.mule.extras.client.MuleClient class defines a simple API that allows Mule clients to send and receive events to and from a Mule server. In most Mule applications, events are triggered by some external occurrence, such as a message received on a topic or a file being deleted from a directory.
The following illustrates how to send an event synchronously to another Mule component:
String componentName = "MyReceiver"; // The name of the receiving component.
String transformers = null; // A comma-separated list of transformers
// to apply to the result message.
String payload = "A test event"; // The payload of the event.
java.util.Map messageProperties = null; // Any properties to be associated
// with the payload.
MuleClient client = new MuleClient();
UMOMessage message = client.sendDirect(componentName,
transformers,
payload,
messageProperties);
System.out.println("Event result: " + message.getPayloadAsString());
An instance of MuleClient requires a server URL to define the endpoint for the remote Mule server to which the MuleClient instance will connect. The URL defines the protocol, the message's endpoint destination, and, optionally, the provider to use when dispatching the event. Endpoint examples are:
* vm://com.jeffhanson.receivers.Default: Dispatches to a com.jeffhanson.receivers.Default destination using the virtual machine provider. The VM provider enables intra-VM event communication between components using transient or persistent queues.
* jms://jmsProvider/accounts.topic: Dispatches a JMS message via the globally registered jmsProvider to a topic destination called accounts.topic.
* jms://accounts.topic: Dispatches a JMS message via the first (default) JMS provider.
Mule event processing
Mule can send and receive events in three different ways:
1. Asynchronously: A given component can simultaneously process multiple events sent and received by different threads.
2. Synchronously: A single event must complete processing before a component can resume execution. In other words, a component that produces an event sends the event and then blocks until the call returns, thereby allowing only one event at a time to be processed.
3. Request-response: A component specifically requests an event and waits for a specified time to receive a response.
The org.mule.impl.MuleComponent implementation class provides a concrete component class that includes all the functionality needed to send and receive data and create events.
Objects that execute synchronously are encouraged to implement the org.mule.umo.lifecycle.Callable interface, which defines a single method, Object onCall(UMOEventContext eventContext). The Callable interface provides UMOs with an interface that supports event calls. Although not mandatory, the interface provides a lifecycle method that executes when the implementing component receives an event. The following illustrates a simple implementation of this interface:
import org.mule.umo.lifecycle.Callable;
public class EchoComponent
implements Callable
{
public Object onCall(UMOEventContext context) throws Exception
{
String msg = context.getMessageAsString();
// Print message to System.out
System.out.println("Received synchronous message: " + msg);
// Echo transformed message back to sender
return context.getTransformedMessage();
}
}
The object returned from the onCall() method can be anything. When the UMOLifecycleAdapter for the component receives this object, it will first see if the object is a UMOMessage; if the object is neither a UMOMessage nor null, a new message will be created using the returned object as the payload. This new event will then publish via the configured outbound router, if one has been configured for the UMO and the setStopFurtherProcessing(true) method wasn't called on the UMOEventContext instance.
A simple event framework using Mule
Let's put the pieces of Mule together to construct a simple event framework. The framework consists of an event manager responsible for registering and deregistering services that can receive events, and for synchronously and asynchronously routing messages to these services.
The Mule "vm" protocol requires that a configuration file be located at a directory named META-INF/services/org/mule/providers/vm, relative to the event manager's working directory. This file defines numerous components for the protocol, such as the connector and dispatcher factory. The file's contents are as follows:
connector=org.mule.providers.vm.VMConnector
dispatcher.factory=org.mule.providers.vm.VMMessageDispatcherFactory
message.receiver=org.mule.providers.vm.VMMessageReceiver
message.adapter=org.mule.providers.vm.VMMessageAdapter
endpoint.builder=org.mule.impl.endpoint.ResourceNameEndpointBuilder
A simple interface defines the event manager's public view:
package com.jeffhanson.mule;
import org.mule.umo.FutureMessageResult;
public interface EventManager
{
/**
* Sends an event message synchronously to a given service.
*
* @param serviceName The name of the service to which the event
* message is to be sent.
* @param payload The content of the event message.
* @return Object The result, if any.
* @throws EventException on error
*/
public Object sendSynchronousEvent(String serviceName,
Object payload)
throws EventException;
/**
* Sends an event message asynchronously to a given service.
*
* @param serviceName The name of the service to which the event
* message is to be sent.
* @param payload The content of the event message.
* @return FutureMessageResult The result, if any.
* @throws EventException on error
*/
public FutureMessageResult sendAsynchronousEvent(String serviceName,
Object payload)
throws EventException;
/**
* Starts this event manager.
*/
public void start();
/**
* Stops this event manager.
*/
public void stop();
/**
* Retrieves the protocol this event manager uses.
* @return
*/
public String getProtocol();
/**
* Registers a service to receive event messages.
*
* @param serviceName The name to associate with the service.
* @param implementation Either a container reference to the service
* or a fully-qualified class name.
*/
public void registerService(String serviceName,
String implementation)
throws EventException;
/**
* Unregisters a service from receiving event messages.
*
* @param serviceName The name associated with the service to unregister.
*/
public void unregisterService(String serviceName)
throws EventException;
}
The event-manager implementation class is encapsulated within a factory class, thereby allowing the implementation to change as needed without affecting the event manager's clients. The event-manager implementation is shown below:
package com.jeffhanson.mule;
import org.mule.umo.*;
import org.mule.extras.client.MuleClient;
import org.mule.impl.endpoint.MuleEndpoint;
import org.mule.config.QuickConfigurationBuilder;
import java.util.HashMap;
import java.util.Map;
public class EventManagerFactory
{
private static HashMap instances = new HashMap();
/**
* Retrieves the event manager instance for a given protocol.
*
* @param protocol The protocol to use.
* @return EventManager The event manager instance.
*/
public static EventManager getInstance(String protocol)
{
EventManager instance = (EventManager)instances.get(protocol);
if (instance == null)
{
instance = new EventManagerImpl(protocol);
instances.put(protocol, instance);
}
return instance;
}
/**
* A concrete implementation for a simple event manager.
*/
private static class EventManagerImpl
implements EventManager
{
private UMOManager manager = null;
private QuickConfigurationBuilder builder = null;
private MuleClient eventClient = null;
private String protocol = null;
private MuleEndpoint receiveEndpoint = null;
private MuleEndpoint sendEndpoint = null;
private EventManagerImpl(String protocol)
{
this.protocol = protocol;
}
/**
* Starts this event manager.
*/
public void start()
{
try
{
builder = new QuickConfigurationBuilder();
manager = builder.createStartedManager(true,
protocol + "tmp/events");
eventClient = new MuleClient();
receiveEndpoint = new MuleEndpoint(protocol
+ "tmp/events/receive");
sendEndpoint = new MuleEndpoint(protocol + "tmp/events/send");
}
catch (UMOException e)
{
System.err.println(e);
}
}
/**
* Stops this event manager.
*/
public void stop()
{
try
{
manager.stop();
}
catch (UMOException e)
{
System.err.println(e);
}
}
/**
* Retrieves the protocol this event manager uses.
* @return
*/
public String getProtocol()
{
return protocol;
}
/**
* Registers a service to receive event messages.
*
* @param serviceName The name to associate with the service.
* @param implementation Either a container reference to the service
* or a fully-qualified class name
* to use as the component implementation.
*/
public void registerService(String serviceName,
String implementation)
throws EventException
{
if (!manager.getModel().isComponentRegistered(serviceName))
{
try
{
builder.registerComponent(implementation,
serviceName,
receiveEndpoint,
sendEndpoint);
}
catch (UMOException e)
{
throw new EventException(e.toString());
}
}
}
/**
* Unregisters a service from receiving event messages.
*
* @param serviceName The name associated with the service to unregister.
*/
public void unregisterService(String serviceName)
throws EventException
{
try
{
builder.unregisterComponent(serviceName);
}
catch (UMOException e)
{
throw new EventException(e.toString());
}
}
/**
* Sends an event message synchronously to a given service.
*
* @param serviceName The name of the service to which the event
* message is to be sent.
* @param payload The content of the event message
* @return Object The result, if any.
* @throws EventException on error
*/
public Object sendSynchronousEvent(String serviceName,
Object payload)
throws EventException
{
try
{
if (!manager.getModel().isComponentRegistered(serviceName))
{
throw new EventException("Service: " + serviceName
+ " is not registered.");
}
String transformers = null;
Map messageProperties = null;
UMOMessage result = eventClient.sendDirect(serviceName,
transformers,
payload,
messageProperties);
if (result == null)
{
return null;
}
return result.getPayload();
}
catch (UMOException e)
{
throw new EventException(e.toString());
}
catch (Exception e)
{
throw new EventException(e.toString());
}
}
/**
* Sends an event message asynchronously.
*
* @param serviceName The name of the service to which the event
* message is to be sent.
* @param payload The content of the event message.
* @return FutureMessageResult The result, if any
* @throws EventException on error
*/
public FutureMessageResult sendAsynchronousEvent(String serviceName,
Object payload)
throws EventException
{
FutureMessageResult result = null;
try
{
if (!manager.getModel().isComponentRegistered(serviceName))
{
throw new EventException("Service: " + serviceName
+ " is not registered.");
}
String transformers = null;
Map messageProperties = null;
result = eventClient.sendDirectAsync(serviceName,
transformers,
payload,
messageProperties);
}
catch (UMOException e)
{
throw new EventException(e.toString());
}
return result;
}
}
}
The Mule framework dispatches messages by the payload's data type. The event framework can exploit this payload-based dispatching mechanism by defining generic event methods to act as event receivers in the services registered with the event manager. The following class defines one of these services with three overloaded event methods named receiveEvent():
package com.jeffhanson.mule;
import java.util.Date;
public class TestService
{
public void receiveEvent(String eventMessage)
{
System.out.println("\n\nTestService.receiveEvent(String) received "
+ "event message: " + eventMessage + "\n\n");
}
public void receiveEvent(Integer eventMessage)
{
System.out.println("\n\nTestService.receiveEvent(Integer) received "
+"event message: " + eventMessage + "\n\n");
}
public void receiveEvent(Date eventMessage)
{
System.out.println("\n\nTestService.receiveEvent(Date) received "
+ "event message: " + eventMessage + "\n\n");
}
}
The event manager's client application sends three events to the test service to test each receiveEvent() method. The client application follows:
package com.jeffhanson.mule;
import org.apache.log4j.Logger;
import org.apache.log4j.Level;
import org.apache.log4j.BasicConfigurator;
import java.util.Date;
public class EventClient
{
static Logger logger = Logger.getLogger(EventClient.class);
public static void main(String[] args)
{
// Set up a simple configuration that logs on the console.
BasicConfigurator.configure();
logger.setLevel(Level.ALL);
try
{
EventManager eventManager =
EventManagerFactory.getInstance("vm://");
eventManager.start();
String serviceName = TestService.class.getName();
String implementation = serviceName;
eventManager.registerService(serviceName, implementation);
Object result =
eventManager.sendSynchronousEvent(serviceName, "A test message");
if (result != null)
{
System.out.println("Event result: " + result.toString());
}
result =
eventManager.sendSynchronousEvent(serviceName, new Integer(23456));
if (result != null)
{
System.out.println("Event result: " + result.toString());
}
result =
eventManager.sendSynchronousEvent(serviceName, new Date());
if (result != null)
{
System.out.println("Event result: " + result.toString());
}
eventManager.stop();
}
catch (EventException e)
{
System.err.println(e.toString());
}
}
}
The Mule platform's simplifications and abstractions that the preceding framework provides enable you to send and receive synchronous and asynchronous events across layers of an architectural stack without knowing the details of the underlying event system. The use of the Factory pattern and SOA principles are exploited to facilitate a loosely-coupled and extensible design.
Summary
Designing an effective event-driven software system can grow complex when services and processes need to interact across multiple tiers and protocols. However, a service-oriented architecture built around a properly designed event-management layer using standard industry patterns can reduce or even eliminate these problems.
The Mule platform provides APIs, components, and abstractions that can be used to build a powerful, robust, event-driven system that is scalable and highly-maintainable.