Enterprise Integration Patterns in Practice

References:
http://www.eaipatterns.com/toc.html

Basic Messaging Concepts

Like most technologies, Messaging involves certain basic concepts. Once you understand these concepts, you can make sense of the technology even before you understand all of the details about how to use it. These basic messaging concepts are:

1. Channels — Messaging applications transmit data through a Message Channel, a virtual pipe that connects a sender to a receiver. A newly installed messaging system doesn’t contain any channels; you must determine how your applications need to communicate and then create the channels to facilitate it.
2. Messages — A Message is an atomic packet of data that can be transmitted on a channel. Thus to transmit data, an application must break the data into one or more packets, wrap each packet as a message, and then send the message on a channel. Likewise, a receiver application receives a message and must extract the data from the message to process it. The message system will try repeatedly to deliver the message (e.g., transmit it from the sender to the receiver) until it succeeds.
3. Multi-step delivery — In the simplest case, the message system delivers a message directly from the sender’s computer to the receiver’s computer. However, actions often need to be performed on the message after it is sent by its original sender but before it is received by its final receiver. For example, the message may have to be validated or transformed because the receiver expects a different message format than the sender. A Pipes and Filters architecture describes how multiple processing steps can be chained together using channels.
4. Routing — In a large enterprise with numerous applications and channels to connect them, a message may have to go through several channels to reach its final destination. The route a message must follow may be so complex that the original sender does not know what channel will get the message to the final receiver. Instead, the original sender sends the message to a Message Router, an application component and filter in the pipes-and-filters architecture, which will determine how to navigate the channel topology and direct the message to the final receiver, or at least to the next router.
5. Transformation — Various applications may not agree on the format for the same conceptual data; the sender formats the message one way, yet the receiver expects it to be formatted another way. To reconcile this, the message must go through an intermediate filter, a Message Translator, that converts the message from one format to another.
6. Endpoints — An application does not have some built-in capability to interface with a messaging system. Rather, it must contain a layer of code that knows both how the application works and how the messaging system works, bridging the two so that they work together. This bridge code is a set of coordinated Message Endpoints that enable the application to send and receive messages.

Enterprise Integration Patterns are divided into seven sections:

1. Messaging Systems,
2. Messaging Channels,
3. Message Constructions,
4. Message Routing,
5. Message Transformation,
6. Messaging endpoints,
7. System management.
Message Channel (from Messaging Systems)


A message channel is a logical channel which is used to connect the applications. One application writes messages to the channel and the other one (or others) reads that message from the channel. Message queue and message topic are examples of message channels.

Message Translator (from Messaging Systems)


Message translator transforms messages in one format to another. For example one application sends a message in XML format, but the other accepts only JSON messages so one of the parties (or mediator) has to transform XML data to JSON. This is probably the most widely used integration pattern.

Publish-Subscribe Channel (from Messaging Channels)

This type of channel broadcasts an event or notification to all subscribed receivers. This is in contrast with a point-to-point channel . Each subscriber receive the message once and next copy of this message is deleted from channel. The most common implementation of this patter is messaging topic.
Dead Letter Channel (from Messaging Channels)


The Dead Letter Channel describe scenario, what to do if the messaging system determines that it cannot deliver a message to the specified recipient. This may be caused for example by connection problems or other exception like overflowed memory or disc space. Usually, before sending the message to the Dead Letter Channel, multiple attempts to redeliver message are taken.

Content-Based Router (from Message Routing)


Content-Based Router examines message contents and route messages based on data contained in the message.

Content Enricher (from Message Transformation)

Content Enricher as the name suggests enrich message with missing information. Usually external data source like database or web service is used. 

Event-Driven Consumer (from Messaging Endpoints)

Event-Driver Consumer enables you to provide a action that is called automatically by the messaging channel or transport layer. It is asynchronous type of pattern because receiver does not have a running thread until a callback thread delivers a message. Example: Message driven beans in Java EE.

Polling Consumer (from Messaging Endpoints)


Polling Consumer is used when we want receiver to poll for a message, process it and next poll for another. What is very important is that this pattern is synchronous because it blocks thread until a message is received. This is in contrast with a event-driven consumer. An example of using this pattern is file polling.

Wire Tap (from System Management)

Wire Tap copy a message and route it to a separate channel, while the original message is forwarded to the destination channel. Usually Wire Tap is used to inspect message or for analysis purposes.

We can make use of the following open source products for Enterprise integration:
1. Apache Camel - framework enabling configuration of routes (from one endpoint to another and the different kinds of intermediate processing that can be done on the messages between the end points based on EIP patterns).
2. Apache Felix Karaf - OSGi container onto which the different enterprise integration services can be deployed. OSGi technology combines aspects of modularity, component-orientation, and/or service-orientation principles to define a dynamic service deployment framework that is amenable to remote management.
3. Apache ServiceMix - unifies several of Apache based open source service frameworks together into a flexible integration container. It provides a complete, enterprise ready ESB exclusively powered by OSGi.

The main features of ServiceMix are:

• reliable messaging with Apache ActiveMQ
• messaging, routing and Enterprise Integration Patterns with Apache Camel
• WS‐* and RESTful web services with Apache CXF
• OSGi-based server runtime powered by Apache Karaf

Apache Camel

Apache Camel is an open source Java framework that focuses on making integration easier and more accessible to developers. It does this by providing:

•     concrete implementations of all the widely used EIPs

•     connectivity to a great variety of transports and APIs

•     easy to use Domain Specific Languages (DSLs) to wire EIPs and transports together

Components are the extension point in Camel to add connectivity to other systems. Components provide an Endpoint interface. By using URIs, you can send or receive messages on Endpoints in a uniform way. For instance, to receive messages from a JMS queueaQueue and send them to a file system directory "/tmp", you could use URIs like "jms:aQueue" and "file:/tmp".

Processors are used to manipulate and mediate messages in between Endpoints. All of the EIPs are defined as Processors or sets of Processors.

To wire Processors and Endpoints together, Camel defines multiple DSLs in regular programming languages such as Java, Scala and Groovy. It also allows routing rules to be specified in XML. 

Example: Java DSL

view source

print?

1.from ("file:/tmp").to("jms:aQueue");

In the above example we define a routing rule that will load files in the “/tmp” directory into memory, create a new JMS message with the file contents, and send that message to a JMS queue named aQueue.

The basic unit of work in Apache Camel is route. Route is simply a flow of EIPs, where an output of one EIP is an input of the other.

Example application:
Problem: I would like to sort the xml files based on their content. When xml element denoting person’s city has value ‘London’ then file needs to be copied to messages/uk directory, in other case tomessages/others directory. Additionally I want to log each copy operation. 

EIPs flow/route diagram:

Apache Camel Context configuration:

1:    
2:      
3:        
4:        
5:          
6:          /person/city = 'London'  
7:            
8:            
9:          
10:          
11:            
12:            
13:          
14:        
15:      
16:    

As the name suggests, from is a declaration of source endpoint that listens to events / polls events. In this example, endpoint is a file poller.

http://camel.apache.org/components.html - Lists all Camel components.

To use Camel with Apache ServiceMix, refer - http://servicemix.apache.org/docs/5.0.x/quickstart/camel.html.