Gold Award WinnerGold Award Winner


Introduction

In the BizTalk world the concept of convoys is well known. Although it is not the same as the general meaning of the word convoy where a group of vehicles, typically motor vehicles or ships, traveling together for mutual support and protection. In the context of BizTalk a convoy is a group or set of messages that are related to achieve a required result. And there can be a business requirement moving you to use convoy. A convoy is known as an aggregator- or scatter-gather pattern, which can support you in aggregating messages. The aggregator- and scatter-gather pattern are categorized as integration patterns, or known as Enterprise Integration Architecture (EAI) patterns, written down by Gregor Hohpe, Bobby Woolf. And these patterns vary from message routing to messaging channels. You can find them on their landing page online http://www.enterpriseintegrationpatterns.com/.

There are two kinds of convoys possible:

  • Parallel Convoy: receive the related messages in any order.
  • Sequential Convoy: receive messages in a predefined order. 

A parallel or sequential convoy can implemented in BizTalk to achieve control of the incoming messages and order how these are processed. The concept of a convoy can be used to support implementation of aggregation or resequencer pattern. 

Parallel Convoy

The following business scenario that requires a parallel convoy can be when a patient is admitted to a hospital. And the requirements for this process are the receipt of three different types of messages before the patient can be admitted into the hospital. These messages contain insurance information, patient history, and general patient data. Note that any of these messages can be the first message for the process of admittance and can create a race condition. To prevent that from happening, three Receive shapes in an orchestration are put into a Parallel Action shape and subsequently for each receive shape the Activate property will be set to “True”. This means any one of the messages can start the orchestration. However all three messages are required before the rest of the orchestration can run and can be combined (aggregated) to one message before entering the hospital system that regulates the admittance.

Picture 1. Schematic diagram of the aggregator pattern patient admittance scenario.

A parallel convoy consists of a correlation set and Parallel Actions shape. Each receive shape in Parallel Actions shape initializes the correlation set, which is based upon for instance Social Security Number (SSN) and PatientID. With a correlation set BizTalk is instructed to associate the correlation type data with the orchestration instance, so it can route all messages that have identical correlation type criteria to same instance. In this case it will be with a SSN and PatientID. And each receive shape has its Activate property set to true and the Initializing Correlation property configured to the same correlation set. One of three receive shapes will receive the first message and handle the activation of the orchestration instance and initialization of the correlation set. Regardless of the fact that the other receive shapes also have the Activate property set to true it will not result in activating a new orchestration instance.


Picture 2. Screenshot of an orchestration resembling the implementation of a parallel convoy.

In case of scatter-gather a different scenario would be applicable. Consider a person looking for the best suitable car insurance based on price. He could through a web site (application) request for the best quote and the request is sent to multiple vendors. Every response is gathered, evaluated and the best quote is pushed back to the web site (app) for the person to review. The gathering of responses can be supported using a parallel convoy.


Picture 3. Schematic diagram of the scatter-gather messaging pattern.

Sequential Convoy

In a scenario, where resequencing of messages is required, a sequential convoy can handle the race condition that occurs when BizTalk Server attempts to process subscriptions for messages received at the same time. A business case can be when orders from a client during the day have to be batched at a certain time before processing them. The order of incoming orders have to be preserved. In case orders cannot be fulfilled the early orders are processed first. 

Picture 4. Schematic diagram of the resequencer messaging pattern.

The implementation would be of a uniform sequential receive convoy in BizTalk. Since many instances of a message have been received and must all be handled by the same business process. Two successive Receive shapes are placed into an orchestration. The first shape will be able to start a new orchestration as the Activate property is set to "True"  and initializes a correlation set to be used for the subsequent (second) Receive shape, which follows this correlation set.


Picture 5. Screenshot of an orchestration resembling the implementation of an uniform sequential convoy.

You can use a Uniform Sequential Convoy implementation within an orchestration to sequentially handle messages. There is a correlation set specified and the ordered delivery flags are set in the receive and send port. First receive shape initializes the correlation set and instructs the BizTalk runtime to associate the correlation type data with the receive port name by which the message was consumed. Subsequently it will associate the correlation type data with the orchestration instance. All messages with identical correlation type criteria, for instance an ID, will be routed to same instance. With the Ordered Processing flag the BizTalk runtime will be instructed to maintain order determining which message should be delivered to the orchestration instance.

Next to a Uniform Sequential Convoy you can implement what’s called Non-Uniform Sequential Convoy. The principle behind this type of convoy is that two or more different types of messages are received in a known order before the rest of the business process can continue. To implement this type of convoy you must position multiple Receive shapes in order inside the orchestration. The first Receive shape activates orchestration by setting the Activate property to "True". The first Receive shape in the convoy initializes a correlation set, which is followed by all the subsequent Receive shapes. The individual messages must all originate from the same port and this port must be marked for ordered delivery and each message is associated with a separate operation inside the port. This differs with the Uniform Sequential Convoy, where messages are associated with one operation in the port.

See Also 

This is not the first article on aggregator and resequencing pattern in BizTalk as there are quite a few available online. The best resources in my view are:

With Convoys you also will encounter zombies, which are described in the following TechNet Wiki Articles:

A general landing page to find message pattern with BizTalk on the TechNet wiki is:

Both parallel and sequential convoys samples are available through MSDN Code Gallery:
Another important place to find an extensive amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.