locked
how to apply correlation in orchestrations with example RRS feed

  • Question

  • HI all,

      There is a requirement explaining below can you please let me know the possible ways to approach this.

    1) The total process is running on file adapter.

    2) i will be receiving number of messages in the File receive location, process it in the BizTalk and then transfer it to file share in the send port.

    3) Customer will take the files from the send port and review it and then drops the files in another folder within two days.

    4) orchestration will listening for 2 days depending on the files received and correlate to the previous send files

    5) receive the messages from the listen shape and then send it to another file send port.

    can you please describe how can we acheive this scenario with brief descrption of the steps and the work to be done.


    Regards, Bala.

    Thursday, August 1, 2013 12:15 AM

Answers

  • You've done a pretty good job of defining the steps yourself. There's really not much else to consider.

    Setting up the correlation is the most complicated part of your scenario.  Here's an example: http://mstecharchitect.blogspot.com/2009/03/message-correlation-in-biztalk-2006.html

    That and using the Listen shape in a loop.  There you have a situation similar to a Sequential Convoy.  Example: http://msdn.microsoft.com/en-us/library/aa561843%28v=BTS.20%29.aspx

    That's assuming you will be receiving multiple responses.  If it's only one, you'd just have the Listen Shape.

    Thursday, August 1, 2013 2:11 AM
    Moderator
  • Hi Bala,

    You can achieve your requirement by implement any of the following ways:

    Synchronous: As suggested by boatseller, use correlation.

    • Receive the message.
    • Process it in orchestration, initialise the correlation and send the message.
    • Let the Orchestration to wait/dehydrate for 2 days or till you get the response.
    • Once response is received, follow the initialised correlation and complete the rest of the process.

    Asynchronous: You can consider this option for your scenario as well:

    • Receive the message.
    • Process it in orchestration; insert the correlated/key field in your custom database. Send the message out.
    • No dehydrated instances in this case. No waiting instances.
    • Once the response is received, compare the correlated/key field in the response to see where we received the corresponding request message by comparing it against the data in your custom database. If so proceed with the rest of the process.

    You can choose anyone of the above options based on various factors. As you mention, if your going to receive more message and waiting time for the response is 2 days or more you can consider using asynchronous option. No dehydrated messages, less in manageability, no risk to losing any dehydrated message etc. You can consider synchronous implementation using correlation and dehydrations if the messages are less and the response message is received is lesser waiting time. This method is easier to implement and does have its drawback with the message volume and waiting time for the response message increase.

    And to reply to your another question to use any of the default BizTalk properties for correlation, default BizTalk properties are not going to be unique for your request and response messages. Any message which is coming into BizTalk will have its specific properties and you can’t create the correlation based on those default BizTalk properties. Think it in common business logic. When you send a Purchase-OrderRequest message with a unique Purchase-Order number to the destination system, if the destination system gives you a Purchase-OrderResponse with a different number, how do you correlate this response message to the many request messages you have sent? You need to have some unique ID in both the request and response to differentiate this set/pair from others.


    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful.

    Thursday, August 1, 2013 8:46 AM
  • No need of depend on the xml schema for promotions. You can create a Property schema , define properties  with MessageContextPropertyBase .

    Create a Custom pipeline component ,Promote the property with File.ReceivedFileName .

    In Orchstration you can play like initializing/following correlations :-)

    Check the link for creating property schema with MessageContextPropertybase-->  http://masteringbiztalkserver.wordpress.com/2011/03/08/difference-between-messagecontextpropertybase-and-messagedatapropertybase/

    Check this link to promote property using pipeline component


    Srikanth Peddy. MCTS-BizTalk Server Please mark as answered .

    • Marked as answer by balagangadhar Friday, August 2, 2013 12:17 AM
    Thursday, August 1, 2013 9:25 AM
  • Hi Bala,

    For example, if you process the Purchase-Order, request and response messages. In this way you need to two orchestrations:

    Request - Process/Orchestration #1:

    • Receive the message Purchase-OrderRequest message.
    • Process it in orchestration; access the Purchase-OrderID from the message using XPath.
    • Insert the accessed Purchase-OrderID value into your custom database table using a C# helper method. Let’s say in to a Purchase-Order table.
    • Send the Purchase-OrderRequest message out.

    Response - Process/Orchestration #2:

    Create another Orchestration, completely isolated of your request orchestration/process.

    • Receive the response message.
    • Process it in the response (2nd) orchestration; access the Purchase-OrderID from the response message using XPath.
    • Check where the Purchase-OrderID exist in your custom Purchase-Order table. If exists, then it means Purchase-OrderRequest for this Purchase-OrderID is already processed/sent to destination. Otherwise you can throw error/exception or do anything for unknown response.
    • Send/Process the Purchase-OrderRequest out.

    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful.



    Thursday, August 1, 2013 9:46 AM

All replies

  • You've done a pretty good job of defining the steps yourself. There's really not much else to consider.

    Setting up the correlation is the most complicated part of your scenario.  Here's an example: http://mstecharchitect.blogspot.com/2009/03/message-correlation-in-biztalk-2006.html

    That and using the Listen shape in a loop.  There you have a situation similar to a Sequential Convoy.  Example: http://msdn.microsoft.com/en-us/library/aa561843%28v=BTS.20%29.aspx

    That's assuming you will be receiving multiple responses.  If it's only one, you'd just have the Listen Shape.

    Thursday, August 1, 2013 2:11 AM
    Moderator
  • Hi , Thanks a lot for the reply. This would works fine if i promote a property according to the article sent by you but as per my scenario,we should not depend on the Promotions in sth eschema. if we are assigning as System.xml.xmldocument , then need to maintain the correlations on the default Biztalk properties then which one can i pick and assign to correlation. can you please explain with steps.

    Regards, Bala.

    Thursday, August 1, 2013 4:48 AM
  • Hi Bala,

    You can achieve your requirement by implement any of the following ways:

    Synchronous: As suggested by boatseller, use correlation.

    • Receive the message.
    • Process it in orchestration, initialise the correlation and send the message.
    • Let the Orchestration to wait/dehydrate for 2 days or till you get the response.
    • Once response is received, follow the initialised correlation and complete the rest of the process.

    Asynchronous: You can consider this option for your scenario as well:

    • Receive the message.
    • Process it in orchestration; insert the correlated/key field in your custom database. Send the message out.
    • No dehydrated instances in this case. No waiting instances.
    • Once the response is received, compare the correlated/key field in the response to see where we received the corresponding request message by comparing it against the data in your custom database. If so proceed with the rest of the process.

    You can choose anyone of the above options based on various factors. As you mention, if your going to receive more message and waiting time for the response is 2 days or more you can consider using asynchronous option. No dehydrated messages, less in manageability, no risk to losing any dehydrated message etc. You can consider synchronous implementation using correlation and dehydrations if the messages are less and the response message is received is lesser waiting time. This method is easier to implement and does have its drawback with the message volume and waiting time for the response message increase.

    And to reply to your another question to use any of the default BizTalk properties for correlation, default BizTalk properties are not going to be unique for your request and response messages. Any message which is coming into BizTalk will have its specific properties and you can’t create the correlation based on those default BizTalk properties. Think it in common business logic. When you send a Purchase-OrderRequest message with a unique Purchase-Order number to the destination system, if the destination system gives you a Purchase-OrderResponse with a different number, how do you correlate this response message to the many request messages you have sent? You need to have some unique ID in both the request and response to differentiate this set/pair from others.


    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful.

    Thursday, August 1, 2013 8:46 AM
  • HI Prabhu,

    Thanks for the reply it was great suggesting in the two ways. can you please briefly describe the steps to be done to achieve the second approach as it is very useful solution for my scenario.


    Regards, Bala.

    Thursday, August 1, 2013 8:54 AM
  • No need of depend on the xml schema for promotions. You can create a Property schema , define properties  with MessageContextPropertyBase .

    Create a Custom pipeline component ,Promote the property with File.ReceivedFileName .

    In Orchstration you can play like initializing/following correlations :-)

    Check the link for creating property schema with MessageContextPropertybase-->  http://masteringbiztalkserver.wordpress.com/2011/03/08/difference-between-messagecontextpropertybase-and-messagedatapropertybase/

    Check this link to promote property using pipeline component


    Srikanth Peddy. MCTS-BizTalk Server Please mark as answered .

    • Marked as answer by balagangadhar Friday, August 2, 2013 12:17 AM
    Thursday, August 1, 2013 9:25 AM
  • Hi Bala,

    For example, if you process the Purchase-Order, request and response messages. In this way you need to two orchestrations:

    Request - Process/Orchestration #1:

    • Receive the message Purchase-OrderRequest message.
    • Process it in orchestration; access the Purchase-OrderID from the message using XPath.
    • Insert the accessed Purchase-OrderID value into your custom database table using a C# helper method. Let’s say in to a Purchase-Order table.
    • Send the Purchase-OrderRequest message out.

    Response - Process/Orchestration #2:

    Create another Orchestration, completely isolated of your request orchestration/process.

    • Receive the response message.
    • Process it in the response (2nd) orchestration; access the Purchase-OrderID from the response message using XPath.
    • Check where the Purchase-OrderID exist in your custom Purchase-Order table. If exists, then it means Purchase-OrderRequest for this Purchase-OrderID is already processed/sent to destination. Otherwise you can throw error/exception or do anything for unknown response.
    • Send/Process the Purchase-OrderRequest out.

    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful.



    Thursday, August 1, 2013 9:46 AM
  • Hi , Thanks a lot for the reply. This would works fine if i promote a property according to the article sent by you but as per my scenario,we should not depend on the Promotions in sth eschema. if we are assigning as System.xml.xmldocument , then need to maintain the correlations on the default Biztalk properties then which one can i pick and assign to correlation. can you please explain with steps.

    What is the reason you cannot depend on Promotions in the schema?

    Next, do you really need to correlate the returned message back to the original instance?  If not, then this whole discussion is mostly irrelevant.

    Assuming you do need to correlate, there has to be something about the returned message you can use to correlate on.  There's no way around that.

    However, it does not have to be a Schema Promoted Property.  For example, the file name can be used, but you'd have to manually Promote that value either in a Pipeline Component or an Orchestration so it can be routed back to the original instance.

    Thursday, August 1, 2013 12:36 PM
    Moderator
  • HI Srikanth,

      Thanks a lot for the suggestions provided. can you please provide some code or example to write custom pipeline component for promoting the properties.

    if possible can you send me your mail id to contact for further queries if i have in Biz Talk please.


    Regards, Bala.

    Friday, August 2, 2013 12:16 AM
  • HI Prabhu,

    Really great and thanks for the steps provided clearly.

    if possible can you send me your mail id to contact for further queries if i have in Biz Talk please, if you are OK.


    Regards, Bala.

    Friday, August 2, 2013 12:17 AM