Handling Large messages in BizTalk RRS feed

All replies

  • You'll put a lot of strain on the message box doing that, the document you linked has lots of useful guidance.  If you're processing these messages in addition to standard size messages the you may want to build a second instance that you can tweak to this sort of work load.

    My main suggestion would be is - do you have to process/transform the whole message?  Often when dealing with large files, we land up just passing data through (binary chunks etc).  What you can do in this situation is dump the large parts of the data to disk in the receive pipeline and then pick it up again in the send pipeline.  Thus keeping the load off of your database.

    I would pay particular attention to ensure that all your work load is done using streaming instead of things like loading up the whole message into an XmlDocument etc., as you'll want to keep your memory usage in check as well.

    You should also consider what will happen with throttling if the destination is unavailable and large amounts of messages begin to pool.

    If this is helpful or answers your question - please mark accordingly.
    Because I get points for it which gives my life purpose (also, it helps other people find answers quickly)
    Read my articles on: BizTalk | .NET

    Thursday, August 29, 2019 11:52 AM
  • if it's just routing then above approch is fine.but we have to transform whole message in biztalk and send it to target what is the best design approach to handle the scenario
    Thursday, August 29, 2019 12:10 PM
  • Are you actually having any issues?  e.g. Errors or Throttling happening due to memory pressure?  

    If you are then you need to do things like

    • Have that process on a separate host/host instance so you don't impact other processes.
    • Look at increasing the memory on the BizTalk Server
    • Sometimes you need to use XSLT rather than the BizTalk map surface if there are inefficient looping occurring in the map.
    • Move the BizTalk TEMP folder to a non-system drive.
    • Consider using something else other than BizTalk.   BizTalk is very good at lots of small messages, but with many large messages might be better off using something like SSIS.

    Sunday, September 1, 2019 11:19 PM