none
Biztalk SFTP Adapter Issue in Receive Location. RRS feed

  • Question

  • Hi !

    We have a Biztalk 2013 R2 environment. We use the out of the box SFTP adapter to connect to sftp receiver.

    We have about 30 Receive locations pointing to different folders on the same sftp server (with different folders / users).

    Issue : Very often the files get stuck on SFTP server and unless the receive location / host is restarted,  they wont pick up. 

    What is already tried : Setting connection Limit at receive locations to 50 , making polling interval at 60 secs (instead of default 5 secs). None of this works.

    Anyone facing similar issues ? Any help is appreciated. Thanks in advance.


    Wednesday, May 18, 2016 4:39 AM

Answers

  • We could not fix the issue with Biztalk SFTP adapter.  We did work arounds and with help of network team , we now have a FILE share location instead.

    However, we tested thoroughly and found  that nsoftware SFTP did NOT have this issue. If you are in a position to purchase third party adapter , you could consider this option. 

    Monday, October 3, 2016 6:45 AM

All replies

  • Hi

    First make sure you are on CU2-

    https://support.microsoft.com/en-in/kb/3119352

    There were multiple fixes for the SFTP adapter after R2 release.


    Thanks Arindam

    Wednesday, May 18, 2016 4:46 AM
    Moderator
  • Thanks for reply. Forgot to mention that , we already have CU2 installed. The receive locations do not stop , there is no error / warning in event viewer . Yet the files do not get picked up until restart. 
    Wednesday, May 18, 2016 5:55 AM
  • Hi 

    During the time when the issue starts, do you see any logs on the SFTP server regarding the BizTalk SFTP connections?

    Maybe you can start a network capture (using Netmon or WireShark) when the issue starts to see if there is any communication over SSH when the Receive Locations apparently freeze.


    Thanks Arindam

    Wednesday, May 18, 2016 6:46 AM
    Moderator
  • Thanks for your reply.  Logs do not indicate anything. Sometimes in event viewer we see Remote Directory listing error. This is random and goes away on its own. Or because we keep restarting hosts / receive locations.

    We have almost concluded that there are bugs in native SFTP adapter.  

    Tuesday, May 31, 2016 5:56 AM
  • Apart from network capture using Wireshark/Netmon, WCF traces may point in the right direction, if you capture both when the issue starts happening-

    https://msdn.microsoft.com/en-us/library/ms733025%28v=vs.110%29.aspx


    Thanks Arindam

    Tuesday, May 31, 2016 6:28 AM
    Moderator
  • Hi,

    We are running into the exact same issue - multiple SFTP receive locations to the same server for different applications and the files just stop getting picked up. No error messages, no log entries, nothing to point to any particular issue that might be causing it. No changes to the configuration settings on any of the receive locations seem to make any difference. We believe the issue may be caused by thread exhaustion - the SFTP adapter does not appear to be releasing resources. We monitored the number of threads being generated by the SFTP adapter and watched it rapidly climb to a particular number (can't recall the exact value) but when it hit that ceiling, no more threads were generated and files stopped getting picked up. As soon as we restart the host instance and released the threads, files would start getting picked up immediately - until the thread count hit that ceiling and once again the files stopped getting picked up. As a work around we have had to create numerous host instances for the SFTP applications to split the load and try to increase the overall number of available threads.


    Thanks, Bruce

    Thursday, August 4, 2016 5:04 PM
  • Are all these receive locations on the receive ports to the same hosts to the same remote location?

    I assume your send ports are set up the same?

    It may be useful to create seperate receive ports, and at least two hosts for sending and receiving to just check that you are not running into a kind of thread starvation caused when a port is doing too many things at once.

    Yours

    Oli Rhys

    Friday, August 5, 2016 1:45 PM
  • Hi,

    What kind of volume you are having on these SFTP location? we had a similar scenario with a high volume workflow host was shared with another SFTP host and they started to show fault on the host, later teh high volume host was isolated and it kind of corrected the issue. It may work for you as well.


    Regards Pushpendra K Singh

    Wednesday, August 10, 2016 7:49 PM
  • Hi,

    We had faced a similar issue.

    Create separate host for send ports ,receive ports and orchestrations. 

    Increase the polling interval and also set different service schedule window.

    Is the ConnectionLimit value set to the default of 5?

    Thursday, August 11, 2016 12:53 PM
  • Hi,

    We are facing a similar issue, were you able to resolve this.

    If yes request to share the solution.

    Thanks

    Kamal

    Monday, October 3, 2016 2:22 AM
  • Hi kamalPat,

    have you tried the suggestions mentioned above

    1. make sure that you are on the latest CU for your BTS version
    2. make sure that you have dedicated Hosts for the SFTP locations
    3. Ensure that your Host are not throttling...if they are analyze the load that youi have on your servers

    Regards


    Mandar Dharmadhikari

    Monday, October 3, 2016 3:41 AM
  • Hi NdBot,

    This could be the issue with the SFTP adapter. We have witnessed the same behaviour but with an external SFTP adapter provider nsoftWare and they are matured and have one of the best SFTP adapters in market.

    We had a similar issue with them as well and the issue was resolved in V4 of the adapter.

    Examine the event viewer closesly to look for any events around the time. It can be a generic error that time but you will notice the pattern.

    We created a event job to restart the SFTP rcv host instance whenever such event was logged and then it took care of the env. You may want to have something similar till the isse is fixed in next CU of next release.


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool

    LinkedIn: LinkedIn

    Monday, October 3, 2016 6:37 AM
  • We could not fix the issue with Biztalk SFTP adapter.  We did work arounds and with help of network team , we now have a FILE share location instead.

    However, we tested thoroughly and found  that nsoftware SFTP did NOT have this issue. If you are in a position to purchase third party adapter , you could consider this option. 

    Monday, October 3, 2016 6:45 AM
  • Ya true Nsoftware is not having this issue. But Version 4.

    They had this issue in V3


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool

    LinkedIn: LinkedIn

    Monday, October 3, 2016 6:48 AM
  • Or use this free adapter:

    https://sftpadapter.codeplex.com/


    Did my post help? Please use "Vote As Helpful", "Mark as answer" or "Propose as answer". Thank you!

    Monday, October 3, 2016 7:17 AM
  • hi ,

    Did you find any solution for your issue.. I am also facing same issue.. I had 35 Recieve Locations . When I am configuring 10-12 Recieve locations, everything is working fine , but when I am configuring all 35 Recieve location using SFTP locations, then files are not getting picked up, however when i restart the host instance, everything working fine, i.e files getting picked up..

    Please update if you figured out any solution ??

    Sunday, May 13, 2018 6:43 PM
  • That very much sounds like this issue for which they issues a fix, for BizTalk 2016 only though in CU5 (June 25, 2018FIX: SFTP adapter receive locations stop working intermittently during higher load in BizTalk Server

    We can only hope that a similar fix is coming for BizTalk 2013 R2


    Thursday, October 4, 2018 8:28 PM
  • Has anyone else worked out what the issue is?

    After having this issue a few times we made sure that all of the SFTP receive locations were on their own host instance so we could restart it that without impacting anything else.  However now even that doesn't work, and the host instance will not stop in a timely manner and it doesn't resolve the issue.  We have to had to reboot the server twice now to get it to work.

    Thursday, June 6, 2019 2:37 AM