Forefront TMG IPSEC tunnel to OpenSwan or Racoon drops after random time

Fri, 03 Jul 2009 20:18:00 Z

Hi ,

In my company we have experience serios problems with IPSEC tunneling .Here is the situation
We have one (1) Forefront TMG running on Quad Core with 3 GB of ram and two interfaces

One of the interfaces is connected to our corporate network - 10.10.10.0/24 and the other interface if connected to public network.
Nothing strange here :)

So we decide to create IPSEC tunnel to our customers to make thrust with their Domain Controllers and other stuff

At our side is Forefront TMG Beta 3 and at the remote point is Ubuntu Server with the latest version of OpenSwan ,StrongSwan or Racoon (Linux stuff :) )
We are configuring IPSEC tunel to work with Preshared Keys and the tunnel to the remote endpoint is bring up.Everything is working Fine but we found that some of the tunnels are going down at random intervals.

At this point we decide that the problems are come from bad link to our customers.(ISP going down ot something) .Couple of months we are restarting the tunnels and test different configurations but still the tunnels are going down randomly .When we start debugging and reading hundreds of articles we found that after changing the times in Quick mode and Main mode some of the links are corrented and works persistently.We found that if we change the encryption some of the tunnels works fine.

This was for about 3-4 months.Every day two or tree times we are resetting the tunnels that are going down.

At the debugging level everything seems to be fine.

Quick Mode starts ,main mode starts and works ,at the time when the keys are re-changed the link seems to be stop and the tunnel is going down.
We had tryed many different configurations - low securty ,different pre-Shared keys,different times for QM and MM but still no luck .

At this time we connect a few ISA 2006 and Linksys IPSEC clients to our Forefront and everithing works fine
Everytime that we are trying to establish IPSEC to Linux with StrongSwan ,OpenSwan or Racoon we going through IPSEC ____.Everytime out Linux guys told to us that the problem is at our side and it seem to be

If we create IPSEC VPN with StrongSwan to other StrongSwan the tunnel is rock solid
If we create IPSEC VPN with Forefont TMG to ISA 2006 - no problems
Forefront TMG --- StrongSwan = Hundreds of hours trying to bring the tunnel UP and monitoring it not to be goind DOWN.

After that we decide to forget Forefront BETA 3 and return to ISA 2006 but the problems with the tunnels still persist.

Some times they start,some times not.We checked configurations thousands times - Everithing is OK

At the end i am going to give up everithing.12 different tunnels to 12 different remote Networks(with no overlap of IP addressing)
3 -4 of them works fine ,the others going down at random times.

Please HELP

if someone wants i can parse Logs from TMG and Strongswan,racoon,Openswan.

THe connections is like this
Our office with FF TMG and IPSEC site to site VPN <----Internet--->Remote customer office with Ubuntu or Debian with configured IPSEC
Random time works,random time not.
If we reset the tunnel at the remote point the tunnel bings up for some time.

Sorry for my bad English

Forefront TMG IPSEC tunnel to OpenSwan or Racoon drops after random time

Thu, 09 Jul 2009 19:23:55 Z

Since you have confirmed the issue seems to be seen even with ISA 2006 , IPsec in TMG as actually making use of Windows IPsec capabilities ( as in ISA 2006) and we just createt filters to allow the traffic and it looks like we do create the filters fine as you seem to establish tunnel fine but only the rekeying is causing some grief.

This can be delat by our regular support line ( for ISA 2006)

http://support.microsoft.com/oas/default.aspx?gprid=11928&timestmp=633827361997460260&acty=ProductList&ctl=productlist&wf=PID&trl=PID%7eProductList&ln=en-us&prid=10405&gsaid=455044

Please send me a mail directly and i will provide a way to collect data and work with our ISA 2006 support team

Bala Natarajan [MSFT]| Sr. Support Escalation Engineer | CSS Security -TMG Beta support team | Email: bala.natarajan@microsoft.com | Office 425.704.4626 Bing it on bing.com