L2TP/IPSEC VPN Connection Issue only Over wan ---- Error:789

Unanswered L2TP/IPSEC VPN Connection Issue only Over wan ---- Error:789

  • lundi 28 juin 2010 22:30
     
     

    I tested the VPN internally using the local ip address and it works fine. As soon as I try it from the wan on a remote computer it will not work. Gives me this

    "error 789: The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with remote computer"

    This is on a server 2008 r2 platform with a L2TP\IPSEC VPN with a preshared key.

    I only have mapped port 1701, do I need 4500 and 500 also?

     

    What I have checked so far

    I have mapped over the port (1701)  in the router, However when I use the online open port checker tool it cant find the service. I tested a http file server on that port and it saw my service to the port is not being block by my router. The Firewall is a set to allow the connection over the correct network interface. Edge translation is allowed

    Any ideas?

    • Déplacé Miles ZhangModerator jeudi 1 juillet 2010 07:41 (From:Windows Server 2008 R2 Networking)
    •  

Toutes les réponses

  • mardi 29 juin 2010 09:09
    Modérateur
     
     

    Hi,

    Thanks for the post.

    From your description, I understand that the error 789: "The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer" is received when trying VPN from the WAN on a remote computer.

    This is a generic error which is thrown when the IPSec negotiation fails for L2TP/IPSec connections.

    Possible causes for this issue could be:

    a> L2TP based VPN client (or VPN server) is behind NAT.

    b> Wrong certificate or pre-shared key is set on the VPN server or client

    c> Machine certificate or trusted root machine certificate is not present on the VPN server.

    d> Machine Certificate on VPN Server does not have 'Server Authentication' as the EKU

    Now please make sure correct certificate is used both on client and server side. In case Pre Shared Key (PSK) is used, make sure the same PSK is configured on the client and the VPN server machine.

    Hope this helps.

    Miles


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
  • mardi 29 juin 2010 13:56
     
     
    I am using a pre-shared key I have verified it is correct. I have been able to verifiy the VPN work by testing it locally using the local ip
  • mardi 29 juin 2010 14:15
     
     

     

    added 4500 and 500 with no luck...

    Router logs show its coming in
    [LAN access from remote] from  remoteip :4500 to 192.168.1.2:4500 Tuesday, Jun 29,2010 06:10:25
    [LAN access from remote] from  remoteip :500 to 192.168.1.2:500  Tuesday, Jun 29,2010 06:10:25

     

    Error changed to error 809

    Router setup Does this mean I can only have two clients? http://img638.imageshack.us/img638/9852/image3cvi.png
  • jeudi 1 juillet 2010 06:34
     
     
    Do you have any firewall which is placed in front of your Windows Server 2008 VPN Server? Do you have the Windows Server firewall turned on? If yes, try to have all these ports and protocols open on them: IKE: UDP Port 500 IKE/IPSec NAT-T: UDP Port 4500 IPSec ESP: IP Protocol 50 IPSec AH: IP Protocol 51 UDP L2TP port: 1701 Cheers
  • jeudi 1 juillet 2010 13:39
     
     
    Yes I have tired all those ports, the router is the front firewall. I have tried turning the firewall off on the server also. It works locally when I put in the local ip,  just wont work external from anywhere. I see connection trying to come threw on the router log but it never connects. I am just left to believe its the router, I am going to set up a ISA server or something.
  • vendredi 18 mai 2012 09:22
     
     

    Hi Everyone,

                 I am too having the same issue. We have Sonic wall Firewall I have opened all the proper ports and the packet capturing show that all the traffic is being forwarded to my VPN Server even I have checked that the traffic is receiving at VPN Server but Still Client gets "Error: 789" also the sonic wall support team has verified that the traffic are forwarded to the VPN Server and suggested me to check VPN Server Settings or consult to Microsoft..

    Thanks,

    Mukesh Rebari.

  • jeudi 14 juin 2012 21:47
     
     

    Hi,

    I'm also having the same issue and it just started a couple of weeks ago. Before that, everything worked fine.

    I confirmed my server (Win 2k3) receives the ISAKMP packets but it never replies anything to the client!

    Tested it with Windows 7 and XP SP3 and always have the same behaviour, 4/5 ISAKMP packest received, no replies to the client of any sort....

  • jeudi 14 juin 2012 22:49
     
     

    For some reason (I suspect MS12-034 updates) my UDP encapsulation setting on the registry disappeared and the server was unable to understand the UDP packets that was receiving to establish the connection.

    All working now.

    http://support.microsoft.com/kb/885407