locked
RDP to a Server 2008 R2 - Error in protocol stream

    Question

  • Machine A runs Server 2008 R2 with Hyper-V role enabled. Machine B is a virtual Server 2008 R2 on Machine A with one NIC "linked to" an external NIC of Machine A. Machine C is a Windows 7 machine trying to RDP to Machine B. This always gives a black screen (after authentication of the RDP session succeeded) shortly afterwards followed by an error message stating ...error in protcol stream .... (On machine B an event is logged in "system" nbr 56: The Terminal Server security layer detected an error in the protocol stream and has disconnected the client. Client IP: 192.168.0.13.

    Pinging Machine B from Machine C succeeds (any packet size and over 1400 for undefragmented) and accessing network shares on Machine B from Machine C also succeeds.

    Can anyone at least point me in the direction of the problem source? (RDP, Hyper-V, Server 2008 R2 ???)

    Wednesday, May 12, 2010 8:30 AM

All replies

  • hi VGDA, can you try removing NLA on the rdsh server and lowering encryption to see if the problem goes away?
    Monday, May 17, 2010 6:25 PM
  • Hi James,

    Thanks for the reply.

    In the "system properties" dialog box (of machine B) I have the "Allow connections from computers running any version of Remote Desktop" option selected. I assume that way NLA isn't used. I do not understand what exactly you mean by "lowering encryption", unless you are assuming I have the Remote Desktop role installed on the server. I am NOT running the server as a terminal server, I simply want to "RDP" to the server to "interactively" manage it.

    Kris.

    Tuesday, May 18, 2010 9:01 PM
  • Hello Kris,

    Please cite the exact error message to us. With that information we can investigate further.

    Thanks.

    Regards,
    Lionel Chen


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, May 19, 2010 3:25 AM
  • Hello Lionel,

    The message I see in the "system" Windows Log of Machine B has an Event ID Nbr 56. The text goes as follows: "The Terminal Server security layer detected an error in the protocol stream and has disconnected the client. Client IP: 192.168.0.13".

    The attempt to "RDP" to the server from the Windows 7 machine always first gives a black screen (after credentials were specified), shortly afterwards followed by a dialog box stating: "Your remote session has ended. The connection to the remote computer was lost, possibly due to network connectivity problems. Try connecting to the remote computer again. If the problem continues, contact your network administrator or technical support."

    Both machines are WORKGROUP members (workgroup name is WORKGROUP on both machines) and I try to connect from Machine C to Machine B using the latter's IP address (IPv4).

    Maybe also of interest: before getting the black desktop I get a warning that the server (to which I'm trying to connect (i.e. Machine B)) has an untrusted certificate (logical since it's self-generated by Machine B and not imported on machine C). I always confirm to proceed anyway, which gives me the black desktop.

    Hope this helps pinpointing potential problems.

    Regards,

    Kris. 

    Wednesday, May 19, 2010 5:10 PM
  • Some further info:

    When trying RDP from a Server 2008 R2 or Windows 7 machine that is also virtual and connected to the same "virtual switch" then there is no problem whatsoever. The black screen and disconnection thus only occurs when connecting from a physical machine to the virtualized server. This looks like a hardware/driver problem on the virtualization server (Machine A that has the Hyper-V role installed): anyone agrees/disagrees?

    Kris.

    Wednesday, May 19, 2010 6:09 PM
  • Hello Kris,

    What applications are installed on the RD Session Host side? Based on my expereince, some security components, such as Symantec Endpoint Protection will lead to similar issues with the same error message.

    Please check it firstly and let us know the result.

    Thanks.

    ·         Lionel Chen

    TechNet Subscriber Support in forum

    If you have any feedback on our support, please contact tngfd@microsoft.com


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, May 20, 2010 10:17 AM
  • Hello Lionel,

    This is a test environment. The servers are Server 2008 R2 with no 3rd party applications and the Windows 7 machines are strictly that: Windows 7, with nothing added. All machines are freshly installed and not even updated (not yet connected to the WSUS network).

    The only role enabled on the server I called Machine A is Hyper-V. Machine B has only the "File Services" role enabled.

    Machine A has two network cards:
    D-Link DGE-528T Gigabit ethernet adapter (this is the one currently being used by the virtual machines)
    Generic Marvell Yukon 88E8056 based ethernet controller

    Before you ask: having the virtual machines use the other NIC gives the same result :(

    Regards,

    Kris.

    Thursday, May 20, 2010 10:56 AM
  • Some more info:

    Opening an RDP session from Machine B to machine C (thus from the Server 2008 R2 machine to the Windows 7 machine) succeeds without problems). There goes the hardware/driver problem theory?

    Kris.

    Thursday, May 20, 2010 8:05 PM
  • on the win 2008 r2 server, if you open up rds sess host config (available even if you don't have rdssh installed), you should see an option labeled rdp-tcp under connections. 

    there should then be options for security layer and encryption level.

    try lowering those and ensure the portion below that is unchecked (allow connections only from computers running remote desktop with nla).

    Thursday, May 20, 2010 9:15 PM
  • Hello James,

    I tried "RDP security layer" as Security layer and "low" as Encryption level but to no avail. I doubt the problem is an RDP configuration problem since RDP from other virtual Server 2008 R2 and Windows 7 machines succeed without problems.

    I also tried to change the "Network adapter" settings tab and change the network adapter to use to "Microsoft Virtual Machine Bus Network Adapter" but that didn't change anything either. It seems it is impossible to RDP "from the outside" to any of the virtual machines (the inverse works without problems, i.e. from a virtual machine to the outside).

    Furthermore I noticed that, although network connections function (as in "map network drive") they are also interrupted intermittedly.

    I'll be trying to capture some network traffic from a physical and from a virtual machine when they connect to Machine B (for RDP and/or file transfer). I hope that'll learn me something. Simply trying out all possible combinations of all setings I can find will take far too long.

    Thanks for the reply anyway.

    Kris.

    Saturday, May 22, 2010 9:44 AM