none
Event 4005 - WinLogin [Windows logon process has unexpectedly terminated]

    Question

  • Folks,

    Just noticed this becoming an issue on a Windows 2012 R2 Terminal Server after the last round of Patch Tuesday updates stemming from August 9th, 2016.

    Typically, I'm rebooting the server every 24 hours to over-correct the issue - rebooting not being the best option here.  

    In previous discussions, it's advised to remove KB3002657 or KB3035132 from the server.  Is this still the best option to restore full functionality even with the last round of patches and updates? Just to confirm, we are not using webroot as an AV solution. 

    Monday, August 15, 2016 11:01 AM

Answers

All replies

  • Hi,

    I am not able to find similar issue caused by the update you mentioned.

    Please also check terminal services related logs to see whether more clues can be found, which are under:

    Event Viewer -> Applications and Services Logs\ Microsoft\ Windows

    Best Regards,

    Amy


    Please remember to mark the replies as an answers if they help and unmark them if they provide no help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, August 16, 2016 7:25 AM
    Moderator
  • As of today we experienced the same issue with a 3 Server 2012r2 Remote Desktop Server farm. Some users had a successful logon, other users had a black screen while connecting with RDP. When some users went for lunch (session disconnect) and came back, their session did not reconnect (black screen in RDP, winlogon 4005 event)  

    All 3 servers are recently updated with updates from patch Tuesday (august 9th). Updates where installed on the 10<sup>th</sup> of august. Today we experienced connection issues for the first time.

    Updates installed on August 9<sup>TH </sup>KB3172614, KB3175443, KB3175887, KB3172729, KB3167679, KB3177108, KB3178034, KB3177725, KB890830

    We tested logon with a single user. When we logon locally from console (as a test) the user session did start successfully. When we try to connect with RDP we had a black screen. So something is going on with RDP… We had no strange logdata from the eventviewer or other indications that something else is wrong. We first rebooted the connection broker servers. No change in behavior. After we rebooted the Remote Desktop Session Host servers users where able to logon again, for now..

    A reboot fixed the problem (workaround). We are waiting for a second occurrence for further actions.
    Tuesday, August 16, 2016 2:38 PM
  • Also experienced the same issue in a TS pool of 30 servers updated last week.  Only about 2 servers encountered the issue per day, until we finally removed all but 1 update from the servers.  We left a different update installed on each server to trackdown the culprit.

    Thanks,

    Tuesday, August 16, 2016 4:15 PM
  • Hi,

    Thank you for sharing this information with us, kindly let us know if the cause has been found!

    Best Regards,

    Amy


    Please remember to mark the replies as an answers if they help and unmark them if they provide no help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Wednesday, August 17, 2016 12:33 PM
    Moderator
  • This is identical to our issue you listed above... thanks for sharing.

    As mentioned, the work-around seems to only hold for 24 hours then completely locks the user with a black screen, then a session disconnect.

    In total, we've recorded 117 events in the last 7 days -- 17 in the last 24 hours.  There's no rhyme or reason as to why certain users are locked out versus users that are successfully able to log-in.

    I did notice 6 additional updates this morning (August 17th) but this has no affect to the issues currently reported.


     

    Thursday, August 18, 2016 12:28 AM
  • Hi,

    As this issue seems to be common, if possible, I suggest you remove KB mentioned above one by one to see which one is causing the problem.

    Best Regards,

    Amy


    Please remember to mark the replies as an answers if they help and unmark them if they provide no help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, August 18, 2016 12:11 PM
    Moderator
  • I've removed the following updates listed above, along with KB3172614 and we're still seeing the same issues as described earlier.  Contributors to Spiceworks have also picked up on the problem... 

    https://community.spiceworks.com/topic/1570976-server-2012-rds-winlogon-process-crashing-event-id-4005?page=3#entry-6121386


    Monday, August 22, 2016 11:37 AM
  • Hi,

    In that case, I would suggest you contact Microsoft Customer Support and Services where more in-depth investigation can be done so that you would get a more satisfying explanation and solution to this issue.

    In addition, if the issue has been proved as system flaw, the consulting fee would be refund.

    Best Regards,

    Amy


    Please remember to mark the replies as an answers if they help and unmark them if they provide no help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, August 23, 2016 7:49 AM
    Moderator
  • Any one getting anywhere with this issue?

    It is becoming a Daily issue for us now on our RDS Farm.

    Rebooting the host fixes this issue but this is a busy production environment so this is not practical.

    Thanks

    Jon

    Wednesday, August 24, 2016 10:03 AM
  • Any one getting anywhere with this issue?

    It is becoming a Daily issue for us now on our RDS Farm.

    Rebooting the host fixes this issue but this is a busy production environment so this is not practical.

    Thanks

    Jon

    We have opened a case with Microsoft for this issue; they said to remove KB3172614. We removed and blacklisted the update across 15 2012 R2 RDS session hosts on Sunday, we have not seen the issue return yet. 
    Wednesday, August 24, 2016 11:22 PM
  • We are uninstalling KB3172614 on some servers this evening. We'll report back if this resolves the problem for us as well.
    Thursday, August 25, 2016 9:59 PM
  • So far, the uninstall of KB3172614 has held for the past 96 hours.  A bit too early to still report... but it seems to be holding for now. 
    Thursday, August 25, 2016 11:00 PM
  • Stable for 1 week with no new developing issues.  Removing & blacklisting KB3172614 seems to be the trick.

    Monday, August 29, 2016 12:27 PM
  • Hi,

    Glad to hear the issue has not re-occurred, kindly mark helpful post as answer so that it would be much more efficient for other community members to find useful information.

    Best Regards,

    Amy


    Please remember to mark the replies as an answers if they help and unmark them if they provide no help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, August 30, 2016 8:29 AM
    Moderator
  • We have removed KB3172614 from all our R2 RDSH servers but the problem still persists (7 in total)

    Has anyone found any new information on this as its starting to get to a point where our RDSH servers are not fit for purpose,

    Best regards,

    Adam

    Tuesday, August 30, 2016 9:07 AM
  • Same here.

    Windows Server 2012 R2 Datacenter with role Remote Desktop Services, logging on via console works fine but logging on via RDP times out, fails, and logs event 4005 "The Windows logon process has unexpectedly terminated."

    Disabling Webroot does not resolve the problem.

    Updates KB2621440, KB2667402, KB3172614 are not installed.

    Update KB3161606 is installed.

    We have not tried rebooting the server as we want to investigate while the problem exists while we can.

    Our NOC is looking into this now so if we or they determine the problem then I'll update here.

    Tuesday, August 30, 2016 10:27 AM
  • We are having the same issue again too. Removing KB3172614 did not solve the issue. 
    • Edited by commdudeaf Tuesday, August 30, 2016 6:45 PM
    Tuesday, August 30, 2016 6:30 PM
  • Same here.

    Windows Server 2012 R2 Datacenter with role Remote Desktop Services, logging on via console works fine but logging on via RDP times out, fails, and logs event 4005 "The Windows logon process has unexpectedly terminated."

    Disabling Webroot does not resolve the problem.

    Updates KB2621440, KB2667402, KB3172614 are not installed.

    Update KB3161606 is installed.

    We have not tried rebooting the server as we want to investigate while the problem exists while we can.

    Our NOC is looking into this now so if we or they determine the problem then I'll update here.

    Ben; In the thread here:

    https://community.spiceworks.com/topic/1570976-server-2012-rds-winlogon-process-crashing-event-id-4005?page=3#entry-6151715

    one user says Microsoft recommends removing KB3161606. I had not heard this from Microsoft but will give it a try; I'm curious if you have since tried removing this update as well? 

    Tuesday, August 30, 2016 7:00 PM
  • Hello,

    We are experiencing the same issue on a RDS 2012R2 Farm. End-users will receive a black screen upon connection and the server should be hard reset until the error occurs again. Previous hotfix (RDS2012R2) or other workaround doesn't seem to work.

    Also, Remote Desktop Local Session is showing these Error messages when a user try to reconnect or establish a RDP session.

    1. An error occurred when transitioning from CsrConnected in response to EvCsrInitialized. (ErrorCode 0x80004005)

    Please note :

    This issue was previously happening but in a different way previous August.
    End-users have the same blackscreen but RDS session was stuck in a down state. (ghosted)
    We though this was related to an antivirus software and made change to it.
    Until this issue comeback again earlier in august with the Winlogon 4005 Error. (First occurence)
    I don't know if this is related, just dropping the information in case of somebody expected the RDS Down Session state (can be listed with qwinsta.exe )

    This new thread is also experiencing a similar issue :

    https://social.technet.microsoft.com/Forums/windows/en-US/4d4d6863-3fc0-4281-918b-235ca790a98a/rds-2012-r2-collections-show-black-screen-and-then-close?forum=winserverTS

    I will try to uninstall KB3172614 installed on the 8/6 and post the result.




    Tuesday, August 30, 2016 7:51 PM
  • Microsoft support has asked us to install hotfix kb2887595 

    https://support.microsoft.com/en-us/kb/2887595

    The support tech suggested this because in the article it states:

    "2897632 Logon screen turns black in Windows 8.1 or Windows Server 2012 R2"

    They did no analysis on crash dumps, but suggested installing this hotfix, rebooting, and waiting to see if it comes back. The hotfix is from 2013, and since our session host servers have been running without issues up until late July / early August of 2016; I'm not convinced an update from 2013 is the solution. But we will try and wait and report back findings. 



    • Edited by commdudeaf Tuesday, August 30, 2016 9:45 PM
    Tuesday, August 30, 2016 9:44 PM
  • The kb2887595 is already installed since a long time in here.
    Also with Hotfix : https://support.microsoft.com/en-us/kb/3047296

    This seems more related to winlogon process than to a Remote Desktop problem.
    Login locally on the server is impossible too and will produce another winlogon Error.

    "The winlogon notification subscriber <GPClient> is taking long time to handle the notification event (CreateSession)." We found no related event before the winlogon error.

    What type of RDS deployment do you have ? (Roaming,UPD ?)
    Tuesday, August 30, 2016 10:45 PM
  • Interesting to still hear issues even after removal of those updates.

    Knock on wood - we're still stable after removing ALL of the patches from August and from the July roll-up.  

    Will continue to monitor any changes, hopefully there is a permanent fix on the way for those still affected by the issue.

    Wednesday, August 31, 2016 12:27 AM
  • We began to get this issue on the 9th August after a round of updates were installed on our primary domain controller. The instance the domain controller came back online, it kicked off two of our RDSH with the black screen 4005 issue. We then spent uptime late nights and weekends going through every form of update.

    we reconfigured out NICs, updated our DNS servers as we thought it was that (as DNS was updating after each machine restart which was a red herring), we upgraded the VM's, as DCs were on 2012 so we have upgraded those to 2012 R2, reconfigured our Anti Virus (ESET), we've scoured ADDS, rectified every warning/error message we can on the domain, no matter how unrelated it may seem, including the DSFR service which was another red herring.

    We even uninstalled KB3172614 from all servers,restarted them, NO JOY!.

    It is worth stating that most of our servers host hundreds upon hundreds of active sessions, although 2 of our RDSH have under 30/40 active sessions and these 2 machines are the only ones to not get taken down by multiple 4005 events, though the odd 4005 event does still hit that machine, but the user tries again and it is fine for them.

    We still get a couple of servers randomly hit per day, no rhyme no reason.

    We have now uninstalled KB3161606 from all machines, restarted them all and are playing it by ear, but we aren't hopeful yet.

    Regardless of all of this, we have also had random black screens on attempted logins for months (actually, over a year now) and have never gotten to the bottom of it. usually on the 2nd or 3rd attempt, the login process fires correctly, but it can never be associated with anything such as peak times or such.

    This is the worst MS-based issue we have suffered in many years and our users are losing faith in the platform.

    Wednesday, August 31, 2016 8:31 AM
  • We began to get this issue on the 9th August after a round of updates were installed on our primary domain controller. The instance the domain controller came back online, it kicked off two of our RDSH with the black screen 4005 issue. We then spent uptime late nights and weekends going through every form of update.

    we reconfigured out NICs, updated our DNS servers as we thought it was that (as DNS was updating after each machine restart which was a red herring), we upgraded the VM's, as DCs were on 2012 so we have upgraded those to 2012 R2, reconfigured our Anti Virus (ESET), we've scoured ADDS, rectified every warning/error message we can on the domain, no matter how unrelated it may seem, including the DSFR service which was another red herring.

    We even uninstalled KB3172614 from all servers,restarted them, NO JOY!.

    It is worth stating that most of our servers host hundreds upon hundreds of active sessions, although 2 of our RDSH have under 30/40 active sessions and these 2 machines are the only ones to not get taken down by multiple 4005 events, though the odd 4005 event does still hit that machine, but the user tries again and it is fine for them.

    We still get a couple of servers randomly hit per day, no rhyme no reason.

    We have now uninstalled KB3161606 from all machines, restarted them all and are playing it by ear, but we aren't hopeful yet.

    Regardless of all of this, we have also had random black screens on attempted logins for months (actually, over a year now) and have never gotten to the bottom of it. usually on the 2nd or 3rd attempt, the login process fires correctly, but it can never be associated with anything such as peak times or such.

    This is the worst MS-based issue we have suffered in many years and our users are losing faith in the platform.

    Thanks for the input MyRyanish. We will try removing KB3161606 as well. Please keep us posted if that helps or not. I also recommend opening a case with Microsoft, the more cases they get, the more known the problem will become within, hopefully allowing them to put some more effort in in diagnosing the root cause. 

    We also are having very heavy loss of work due to this issue. 

    I'm curious as to the difference in your RDSH's that serve 30/40 users versus the ones that serve 100 users. What are the resources in place for these (RAM / CPU)? And are they all on a VMware environment? 


    • Edited by commdudeaf Wednesday, August 31, 2016 10:07 AM
    Wednesday, August 31, 2016 10:06 AM
  • The kb2887595 is already installed since a long time in here.
    Also with Hotfix : https://support.microsoft.com/en-us/kb/3047296

    This seems more related to winlogon process than to a Remote Desktop problem.
    Login locally on the server is impossible too and will produce another winlogon Error.

    "The winlogon notification subscriber <GPClient> is taking long time to handle the notification event (CreateSession)." We found no related event before the winlogon error.

    What type of RDS deployment do you have ? (Roaming,UPD ?)
    Nathanael; Thanks for that input, as I imagined the update from 3 years ago would not help a problem stemmed within the last two months. We are using roaming profiles. 
    Wednesday, August 31, 2016 10:09 AM
  • Will do commdudeaf. We'll open a case shortly.

    Our RDSH's run 64GB of RAM and tend to be E5-2430 v2 CPUs. Always on SSD in RAID1 or RAID10.

    DC's, Gateways and additional services are run on VMware VM's with dual CPU / 4GB capacity and I believe the data backbone is RAID50 running SAS drives.

    Wednesday, August 31, 2016 10:18 AM
  • We also are experiencing the same problem on multiple rd host servers on many of our customers. On one of our customers we experienced the problem on 10-08 this month. The only update that was installed before this date was on 06-08 (KB3172614)

    The server was last updated before the problems in may (when there were no problems)

    We are now removing the update KB3172614 on the servers, that are experiencing the 4005 winlogon error. It's strange that not all of the servers are experiencing the problem

    Wednesday, August 31, 2016 10:36 AM
  • I only investigated and left any changes to our NOC. Their update:

    "--Unable to logon the server through lmi and rdp ,lmi is shows hostoffline and rdp received error .
    --Logged onto host "<Hyper-V server>" through lmi and connected the server "<RDS server>"
    --Observed the internet is not working .
    -- We have corrected the preffred DNS point on the member server "<RDS server>" and ran the ipconfig /flushdns & ipconfig /registerdns command.
    -- Now are able to access the web sites and internet working fine.
    --We are unable to took rdp ,it is struck in connecting .
    --We tried to restart the Remote Desktop service ,but is fail in stopping mode.
    --Spoke to Ben on this no: <our phone number> and got the approval for reboot the server .
    --Rebooted the server successfully.
    --Now seerver is up and rdp is working fine .
    --Observed that the all related services are in running state.
    --Logged onto another server "<DC server>" and able connect the server "<RDS server> " via rdp successfully."

    I'm pretty sure that there wasn't a DNS problem and I saw that the RDS service was running before they started working on it but, either way, they didn't uninstall update KB3161606 and rebooting the server seemed to resolve the problem, although I'm sure only temporarily.

    Wednesday, August 31, 2016 12:40 PM
  • Just an update - removing KB3161606 has not fixed the issue and we have just had another server go down.
    Wednesday, August 31, 2016 12:53 PM
  • I only investigated and left any changes to our NOC. Their update:

    "--Unable to logon the server through lmi and rdp ,lmi is shows hostoffline and rdp received error .
    --Logged onto host "<Hyper-V server>" through lmi and connected the server "<RDS server>"
    --Observed the internet is not working .
    -- We have corrected the preffred DNS point on the member server "<RDS server>" and ran the ipconfig /flushdns & ipconfig /registerdns command.
    -- Now are able to access the web sites and internet working fine.
    --We are unable to took rdp ,it is struck in connecting .
    --We tried to restart the Remote Desktop service ,but is fail in stopping mode.
    --Spoke to Ben on this no: <our phone number> and got the approval for reboot the server .
    --Rebooted the server successfully.
    --Now seerver is up and rdp is working fine .
    --Observed that the all related services are in running state.
    --Logged onto another server "<DC server>" and able connect the server "<RDS server> " via rdp successfully."

    I'm pretty sure that there wasn't a DNS problem and I saw that the RDS service was running before they started working on it but, either way, they didn't uninstall update KB3161606 and rebooting the server seemed to resolve the problem, although I'm sure only temporarily.

    Hi Ben; The workaround is to reboot the session host. I am assuming you are using Continuum NOC, we are troubleshooting with them as well outside of Microsoft. Rebooting the server temporarily will resolve the issue, but does not solve the solution permanently. After a server reboot we have seen the RDSH be stable for over a week and then fail again. 
    Wednesday, August 31, 2016 1:18 PM
  • Just an update - removing KB3161606 has not fixed the issue and we have just had another server go down.
    Thanks for the update; We will scratch that one off the list as possible cause as well. 
    Wednesday, August 31, 2016 1:18 PM
  • Just to add to this, we have investigated all our RDSH servers and when Windows Updates were applied, and machines were being effected have a disparity in that some had no updates applied to them from August yet still went down.

    This leads us back to the time when on the morning this first happened, we updated both our gateway servers and both our DC's with a round of Windows updates. We're focusing on these VM's now. I will be uninstalling any updates from August on all VM's and rebooting ASAP as a further test, but we are fast running out of things to try.

    Wednesday, August 31, 2016 2:34 PM
  • We have experienced the same on two servers.  Problem started with the 8/9/16 Windows Updates.  We uninstalled KB3157670, 3177108, 3178034 - seemed to help but problem returned.
    Wednesday, August 31, 2016 8:59 PM
  • Just to add to this, we have investigated all our RDSH servers and when Windows Updates were applied, and machines were being effected have a disparity in that some had no updates applied to them from August yet still went down.

    This leads us back to the time when on the morning this first happened, we updated both our gateway servers and both our DC's with a round of Windows updates. We're focusing on these VM's now. I will be uninstalling any updates from August on all VM's and rebooting ASAP as a further test, but we are fast running out of things to try.

    I was wondering if you already have some more information about it... thanx
    Thursday, September 1, 2016 7:09 AM
  • We have been asked by our NOC to remove KB3177108, KB3167679, and KB890830 and check on results. We will perform this and get back. Also we are still waiting on Microsoft, our case has been open for weeks but they still have not escalated to a team who can properly analyze the problem. Still waiting on response from them, hopefully within the next week or so. 

    If you have a case open with Microsoft, please ask to be escalated to the CPR (Critical Problem Resolution) team. 

    • Edited by commdudeaf Thursday, September 1, 2016 12:22 PM
    Thursday, September 1, 2016 12:09 PM
  • And it's back...

    12 days without any issues and the original problem returns.  Rebooting solved the issue for the time being.  Back to square one.

    *frustrating*

    Friday, September 2, 2016 12:09 AM
  • And it's back...

    12 days without any issues and the original problem returns.  Rebooting solved the issue for the time being.  Back to square one.

    *frustrating*

    Bugga.

    Happening to two sites that we look after. Thought I was onto a solution - One site uses vhd for profiles but the other does not. So I will join the awaiting crowd of followers.

    Thanks to all the suggestions. 

    Friday, September 2, 2016 5:41 AM
  • We have been asked by our NOC to remove KB3177108, KB3167679, and KB890830 and check on results. We will perform this and get back. Also we are still waiting on Microsoft, our case has been open for weeks but they still have not escalated to a team who can properly analyze the problem. Still waiting on response from them, hopefully within the next week or so. 

    If you have a case open with Microsoft, please ask to be escalated to the CPR (Critical Problem Resolution) team. 

    We are using different patch days on different servers/clients. These updates you mentioned are installed two days after we first experienced the problem at one of our customers (we are using gfi patch mgt with an event log check on winlogon 4005)

    So i guess these updates are not the problem..

    The first time we experienced the problem was on 08-08

    The only update installed on 03-08 was KB3172614

    These are the windows updates that are installed in july (20th)

    kb3170455
    kb3172727
    kb3170377
    kb3169704
    kb3173424

    Friday, September 2, 2016 6:03 AM
  • I don't want to make flippant suggestions, but is there any way that an update to the MS RDP client could have caused this by any chance? Hence the reason why removal of updates on the server doesn't appear to change anything.

    The only other thought is, are these updates being removed fully? There is also one update that we cannot remove which is a security patch related to boot up - KB3173426

    Because if not, I'm tempted to look into what services connect with any external MS systems.

    Friday, September 2, 2016 10:35 AM
  • I don't want to make flippant suggestions, but is there any way that an update to the MS RDP client could have caused this by any chance? Hence the reason why removal of updates on the server doesn't appear to change anything.

    The only other thought is, are these updates being removed fully? There is also one update that we cannot remove which is a security patch related to boot up - KB3173426

    Because if not, I'm tempted to look into what services connect with any external MS systems.

    Kb3173426 or kb3173424 are not the problem I guess... these updates were installed on one of the problem servers a lot later then when the first experienced the problem
    Friday, September 2, 2016 11:26 AM
  • All, If you have a server in a failed state please contact Microsoft. They are aware of this issue and need to collect crash dumps in order to analyze and determine a cause. We currently do not have any failed server so we cannot provide them with data they need. We have been rebooting the servers when they fail with the black screen / winlogon error to get them back in production. If anyone can afford to leave one in the failed state, please have Microsoft collect dumps on that server. If you have questions feel free to PM me. 
    Friday, September 2, 2016 1:46 PM
  • We are experiencing the same issues. We have a four server farm, and may be able to leave one in a crashed state.
    Friday, September 2, 2016 2:39 PM
  • What I have notice is that when I use one of my Broker servers i get random black screens on some of my Sessions Hosts, but when I failover to my other broker server I don't receive a black screen during remote login. However I did deal with the issue of task manager being populated with user accounts with no name and the CPU was at 100% when using my other broker. Like the accounts would never log off. An this happened on the Session hosts that I never received a black screen. Makes me feel that I should redo my broker servers.
    Friday, September 2, 2016 3:08 PM
  • Just to add to this, we have investigated all our RDSH servers and when Windows Updates were applied, and machines were being effected have a disparity in that some had no updates applied to them from August yet still went down.

    This leads us back to the time when on the morning this first happened, we updated both our gateway servers and both our DC's with a round of Windows updates. We're focusing on these VM's now. I will be uninstalling any updates from August on all VM's and rebooting ASAP as a further test, but we are fast running out of things to try.

    A member on the Spiceworks thread just stated they are having the same issue on (3) individual RDS servers that are not part of a farm, so that may rule out the RDGateway / broker being a potential cause. 
    • Edited by commdudeaf Friday, September 2, 2016 7:52 PM
    Friday, September 2, 2016 7:52 PM
  • Same here. Broker crashing. I guess this is not relevant to Session Host only but for all 2012R2 OS.

    How can we take a dump of Winlogon process ?
    Friday, September 2, 2016 8:28 PM
  • Alright, tonight I removed KB3172615 from the VM Host running the problematic RDS server.  KB3172614 was removed from the VM two weeks ago with no success.  Both roll-ups from July are similar with improvements and fixes.  The VM host is running DataCentre 2012 - the VM running RDS roles is Server Standard 2012 R2.  

    We'll see how this works over the long weekend.

     

    Saturday, September 3, 2016 12:19 AM
  • It seems quit on my side... anyone has an update what i could be?
    Monday, September 5, 2016 12:44 PM
  • Hello everyone,

    We have an actual server farm with the issue live right now with some good information
    The farm is composed as follow: 5 RDSH, 1 Broker1 Gateway > Patch 06/08)

    The Winlogon Error EventID 4005 is happening for all RD SH servers, and also on the RD Broker Server.The server Broker has encountered the Error AFTER the last reboot, (Pending Update)

    Guessing that a Winlogon crashing at the same moment is indicative to something related to network or resources ? So I''ve searched DC, File or other related server and did not find anything relevant with this timestamp.

    However, these servers has not crashed/freezed at this time and winlogon error seems to resolve by itself. Users still had black/screen RDS issue.

    • The Windows logon process has unexpectedly terminated.
    • The Desktop Window Manager has exited with code (0xd00002fe)

    BUT, this issue showed up on 3/5 SH at the SAME moment: 18:34:53

    (* we have sometimes "A fatal alert was received from the remote endpoint. The TLS protocol defined fatal alert code is 20/40" error, but never along the 4005 event).

    • Can we say this can be a network/connectivity/resource issue since August update?

    I know past connectivity issue impacting RDS Session and RDP(network) stability was encountered and had the same symptom as the Blackscreen / disconnecting issue. Hotfix and sometimes disabling TCP Offloading/RSS was correcting the issue. As explained here and here,

    Can this be related to bad NIC Driver or RSS configuration? And happening on very high latency and on heavyloaded server ? (1 CPU Core Bottelneck?)

    Below, adapter configuration. You can observe than RSS is enabled (as default) but is not active on VMXNET3 NiC which I guess can lead to CPU Bottleneck ?

    This is hair pulling, but maybe give a try to active (if possible) to enable RSS on the NiC ?
    What do you think ?


    netsh int tcp show global
    Querying active state...
    
    TCP Global Parameters
    ----------------------------------------------
    Receive-Side Scaling State : enabled
    Chimney Offload State : disabled
    NetDMA State : disabled
    
    Get-SmbServerNetworkInterface
    
    Scope Name Interface Index RSS Capable
    ---------- --------------- -----------
    * 12 False



    Tuesday, September 6, 2016 7:06 AM
  • Thanks for the update Nathanaël, we're pretty much dealing with the exact same issue as you. I was having a chat with one of my friends at the weekend who is a consultant and straight away, without even looking at the issue, he is adamant that it will be down to some Windows Driver updates, as these do not roll back even after you uninstall the Windows Updates themselves. He thinks the driver is leaking handles or something and causing it to crash, hence why it is really only effecting our more heavily used servers.

    We'll be investing today and coming back to everyone once we have confirmed 

    Tuesday, September 6, 2016 8:19 AM
  • Could that be KB3177725Security update for Windows kernel-mode drivers: August 9, 2016??


    Also in, KB3177108 Security update for Windows authentication methods: August 9, 2016, winlogon.exe is mentioned for Server 2012 R2!
    • Edited by Fr0ns Tuesday, September 6, 2016 1:38 PM
    Tuesday, September 6, 2016 1:35 PM
  • Same issue here... since the updates from August our RDSH Farm isn't stable anymore and the 7011 and the winlogon 4005 error appears. I hope MS get's this fixed soon!
    Wednesday, September 7, 2016 8:35 AM
  • We have remote FX configured via group policy also. 

    Does anyone have classic shell installed? 

    Wednesday, September 7, 2016 12:07 PM
  • We have some RDS servers in the cluster-and-gateway environment in our hosting environment go crazy over this, but also some standalone RDS servers on hyper-v hosts at customers.

    Strange part is that we, as opposed to some people above, are able to logon locally but not via RDS

    The system event log always starts to spit out Schannel event 36888 error 1203 before the winlogon process starts to malfunction, so maybe this is more related to TLS functions and maybe the hyper-v NIC driver then to RDS client or host


    Wednesday, September 7, 2016 12:43 PM
  • We have some RDS servers in the cluster-and-gateway environment in our hosting environment go crazy over this, but also some standalone RDS servers on hyper-v hosts at customers.

    Strange part is that we, as opposed to some people stated above, are able to logon locally but not via RDS

    The system event log always starts to spit out Schannel event 36888 error 1203 before the winlogon process starts to malfunction, so maybe this is more related to TLS functions and maybe the hyper-v NIC driver then to RDS client or host

    I think we can rule out Hyper V NIC drivers, we are running the environments on VMware. Also our issue is the same, cannot RDP in but CAN console in. 
    Wednesday, September 7, 2016 12:45 PM
  • Thanks for the input, that rules out quite some options.

    Are you also experiencing the schannel errors ?


    Wednesday, September 7, 2016 12:53 PM
  • I did find schannel errors but they do not look to correlate with the winlogon errors. 

    The next member who experiences this issue; instead of rebooting the faulty server, please instead try to restart the Remote Desktop Services service and see if that (temporarily) resolved the problem. Please let us know your results. 


    • Edited by commdudeaf Wednesday, September 7, 2016 1:58 PM
    Wednesday, September 7, 2016 1:57 PM
  • Tested previously, restarting Remote Desktop Services, will always result in a error "could not start in a timely .."

    Someone has another problem with other roles or server ? ( Maybe we have an issue with a Failover Clustering too)
    Wednesday, September 7, 2016 2:04 PM
  • I'm having the same problem on a remote desktop server. This started yesterday and I installed the last updates a couple of weeks ago.

    I have restarted every single service, but no luck. Indeed when you try to restart the remote desktop services it wil result in: "could not start in a timely .."

    Only "solution" so far is to reboot the server. Hoping for a solution.

    Wednesday, September 7, 2016 3:17 PM
  • I opened a ticket with Microsoft support on Sunday, and had three of their team members on the phone pulling data and watching the error happen live from our server that went down Thursday. Today I got an e-mail with 4 hotfixes to try.

    3132080 <- Sounded promising, but it is for a slow/hanging logon due to a home directory re-point to a DFS share, and it install told me it was not applicable to my system, probably because it was succeeded by another update as I couldn't find it installed on my system. Release date Feb 2016

    2927267 <- Hotfix for not logging in after password change. Last updated April 2014

    3049843 <- Hotfix for DPAPI data after password change. Last updated July 2015

    2919355 <- cumulative update already installed on the system.

    So basically, they sent me nothing :\

    We have not experienced this issue on other servers the IT team RDP's into regularly, only our "application" term servers in the farm, and these guys do get hit hard regularly. I'm starting to lead towards a teetering-point situation, that after so many logons WINLOGON fails. Right now I'm going to power cycle our downed server to get back into production and wait for the next fail and pull the local Terminal Service session manager logs and and do a count of how many logon, logouts, and disconnects.

    Wednesday, September 7, 2016 3:41 PM

  • Only "solution" so far is to reboot the server. Hoping for a solution.

    I am also experiencing the black screen on multiple servers (event 4005 Winlogon failed). These are Domain controllers and SQL servers.

    I can access through console and the fix I documented for our team so that they dont have to reboot is to attempt to manually restart the service and if it does not work:

    Run CMD as admin>taskkill /PID xxxx /F

     

    In order to figure out the process ID (PID) -> open task manager and find the TermService Service  and identify the PID

    Then restart the remote desktop service. Hope this helps to avoid rebooting productive servers.

    Still need a good solution :)

    Wednesday, September 7, 2016 4:07 PM
  • Additionally, when the Winlogon error is logged I can see 3 corresponding errors under the logs for: Microsoft->Windows->RemoteDesktopServices-RDPCoreTS 

    RDP_UDP: An error was encountered when transitioning from UdpStateProcessTransportHandshake in response to UdpEventErrorOnMtReqComplete (error code 0x800705B4).

    RDP_UDPLOSSY: An error was encountered when transitioning from UdpLossyStateProcessTransportHandshake in response to UdpEventErrorOnMtReqComplete (error code 0x800705B4).

    The RDP protocol component X.224 detected an error (0) in the protocol stream and the client was disconnected.

    The time stamp matches exactly with the Winlogon error. Not sure yet what these errors are indicating or if others are experiencing the same behavior.

    Wednesday, September 7, 2016 6:03 PM
  • All,

    We are having the same problem here, here being a College of Business at a research University. Our environment:

    RDS Session Host Farm - nine 2012R2 hosts, 32GB RAM, 8-vcpu; fully patched

    Host Cluster - five 2012 R2 Hyper-V hosts on HP G8 blades; fully patched

    Connection Broker - one 2012R2, local database

    Kemp VLM provides initial referral and gateway services

    Problem first appeared 8/31/2016, as returning users began ramping up usage. Typical load is 4-15 connections per session host. We use roaming profiles on a DFS Namespace share (no replication at this point), and folder redirection to the student's personal drives.

    So far we are remediating via the ever-popular host reboot (with a second reboot JIC). The first restart is a shutdown via Hyper-V Manager, although I can log in via a console session initiated from H-V Mangler. We have not, as yet, opened a support incident with  MS.

    Thanks,

    LtheM


    • Edited by Loel Wooldridge Wednesday, September 7, 2016 9:39 PM clarification of load parameter
    Wednesday, September 7, 2016 9:10 PM
  • Update:  After removing the July roll-ups (KB317214 & KB317215) from ALL 2012 servers attached to the domain, all appears to be well.  

    I'm not saying this is the solution - but responsiveness on all servers has improved since being removed back on September 2nd.   Ironically, an older (non production) 2008 server did not have any issues running Terminal Services when things went wonky in early August.

    Again, too early to tell... so I'm holding steady and keeping my fingers crossed.  

    Thursday, September 8, 2016 12:14 AM
  • Hi all,

    same problem for a few weeks its an absolute nightmate

    we are going to use procdump to monitor the winlogon and create a crashdump for microsoft hopefully the next time it happens which is getting more frequent

    they seem to think removing KB3172714 fixes - but it hasnt for us

    on rds farm hyper-v vm's with 3 session hosts, happens randomly on any server and only solution is to get current users to log off to reboot server

    Thursday, September 8, 2016 6:10 AM
  • KB3172614... must be the problem

    Look at rdpcore.dll and rdpcorets.dll. With this update installed the version is 6.3.9600.18377

    Without the broken update the version is 6.3.9600.18255.  I rolled back the update on some servers, and all servers rdpcore.dll is rolled back to the older version

    Thursday, September 8, 2016 2:18 PM
  • KB3172614... must be the problem

    Look at rdpcore.dll and rdpcorets.dll. With this update installed the version is 6.3.9600.18377

    Without the broken update the version is 6.3.9600.18255.  I rolled back the update on some servers, and all servers rdpcore.dll is rolled back to the older version

    Has the black screen re appeared for you after removing KB3172614?

    Thanks

    Thursday, September 8, 2016 6:10 PM
  • I removed KB3172614, KB3177725 and KB3177108 and just got another server experiencing the black screens and 4005 winlogon errors 15 minutes ago. I was hoping this finally resolved our issue. Still looking for a fix.
    Thursday, September 8, 2016 9:28 PM
  • John,

    I posted the following on Spiceworks do you see the same error on your servers?


    Under logs for Server Roles->Remote Desktop services

    I see an error showing  at the time of the issue:

    An error occurred when transitioning from CsrConnected in response to EvCsrInitialized. (ErrorCode 0x80004005)

    EventID: 36

    Source: TerminalServices-LocalSessionManager

    Can someone that is experiencing the black screen issue please check the logs for  Server Roles->Remote Desktop services to confirm if they also see the same?

    There is a hot fix for that error released sometime on 2015.  Have not tried it so far.

    https://support.microsoft.com/en-us/kb/3047296

    Thanks


    • Edited by The_Doctor46 Thursday, September 8, 2016 11:26 PM mistake
    Thursday, September 8, 2016 11:25 PM
  • I have to say no for now.... so i'm hoping it will not reappeer

    But one of my friends, is working in a big company with the update kb3172614 installed on 700+ servers and are experiencing none of the problemes.

    Friday, September 9, 2016 6:22 AM
  • We have remote FX configured via group policy also. 

    Does anyone have classic shell installed? 

    We have indeed installed the classic shell on all the servers... anyone else?
    Friday, September 9, 2016 7:17 AM
  • We have classic shell installed also, its been on for some time though
    Friday, September 9, 2016 3:40 PM
  • Crash and burn - Event 4005 appeared late afternoon from a 5 day hiatus.  To confirm, removing KB317614 and KB317615 from all 2012 Servers made no difference.

    Running out of idea's... and becoming increasingly frustrated with this.

    Friday, September 9, 2016 9:38 PM
  • Crash and burn - Event 4005 appeared late afternoon from a 5 day hiatus.  To confirm, removing KB317614 and KB317615 from all 2012 Servers made no difference.

    Running out of idea's... and becoming increasingly frustrated with this.

    Do you happen to have the classic shell installed?

    Saturday, September 10, 2016 1:32 PM
  • Classic shell is not installed. 
    Saturday, September 10, 2016 7:55 PM
  • Crash and burn - Event 4005 appeared late afternoon from a 5 day hiatus.  To confirm, removing KB317614 and KB317615 from all 2012 Servers made no difference.

    Running out of idea's... and becoming increasingly frustrated with this.

    It is very frustrating. We removed KB317614 and it has not re-appeared yet.   

    On another thread describing the black screen issue someone apparently got good results by disabling Large Send Offload on the server's network interface. I have not been able to confirm if the issue that was affecting that user was in fact the same. 

    We cant replicate the issue at will but if you have an affected server, could  you try this suggestion and report back

    https://community.spiceworks.com/topic/693490-black-screen-at-remote-desktop-login-to-server-2012-r2?page=2#entry-6192997

    Thanks

    Tuesday, September 13, 2016 8:10 PM
  •  Having the same issue with 2 RDS servers.  Very random.  Was happening about every 2-3 days started august 8th as well.  Have done all the above.  Here is the kicker.  Restored back to Aug 7th backup 2 weeks ago. Blocked all patching on the RDS server.  Was fine for 8 days then we started to get the 4005 errors again.  Really mind blowing.
    Wednesday, September 14, 2016 2:47 AM
  • Hello everyone, this is Sasha Loncarevic from Microsoft Support.  I am replying to this top thread hoping it is most visible.

    We believe there is a problem in a code change we made to a file that is in both the July and August rollups - investigations are still under way.  The symptoms are that the Remote Desktop Server service gets into a deadlock during virtual channel management and all new connections go no further than a black screen.  Since the RDS service is deadlocked it cannot be restart or stopped.  You can however kill the relevant SVCHOST.EXE process with Task Manager and manually start the RDS service to recover.  Existing users can reconnect and new users can logon. 

    To find the RDS svchost, use Task Manager, Services tab to look for TermService.  Right-click on it and choose Go To Details.  This will highlight the svchost.exe in the Details tab.  Right-click on that and choose Create Dump File (this is for confirmation and further investigation should you wish to have us review it), and then right-click again and choose End Task.

    As above, we are investigating further, but unfortunately with complex issues like this there is no ETA.  The workarounds are to uninstall July and August rollups, or manually End Task and restart the RDS service as above.

    If you would like us to confirm that you are seeing this specific issue, then please raise a Service Request with Microsoft Support; providing us the dump file (when the issue is occurring) captured with the steps above.

    Regards,

    Sasha Loncarevic

    Wednesday, September 14, 2016 2:30 PM
  • Hello everyone, this is Sasha Loncarevic from Microsoft Support.  I am replying to this top thread hoping it is most visible.

    We believe there is a problem in a code change we made to a file that is in both the July and August rollups - investigations are still under way.  The symptoms are that the Remote Desktop Server service gets into a deadlock during virtual channel management and all new connections go no further than a black screen.  Since the RDS service is deadlocked it cannot be restart or stopped.  You can however kill the relevant SVCHOST.EXE process with Task Manager and manually start the RDS service to recover.  Existing users can reconnect and new users can logon. 

    To find the RDS svchost, use Task Manager, Services tab to look for TermService.  Right-click on it and choose Go To Details.  This will highlight the svchost.exe in the Details tab.  Right-click on that and choose Create Dump File (this is for confirmation and further investigation should you wish to have us review it), and then right-click again and choose End Task.

    As above, we are investigating further, but unfortunately with complex issues like this there is no ETA.  The workarounds are to uninstall July and August rollups, or manually End Task and restart the RDS service as above.

    If you would like us to confirm that you are seeing this specific issue, then please raise a Service Request with Microsoft Support; providing us the dump file (when the issue is occurring) captured with the steps above.

    Regards,

    Sasha Loncarevic

    Thanks for the update Sasha, Your input to this thread is greatly appreciated. Please keep us all updated; especially for the many support teams who are finding these threads on Google but do not have support contract with Microsoft and are relying on the community for resolution. 
    Wednesday, September 14, 2016 2:36 PM
  • Sasha; Would you mind updating this post as well:

    https://community.spiceworks.com/topic/1570976-server-2012-rds-winlogon-process-crashing-event-id-4005-black-screen?page=4&source=navbar-community-notifications

    This also is a very lengthy post for same issue with many followers. Thank you. 

    Wednesday, September 14, 2016 4:25 PM
  • Hello again,

    We are planning to update the July and August rollup KBs to list this problem as a known issue.  I believe some people report the issue persists after removing both rollups.  If this applies to you can you please check that the date on the c:\windows\system32\rdpcorets.dll file pre-dates July 2016?  If the file is dated July 2016 or later then you haven't rolled back all updates that contain the file - which at present should only be the July (KB3172614) and August (KB3179574) rollups.  At the moment we are reasonably sure that the issue is within rdpcorets.dll.

    Thanks,

    Sasha Loncarevic (MSFT)

    Wednesday, September 14, 2016 6:16 PM
  • Hello again,

    We are planning to update the July and August rollup KBs to list this problem as a known issue.  I believe some people report the issue persists after removing both rollups.  If this applies to you can you please check that the date on the c:\windows\system32\rdpcorets.dll file pre-dates July 2016?  If the file is dated July 2016 or later then you haven't rolled back all updates that contain the file - which at present should only be the July (KB3172614) and August (KB3179574) rollups.  At the moment we are reasonably sure that the issue is within rdpcorets.dll.

    Thanks,

    Sasha Loncarevic (MSFT)

    Hi Sasha,

    Can you please post the file versions affected? The "date modified" field under the "details" tab seems to get updated with the date in which the dll was replaced. 

    I rolled back KB3172614 which then rolled back the dll from version 6.3.9600.18377 to 6.3.9600.18255 but the "modified date" matches with the date of the rollback.

    Thanks

    Wednesday, September 14, 2016 6:57 PM
  • Hello,

    The created date should show the actual compile date.  RdpCoreTS.dll 6.3.9600.18377 has a created date of 10-Jun-2016, anything of this version or later has the problem (according to the results of our current investigation). 

    Version 6.3.9600.18255 is from KB3146978, and so this should no longer cause the issue.

    We are also nearing completion of a fix - will post as soon as I have more details.

    Thank you,

    Sasha Loncarevic

    Wednesday, September 14, 2016 8:33 PM
  • This is excellent news... thanks for the update, Sasha!

    Looking forward to the end result (I'm sure we're all feeling the same way!)

    Thursday, September 15, 2016 12:03 AM
  • Hello all,

    We are now testing a fix to the problem.  Some customers have already engaged Microsoft product support via a Service Request and they will be offered the private fix for testing.  I am checking into our policy of allowing the fix to be tested by customers without Enterprise Agreements in place with Microsoft (i.e. contractual disclaimers and other legal terms already in place); in the meantime I will update with any test results that we get in.  Given the nature of the deadlock we will be looking at a minimum of 2 to 3 weeks of testing before we can move to a public release.  Hopefully the workarounds already discussed will keep you going until that time.

    Regards,

    Sasha

    Thursday, September 15, 2016 12:37 PM
  • Thanks for the update Sasha.

    To confirm, we have a work around based on Sasha's advice.

    When the server begins to black screen, we login to another server on the network:

    1. VIEW the processes by process name and get PID's: Tasklist /S serverX /u username /FI "IMAGENAME eq svchost.exe"
    2. The PID with usually be the largest memory footprint will be the TermService service
    3. KILL the process by PID: taskkill /S serverX /u username /PID ****
    4. sc \\serverX start termService

    This does require you to split up the SVCHost via running:

    sc config TermService type= own

    To isolate the service and to ensure you don't kill anything else.

    We'll be automating this via a PS script over the next day, as we currently monitor our services via PRTG and this can set scripts off based on the criteria of the 4005 error.

    Thanks


    • Edited by MrRyanish Thursday, September 15, 2016 2:11 PM
    Thursday, September 15, 2016 2:11 PM
  • Hi Sasha,

    Any news on the fix? This is causing major problems for us still.

    Thanks

    Monday, September 19, 2016 7:37 AM
  • We are also facing this issue.  Date modified on rdpcorets.dll is 07/03/2016.  Version is 6.3.9600.18402.

    Would replacing this .dll with a previous version from a known-good RDS server resolve this?

    Monday, September 19, 2016 7:04 PM
  • Hello,

    Just chiming in to say that the current workaround of ending the relevant svchost task followed by restarting the Remote Desktop Service appears to be working, thank you very much for this.

    We would however also be interested in testing the private fix beforehand if at all possible, is there any way we could get to try it, and if so how would be go about to do it?


    Tuesday, September 20, 2016 8:10 AM
  • It may be easier to script the workaround by just directly doing the kill.

    You still have to configure the service to run in its own process:

    sc config TermService type= own

    But then you can just kill the PID of the service as in:

    taskkill /S systemname /FI "SERVICES eq TermService"/F

    Then either wait for the Services manager to restart it automatically or do:

    sc start TermService

    You can even run it on the local system having the issue (in which case you can omit the "/S systemname") and configure the Task Scheduler to run the script when it sees a 4005 error from Winlogon.  This is the best workaround until they fix it.

    • Edited by The Great Quux Wednesday, October 12, 2016 5:33 PM add /F on taskkill
    Tuesday, September 20, 2016 12:27 PM
  • Any further updates?  Has anyone received a fix?
    Wednesday, September 21, 2016 6:37 PM
  • Just found this thread and we are in the same situation.

    We have a Farm of 4 Session Hosts and 2 RDP Servers not in the farm.  Black Logon screens happen to all of them.  We are getting same Black Screen Logins so I'm pretty sure it's not Gateway or Host.  We opened a ticket with Microsoft.  They told us to remove KB3172614.  It didn't work. We got the black screens on one server the day after.  

    We use Terminal services Manager from Lizard Systems  lizardsystems.com.  It let's us kill processes remotely.  We have found that if we kill all svchost.exe on the server, we can then login.  However that also disconnects all users but let's them continue where they were once connected back.  It's better then rebooting the service.  

    Still waiting on Microsoft to hear of anything else to try.  Please update this thread if anyone has a fix that runs for awhile.  We will do the same.

    Thursday, September 22, 2016 7:23 PM
  • hsheldon, can you confirm that killing all svchost.exe processes leaves your system in a stable state?  Nothing to restart?  All prior users can log in with no problem after getting disconnected?
    Friday, September 23, 2016 3:39 AM
  • Anyone notice the Reliability Roll-up that came in this week?  I wonder if we should be seeing any improvement?

    https://support.microsoft.com/en-us/kb/3185279

    Friday, September 23, 2016 9:44 AM
  • I would not advise you to kill all svchosts as that will leave your system in a unstable condition.  We have managed to shorten the process down to the following powershell snippet.  This will disconnect all users but not log them out and saves rebooting the server

    Invoke-Command -ComputerName SERVERNAME -ScriptBlock {
        $ID = (get-wmiobject win32_service | where { $_.name -eq 'TermService'}).processID
        kill -Id $ID -Force
        Start-sleep 3
        Start-Service TermService

    }

    Friday, September 23, 2016 10:05 AM
  • Hello,

    I see some people installed https://support.microsoft.com/en-us/kb/3047296 and it did not help them, can anyone confirm if they installed the update (following the prerequesites) and it resolved their issue?

    Also this thread might be of help: https://community.spiceworks.com/topic/558846-server-2012-r2-rds-collection-black-screen-upon-connect

    It is quite old, but the problem seems identical and was caused by hardware.

    Friday, September 23, 2016 2:15 PM
  • Izgler,  

    I can confirm.  Server has been working for 48 hours since we killed all the svchost.exe.  All users were disconnected but connected back into their sessions fine and others also connected into server.

    Hope that helps.

    Friday, September 23, 2016 2:52 PM
  • We experience the same problem at our hosts.

    Is there already a fix?

    Monday, September 26, 2016 12:02 PM
  • Hello everyone,

    I had the same issue last week in one of our servers, I installed kb2887595 and removed KB3172614 and KB3179574.

    the file: RdpCoreTS.dll  was 6.3.9600.18402 and now (after removing the updates) it's 6.3.9600.18255 and now all is working fine since a week. hope it is the permanent fix and we are waiting the final fix from Microsoft.

    Thanks


    Monday, September 26, 2016 6:13 PM
  • We have this happening in our RDS Farm with Server 2012, but also have it happening to Windows 7 machines for some of our remote users.  They remote in with RDP and have the same exact problems as our Server 2012 RDS farm.  Anyone else experience this with Windows 7?
    Tuesday, September 27, 2016 12:28 PM
  • Hello,

    for me only Server 2012

    Thanks

    Tuesday, September 27, 2016 12:53 PM
  • I didn't see anything in this rollup that would fix this particular issue unfortunately. We are seeing the same black screen issue with the event ID 36 that everyone else in this thread is. We removed the August and July rollups based on information we got from Microsoft and the system seemed fine for a couple of weeks. However, we just had the problem on two of our session hosts today. I've looked at every thread I can find and have tried everything I've found and still don't have a resolution. We recently migrated from Citrix to RDS since we had heard so many good things about it. However, I'm starting to regret not upgrading our Citrix environment.
    Tuesday, September 27, 2016 1:15 PM
  • According to Premier Support, KB3172614 is the issue: https://support.microsoft.com/en-au/kb/3172614

    No fix yet.

    Wednesday, September 28, 2016 4:32 AM
  • did you install the kb2887595 update?
    Wednesday, September 28, 2016 1:44 PM
  • Any news regarding a fix?
    Friday, September 30, 2016 11:14 AM
  • We removed KB3172614 over 2 weeks ago and this problem persists.

    Per your post, I removed KB3179574 this morning from an affected server, and did not install kb2887595.  After rebooting, I'm able to log in, which is expected.  Will monitor this server over the coming week and follow up.

    It's worth noting that RDPCoreTS.dll is now on version 6.3.9600.18255

    Friday, September 30, 2016 3:02 PM
  • Any updates on a fix?
    Friday, September 30, 2016 6:37 PM
  • Approaching 96 hours since I removed KB3179574, in addition to KB3172614 a few weeks ago - we're still good!  Not sure of the official fix, but this is working great so far in our environment.
    Tuesday, October 4, 2016 5:28 PM
  • Just throwing my hat in the ring, we have been having the blank screen issue for a couple of weeks. Sometimes the server is fine for a week and sometimes I have to reboot numerous times a day. I have removed KB3179574 and KB3172614 just now and am hoping to have my users off my back. PLEASE PLEASE PLEASE MS FIX THIS ISSUE!!!
    Wednesday, October 5, 2016 5:58 PM
  • We have also discovered this issue with some of our RDS servers.

    I've followed @mahyazid's advice and removed KB3179574 and KB3172614 (which wasn't installed actually, so reports of this being the issue are wrong)

    The server is pending reboot, so will check tomorrow whether the RdpCoreTS.dll is then 6.3.9600.18255 (it is currently 6.3.9600.18402) 

    Agreed with @JSUWA the update rollup article kb2887595 doesnt mention any change to RdpCoreTS.dll but I've installed this anyway.



    • Edited by AJB_NZ Thursday, October 6, 2016 4:04 AM error
    Thursday, October 6, 2016 4:02 AM

  • For us, setting the idle session limit and setting "End the session" when limit is reached or broken, seems to stabilise RDP and no black screen of death. If i don't have these settings enabled, i would have to restart daily due to black screen logins.


    Thursday, October 6, 2016 7:06 AM
  • Having the same problem, a fix is needed ASAP.

    Friday, October 7, 2016 1:06 PM
  • Can you try a few things we did. We only had one terminal environment out of 200+ that suffered for months from this. Go to the Local Area Connection, go to the Advanced properties and set the Jumbo Packet property to Disabled, preferably while the server is down if you're able to console into it. See if that fixes your issue.

    Otherwise, we installed dozens of updates from in this thread and recent ones from August to see if they would fix it. The last two I did were KB3177108 and KB3178034. Please let me know!

    Friday, October 7, 2016 6:21 PM
  • Any news on the fix??
    Wednesday, October 12, 2016 1:52 PM
  • Bump again, any news on the fix?  I have been suffering this non-stop... does anyone have links for opening a ticket with Microsoft directly?
    Wednesday, October 12, 2016 7:34 PM
  • *Looks at not recently patched server which is running stable as can be, hmmm.... updating is good, closes security holes etc, maybe I should run the updates.  Looks at other clients who we have updated, GPO behaviour change and broken RDS servers... updating not so good?!*

    We have some clients who's machines aren't as up to date as they should be and thank god!  Our clients who we routinely patch were hit with this bug and earlier the change to GPO which removed drive maps and individual lock downs.   

    Maybe I just won't update anything anymore, the updates in the last 6 months to a year have done more damage than good.  And it's not just their O/S team, Office 365's hosted Exchange has had way more issues than our on prem systems, autodiscover not working properly for iOS / Android took weeks to fix, recurring admin side issues in their portal, email delivery delays, unable to search within Outlook all within the last 6 months.  

    *end rant*

    Can we please get an update as to where the fix stands, we have 3 clients who are experiencing this all of which rely solely on an RDS setup for their business needs.



    • Edited by jpaterson842 Thursday, October 13, 2016 12:51 PM
    Thursday, October 13, 2016 12:32 PM
  • We also made a ticket to this and the response from the MS Expert was: "uninstall update KB3179574 and KB3172614" ... "There's a fix for this Problem in this or next month coming via Windows Update".. Well, the updates for octobre are out, so i guess the fix is coming next month.

    We didn't had any problems so far, but it's just been 2 days now and as you guys know - the error is occuring every now and then..

    Thursday, October 13, 2016 1:41 PM
  • I am having the same issue with multiple clients as well.  All Azure VMs.  Seems to be happening only with my clients that have A series VMs.  DS and DS2 VMs are not having this issue. 
    Thursday, October 13, 2016 10:41 PM
  • We have also opened a ticket with MS and after all they admitted they know about this issue and working on a fix.

    They also recommend uninstalling KB3172614 and KB3179574.

    The fix should be released in November update. So let's hope...

    Friday, October 14, 2016 11:22 AM
  • Same issue here.

    We load a backup from 1.5.2016 but this don't fix it. Rds session host runs in hyper-v.

    any ideas?



    • Edited by hans-8547 Friday, October 14, 2016 5:56 PM
    Friday, October 14, 2016 3:07 PM
  • All,

    The "October preview of Monthly Rollup", KB3192404, does not seem to include the fix for this issue.

    Sasha, could we get an update on the testing and a release-to-public date, please?

    Loel

    Tuesday, October 18, 2016 5:42 PM
  • Did you just remove these from the RDS hosts or did you need to remove it from Gateways as well?
    Wednesday, October 19, 2016 3:10 PM
  • I even uninstalled the two updates on our DC and every other Server.

    We had some issues with our domain controller recieving black screens too, so it's not just a Terminal Server problem.

    But so far we didn't had any issues after uninstalling KB3172614 and KB3179574. ( 1 week now )


    • Edited by Mweiss86 Thursday, October 20, 2016 6:36 AM
    Thursday, October 20, 2016 6:36 AM
  • We are having the same issue on an 8-server 2012R2 RDS farm that began at the end of September right after we installed KB3172614.

    I uninstalled it on all servers last week, and the issue just re-appeared today. 5 of the 8 servers in the farm have now had this issue, and it is completely tanking user/management support for this product. I didn't know about KB3179574; we just opened a Premier case today, so will see if they ask for that one as well.

    Terrible, terrible bug that they should be burning the midnight oil to fix.

    *Oddly enough, we have a virtually identical deployment in the EU with the same updates installed at the same time and where this is not happening at all...
    Thursday, October 20, 2016 5:43 PM
  • @Soundslikedelicious: Do you use `Token Redirection` from a dedicated router/haproxy for RDS farm routing?

    I've noticed that when we have troubles with client connections to the border router/haproxy the rate of 4005 errors increases until the session host "crashes" and refuses to show logon UI on new connections (that is until TermService process is terminated and restarted).

    When ext network is stable (i.e. RDC clients don't reconnect or hung the TCP conn) there are no 4005 logged.

    cheers,
    </wqw>


    Monday, October 24, 2016 8:10 AM
  • Same issue here with a five-node 2012 R2 RDS farm. 

    Removing the July and August hotfix rollups did resolve our problem. 

    Somehow KB3179574 slipped back onto one of our five session hosts, despite blacklisting the thing in SCCM, and we experienced the lockup/freezing issues again. 

    So yes, nuke KB3172614 and KB3179574, blacklist it if you use WSUS, SCCM, or some other patch management process, and hope that version of the dll doesn't show up in any of the September or October rollups/Quality Updates...

    If you're still having the issue, double, triple, and quadruple check the version of rdpcorets.dll.

    Good: 6.3.9600.18255
    Bad, bad, kill it with fire!: 6.3.9600.18402
    Tuesday, October 25, 2016 8:54 PM
  • I created a case with Microsoft about this today, they immediately recognized the problem and suggested the following:

    Please remove July and August update from the server and restart.

    KB3172614 July

    KB3179574 August

    Action Plan:

    Make sure to take backup of registry before making any changes

    1.Create a  registry value  at this location

       HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services

      "SelectTransport"=dword:00000001

    2.Disable UDP on RDS environment using below group policy.

       Computer -> Administrative Templates -> Windows Components ->Remote Desktop Services-> Remote Desktop Session  Host->Connections -> Select RDP Transport protocols

       Set it to enabled - use TCP only.

    3.Configure below policies on server :

      Computer Configuration\Policies\Administrative Templates\Windows Components\Remote   Desktop Services\Remote Desktop Session Host\Session Time Limits

      Set time limit for disconnected sessions - enabled - set it to 5 mins

      Set time limit logoff of RemoteApp sessions - enabled - immediately

      Terminate session when time limits are reached - enabled

    4.Restart the server and monitor.

    I have not implemented the fix yet and do not know for certain if it will work. I'll be executing the above steps tonight and will update the group in the morning.

    Wednesday, October 26, 2016 9:13 PM
  • Interesting!  Keep us all posted.  
    Wednesday, October 26, 2016 11:49 PM
  • @ Kyle

    On the DCs or RDS Hosts?

    Thursday, October 27, 2016 11:48 AM
  • We are having the same issue with our 100+ RDS server farm. When the issue occurs on one of our RDS servers, we have monitoring software listening for the 4005 event. We will then set the server to not allow new connections until reboot. However, the users who have a disconnected session on this host, will still connect to this server and receive a blank screen and we must force logoff the user with the chance of them losing some work.

    We have automated the entire deployment process from the very build of the image, so the servers are always up-to-date and just careless killing processes or uninstalling updates is not an option. We are looking forward to a solution!

    Thursday, October 27, 2016 12:02 PM
  • We are having the same issue with our 100+ RDS server farm. When the issue occurs on one of our RDS servers, we have monitoring software listening for the 4005 event. We will then set the server to not allow new connections until reboot. However, the users who have a disconnected session on this host, will still connect to this server and receive a blank screen and we must force logoff the user with the chance of them losing some work.

    We have automated the entire deployment process from the very build of the image, so the servers are always up-to-date and just careless killing processes or uninstalling updates is not an option. We are looking forward to a solution!

    Hi Thomas,

    I am curious as to the density on each of your RDS session hosts / how many users on average per RDSH? We have seen this issue occur more often on RDSH servers that have a higher volume of users. Thanks! 

    Friday, October 28, 2016 1:01 PM
  • We are seeing the same, our deployments that only have a few concurrent users (<5) have much less of an issue than the ones which have 10-30+ active sessions.  
    Friday, October 28, 2016 2:02 PM
  • Hi,

    We have same problem on Windows 8.1 Pro. When connecting from windows client We have black screen, from linux via freerdp We see login screen but mouse click don't work.  

    Friday, October 28, 2016 3:57 PM
  • I made the changes i posted on all of the Session Hosts.... 2 days and no outages.
    Friday, October 28, 2016 6:20 PM
  • We are having the same issue with our 100+ RDS server farm. When the issue occurs on one of our RDS servers, we have monitoring software listening for the 4005 event. We will then set the server to not allow new connections until reboot. However, the users who have a disconnected session on this host, will still connect to this server and receive a blank screen and we must force logoff the user with the chance of them losing some work.

    We have automated the entire deployment process from the very build of the image, so the servers are always up-to-date and just careless killing processes or uninstalling updates is not an option. We are looking forward to a solution!

    Hi Thomas,

    I am curious as to the density on each of your RDS session hosts / how many users on average per RDSH? We have seen this issue occur more often on RDSH servers that have a higher volume of users. Thanks! 

    Hello,

    We have around 6-8 users on each session host. This very morning another session host just crashed, but it only had one user on it at that time. Everyone else who attempted to connect would get a black screen.

    Out of the 100's of session hosts that we have, split between 6 collections, it only occurs 2-4 times a week on a random session host each time. We then drain the session host and log off disconnected users. This will tell the deployment not to allow new connections to the sick session host, until we reboot it.

    Kind regards,
    Thomas Schmidt


    Monday, October 31, 2016 7:14 AM
  • Hi

    How is there no update that resolves this yet?

    To see an October rollup that does not address this issue causes much frustration.

    Has anyone tried simply copying over the rdpcorets.dll from a non-affected server?

    Isaac

    Tuesday, November 1, 2016 2:05 AM
  • Hello all,

    We are now testing a fix to the problem.  Some customers have already engaged Microsoft product support via a Service Request and they will be offered the private fix for testing.  I am checking into our policy of allowing the fix to be tested by customers without Enterprise Agreements in place with Microsoft (i.e. contractual disclaimers and other legal terms already in place); in the meantime I will update with any test results that we get in.  Given the nature of the deadlock we will be looking at a minimum of 2 to 3 weeks of testing before we can move to a public release.  Hopefully the workarounds already discussed will keep you going until that time.

    Regards,

    Sasha

    It's now 7 weeks ago, that a Microsoft representative gave an estimate of 2-3 weeks for a fix. How far have you've come at this point?
    Wednesday, November 2, 2016 9:36 AM
  • I haven't heard anything either.   I have had success with removing the two listed updates.... but the problem remains, I don't feel confident performing windows updates of necessary importance on terminal services until this is resolved... this should be fixed with a higher priority.

    *sad face*

    Wednesday, November 2, 2016 8:12 PM
  • So we're starting to see the same issue on a 2012 r2 rds server, 4005 logon process terminated.

    Looking for the two updates mentioned in one of the above posts I'm finding KB3172615 and KB3179575 instead of the two below that are recommended for uninstall.

    KB3172614 July

    KB3179574 August

    Are the two kb's I mentioned updated versions of these updates?  Should we uninstall them.  FYI we are running webroot av and have version 8.8.88 installed.

    Thursday, November 3, 2016 1:25 PM
  • We "have"/are facing the same issue with webroot. And this one. (I think this is an other one)
    Webroot gave us an set of settings 
    https://download.webroot.com/Citrix/Citrix.pdf
    We applied only the webroot settings no reg settings because we don't have citrix running.


    • Edited by Marcmk Thursday, November 3, 2016 11:27 PM
    Thursday, November 3, 2016 11:24 PM
  • Hi All,

    We also had the same problem, after removing both windows updates the customer hasn’t had any problems anymore.

    Microsoft support, we all like J

    Regards

    Stephan

    Friday, November 4, 2016 8:45 AM
  • Hi All,

    Just want to know if this issue were already solved? Is there any updates from Microsoft?.

    We also have the same problem regarding this.

    Regards,

    Mark Edel

    Tuesday, November 8, 2016 7:55 AM
  • We are having the same issue with our 100+ RDS server farm. When the issue occurs on one of our RDS servers, we have monitoring software listening for the 4005 event. We will then set the server to not allow new connections until reboot. However, the users who have a disconnected session on this host, will still connect to this server and receive a blank screen and we must force logoff the user with the chance of them losing some work.

    We have automated the entire deployment process from the very build of the image, so the servers are always up-to-date and just careless killing processes or uninstalling updates is not an option. We are looking forward to a solution!

    Unfortunately we are getting sporadic 4005 events w/o the host immediately experiencing blank logon screen. So we automated the whole thing up to sending a notification mail to admins who have to manually check if host is responsive.

    If blank logon is confirmed then we restart TermService so that reconnects (routed by conn broker) do not hang on blank screens, using a batch file like this

    :: restart_rds.bat
    @echo off
    setlocal
    echo Starting at %date% %time% >> %~dpn0.log
    for /f "tokens=2" %%i in ('tasklist /svc ^| findstr /i termservice') do set pid_to_kill=%%i
    echo pid_to_kill=%pid_to_kill% >> %~dpn0.log
    taskkill /f /pid %pid_to_kill% >> %~dpn0.log 2>&1
    ping -n 2 localhost > nul
    net start termservice >> %~dpn0.log 2>&1
    echo Done at %date% %time% >> %~dpn0.log

    Our hosts desnsity is 50-100 users per host. The servers with more that 100 users are more vulnerable as these get more reconnects obviously. We experience 4005 daily on all our hosts and need to restart TermService 2-3 times (daily).

    We are not planning on uninstalling offending monthly rollups till the end of the year as we expect MS to (eventually) fix the whole mess.

    Will test TCP only suggestion and post results back here.

    cheers,
    &lt;/wqw&gt;

    Tuesday, November 8, 2016 12:06 PM
  • Hi Everyone,

    Have been hanging out for the fix as well.

    Microsoft released there Nov 2016 updates here https://technet.microsoft.com/en-us/library/security/ms16-nov.aspx

    Cannot see any reference to this particular issue (unless they have hidden it within a security update).

    I will most likely perform updates late this week.

    regards,

    Wednesday, November 9, 2016 1:17 AM
  • Hi Everyone,

    Have been hanging out for the fix as well.

    Microsoft released there Nov 2016 updates here https://technet.microsoft.com/en-us/library/security/ms16-nov.aspx

    Cannot see any reference to this particular issue (unless they have hidden it within a security update).

    I will most likely perform updates late this week.

    regards,

    I'm running my patches tonight in hopes that this issue will be resolved.  
    Wednesday, November 9, 2016 12:49 PM
  • We also see the Black screen of logon death on one/two randoms out of our 30 server Farm.  The issue is documented here: https://support.microsoft.com/en-gb/kb/3172614

    Issue 1

    Symptoms
    After you apply this update on a Remote Desktop Session (RDS) host, some new users cannot connect to an RDP session. Instead, those users see a black screen, and they are eventually disconnected. This issue occurs at unspecified intervals. 

    Cause

    During virtual channel management, a deadlock condition occurs that prevents the RDS service from accepting new connections.

    The workaround is as others have mentioned:

    To find the RDS svchost server, follow these steps:

    1. Start Task Manager.
    2. On the Services tab, search on TermService.
    3. Right-click the TermService entry, and then click Go To Details

      Note This highlights the svchost.exe entry on the Details tab.
    4. Right-click svchost.exe, and then click End Task.

    5. We also have to restart the RDS service.

    We have begun plans to remove the offending two patches: KB3179574 and KB3172614 .

    An additional thread can be found here from Page 3.  Some interesting discussion about rdpcorets.dll -

    Hello again,

    We are planning to update the July and August rollup KBs to list this problem as a known issue. I believe some people report the issue persists after removing both rollups. If this applies to you can you please check that the date on the c:\windows\system32\rdpcorets.dll file pre-dates July 2016? If the file is dated July 2016 or later then you haven't rolled back all updates that contain the file - which at present should only be the July (KB3172614) and August (KB3179574) rollups. At the moment we are reasonably sure that the issue is within rdpcorets.dll.

    Thanks,

    Sasha Loncarevic (MSFT)

    Lea

    Sorry just seen this has all been covered previously in this thread...so I'm not surprised this thread is HUGE, this really is a bombshell with no real resolution so far, only workarounds!



    • Edited by LeaUK Wednesday, November 9, 2016 2:37 PM
    Wednesday, November 9, 2016 2:31 PM
  • We only had KB3172614.  Removed it from the 2 servers in the farm having the issue and it's been 2 weeks since our last incident. It was happening everyday so we are hopeful that is the fix until Microsoft finally releases an updated patch.
    Wednesday, November 9, 2016 8:43 PM
  • Hi Everyone,

    Have been hanging out for the fix as well.

    Microsoft released there Nov 2016 updates here https://technet.microsoft.com/en-us/library/security/ms16-nov.aspx

    Cannot see any reference to this particular issue (unless they have hidden it within a security update).

    I will most likely perform updates late this week.

    regards,

    I'm running my patches tonight in hopes that this issue will be resolved.  

    while November patch day didnt patch the rdpcorets.dll the Problem wont be fixed through this...
    Thursday, November 10, 2016 1:15 PM
  • As this is causing severe disruption, why are some choosing to wait for a fix? Surely the security risk mitigation of the patch doesn't outweigh the horrendous inconvenience to users in the short term?

    Interested in your views about the decision to keep known bug ridden patches installed?

    Also if the business decision is such to keep these patches installed, perhaps a simple revocation of rdpcorets.dll may resolve and satisfy both requirements?  Has anyone tried this?

    Lea

    Thursday, November 10, 2016 3:18 PM
  • Kyle,  Did the issue ever reoccur after you made the above changes?
    Friday, November 11, 2016 4:42 PM
  • Hello Sasha,

    --------------------------------------------

    "July 21, 2016 – KB3172614"

    Known issues in this update

    Issue 1

    Symptoms
    After you apply this update on a Remote Desktop Session (RDS) host, some new users cannot connect to an RDP session. Instead, those users see a black screen, and they are eventually disconnected. This issue occurs at unspecified intervals.

    Cause

    During virtual channel management, a deadlock condition occurs that prevents the RDS service from accepting new connections.

    Status

    Microsoft is researching this problem and will post more information in this article when the information becomes available.

    ---------------------------------------

    Does it mean the issue is still not fixed?

    >>>During virtual channel management.

    I have noticed issue on dynamic virtual channels on servers with new rdpcorets.dll, where I am wondering why it happens.

    As example I reuse dynamic virtual channel and I have noticed that when I register it under proposed ID as example under ID=11, after disconnecting/reconnecting with connection restoring to same session the ID gets changed to 10 even thought expected this ID to be either the same or at least incremented, but not decremented. That happens now with rdpcorets.dll version 6.3.9600.18402. even before the black screen issue occurs. So I assume these issues are common since virtual channel related, but it did not behave so before. I hope this info helps you by your researchings to release the fix asap.

    Friday, November 11, 2016 8:14 PM
  • All,

    MS have just announced that the fix for the deadlock issue is included in the Nov, 2016, Rollup Preview. Here's a snippet in the known issues section of the August rollup doc'n:

    >>>>>>>>>>>>>>>>>>>

    Cause

    During virtual channel management, a deadlock condition occurs that prevents the RDS service from accepting new connections.

    Resolution

    To fix this issue, install November 2016 Preview of Monthly Quality Rollup for Windows 8.1 and Windows Server 2012 R2 (KB3197875).

    >>>>>>>>>>>>>>>>>>>

    Here's the URL for the Nov Preview bulletin:

       https://support.microsoft.com/en-us/kb/3197875

    I haven't downloaded the csv with the detailed file information and the web page that will contain that info is not yet posted.

    I'll test this as soon as I can free up one of my test nodes from other tasks.

    Loel




    Tuesday, November 15, 2016 6:04 PM
  • All,

    In addition to the information provided by Loel.

    If possible, please install the November 2016 Preview of Monthly Quality Rollup for Windows 8.1 and Windows Server 2012 R2 and reply back here if the issue still exists.

    You may download details regarding the files included in the update here:

    http://download.microsoft.com/download/6/D/B/6DBDF0E0-FF0D-45A5-9605-ED14D395662D/3197875.csv

    Description of fixes:

    https://support.microsoft.com/en-us/help/24717

    Thanks.

    -TP

    • Marked as answer by HMAR Tuesday, November 15, 2016 11:29 PM
    Tuesday, November 15, 2016 7:30 PM
    Moderator
  • All,

    The new version number for "rdpcorets.dll" is 6.3.9600.18512, 8 October, 2016.

    Tuesday, November 15, 2016 9:10 PM
  • I'm going to install this evening and hope it's fixed.
    Wednesday, November 16, 2016 9:05 AM
  • I installed it last night on our RDC server and so far no issues this morning.
    Wednesday, November 16, 2016 3:16 PM
  • Overview of known issue:

    Webroot has been actively tracking and monitoring an issue that affects some customers operating Terminal Servers in a virtualized environment. During a fault condition, a Winlogin 4005 error occurs where an RDS user is unable to connect to the RDS server, and the event logs display a message indicating that Winlogin has stopped responding.

    Update: Interim advice issued by Webroot engineering teams recommended setting the WSAB Self-Protection function to ‘Minimum’ on all Server Policies as this significantly reduced the frequency of Winlogin 4005 errors in most cases. Webroot engineering teams have developed a fix which is currently undergoing regression and environment testing. The engineering and QA teams continue to treat this as their highest priority and are predicting this new build will be targeted for release the week of 28th November. Webroot believes this fix will eliminate the errors seen. Following successful roll out of the fix, and a short review period, Webroot will remove the recommendation to operate Self Protection at the minimum setting.

    Next Steps: Webroot will communicate an update to this message on/by Friday 25th November. If you face excessive issues with 4005 errors, please contact Webroot Support for assistance. In specific casesWebroot may be able to provide a beta build of the fix. Note that this beta version will need to be deployed with assistance from the Webroot Support team and involves additional steps not required with the fix release. Webroot apologizes for the disruption and inconvenience caused and for the time taken to manage this complex fault.

    We also thank you for your continued patience and understanding in this matter.

    Kindest regards,

    The Webroot Product Management Team
    Friday, November 18, 2016 6:04 PM
  • We are having the same symptoms on a couple of newly installed Windows Server 2016 terminal servers with XenApp 7.11 . Any idea if this might be the same issue?

    The only hint I'm getting is from the Application Eventlog which is giving me the following error:
    Event 4005 Winlogon - The Windows logon process has unexpectedly terminated.

    Followed by this information event:
    The Citrix Desktop Service detected that a user session has ended. Session ef6966e5-d13e-4847-b337-cc2ec4028d34 for user '*****' has ended; reason code Logoff.

    Our Windows Server 2012 R2 with XenApp 7.11 are not having this issue, but we didn't patch them with the affected updates mentioned earlier in this post.

    Tuesday, November 22, 2016 9:27 AM
  • Has been over 2 weeks and no further issues.

    Hopefully call it a win.

    Yah,

    Tuesday, November 29, 2016 4:36 AM
  • All,

    It's been two weeks (14 days exactly) since I finished patching our production 2012R2 RDS Session Host farm, via  KB3107875 ("November 2016 Preview of Monthly Quality Rollup"). There has been no recurrence of the deadlock condition, so we are closing our ticket with MS and closing our internal incident ticket.

    Cheers!

    Loel

    Tuesday, December 6, 2016 6:46 PM
  • Same here. No more RDS black screen issues after installing  November 2016 Preview of Monthly Quality Rollup. Regs, Martin
    Friday, December 23, 2016 7:05 PM
  • Well, the Nov rollout is gone and the new Dec one doesn't resolve the issue.

    We installed the bug back in October. We never applied the nov. rollup since it was optional, and by the time we got here and saw that is was the solution, it was gone.

    We still have a ticket with MS on this, but they don't seem to have a grasp on it. Their recommendation was to uninstall Dec. rollup and apply nov's which isn't even avail.


    Ryan Mathes

    Wednesday, December 28, 2016 6:48 PM
  • Well, the Nov rollout is gone and the new Dec one doesn't resolve the issue.

    We installed the bug back in October. We never applied the nov. rollup since it was optional, and by the time we got here and saw that is was the solution, it was gone.

    We still have a ticket with MS on this, but they don't seem to have a grasp on it. Their recommendation was to uninstall Dec. rollup and apply nov's which isn't even avail.

    Hi Ryan Mathes,

    I just went to the Microsoft Update Catalog and tested and was able to download the November update.

    -TP

    Wednesday, December 28, 2016 7:06 PM
    Moderator
  • Sure enough. I recieved this last night and this morning and reported it to MS.


    Ryan Mathes

    Wednesday, December 28, 2016 7:34 PM
  • Hi Ryan,

    Chrome or Firefox work fine.  For IE you can hit back after the error is displayed.  Of course it needs to be fixed, but in the meantime it is easy to get around the problem.

    -TP

    Wednesday, December 28, 2016 8:14 PM
    Moderator
  • thanks for your help!

    I've scheduled a maintenance window for tomorrow evening to remove Dec. rollup and install the nov. rollup.



    Ryan Mathes

    Wednesday, December 28, 2016 10:29 PM
  • So an interesting observation.  We had the KB3197875 patch installed, then when the December roll-up was installed, it no longer shows KB3197875 as being installed.  We installed the December roll-up and a few days later we had another black screen on one of our servers.  So is the fix from KB3197875 not contained in the December roll-up and is it removed when you install the december roll-up?
    Tuesday, January 3, 2017 4:20 PM
  • Dec. Rollup is a no go. Definitely causes the same error.

    Last week we removed Dec's rollup, and installed Nov's rollup. It's too early to tell, but we'll update back here in a few weeks. I'll also pass this info along to MS.


    Ryan Mathes

    Tuesday, January 3, 2017 4:26 PM
  • We installed the Dec Rollup and the black screens on Winlogon 4005 crashes have stopped, so apparently the deadlock problem is solved.

    However, the number of Winlogon 4005 crashes has actually increased.  A year ago, these crashes would result in users that could not be logged off or their processes terminated, which required a server reboot to recover.  Then after a few months (and presumably some hotfixes acquired through Windows Update), the server would slowly over the course of 15-20 minutes clean up the hung threads and eventually drop the user.  Then a few months later we started seeing the black screens on Winlogon 4005 crashes, again requiring a server reboot to recover (until this thread revealed a way to isolate the TermServices process which can be killed and the service started again as a workaround).

    In all this time, the Winlogon crash logging a 4005 event is at the root of each of these issues.  I suppose it's possible that it's not causal and that something deeper is at root here, but I just don't believe it with the evidence at hand.

    So the question I still have to ask is -- when is Microsoft going to fix Winlogon so it doesn't crash?  Why must we struggle with all the symptoms and hotfixes and workarounds while the core problem remains unfixed over the course of a dozen plus months?

    Friday, January 6, 2017 6:02 PM
  • Perhaps even more disturbing is that this issues is occurring on Windows Server 2016! I am faced with a daily restart schedule on a RDS server running Windows Server 2016 because of this same 4005 Winlogon error, so not only has it not been addressed but it has been rolled over into the next generation.

    At least with Server 2012 R2 there is a large enough body of users to force some prompt (?) fix but I suspect this will become another example of Microsoft punishing the early adopter?

    Saturday, January 7, 2017 9:07 PM
  • Is Microsoft still following this thread? I must say I'm in doubt to install KB3205401 (Dec Rollup) on my Production RDS servers next week, when I read this thread. The knowledge base article for the Dec rollup doesn't mention any known issues regarding Remote Desktop after installing the rollup.

    It's hard for me to replicate the issue on non-production RDS hosts, as it seems to be quite random.

    Thursday, January 19, 2017 11:22 AM
  • Hi Microsoft, Is there still a solution after KB3205401 (Dec Rolloup)? Should we try to install the Nov. Patch manually  again?

    Best regards

    @Arcom76: We do have installed the DEC Patch and still having these issues. Do you have installed DEC and NOV Patches? Still facing the issue?

    Tuesday, February 14, 2017 9:26 AM
  • It happens to me on a very vanilla Windows 10 machine too.    VNC logins are no problem.   Just RDP. 
    • Edited by DJRobx Sunday, March 12, 2017 5:55 PM
    Sunday, March 12, 2017 5:53 PM
  • Thanks.

    It worked for me.


    Regards, Saikrishnna K

    Tuesday, March 14, 2017 4:05 PM
  • Hey

    I have the exact same problem using Windows Server 2016.

    Any solution?

    Michael

    Sunday, May 28, 2017 9:12 AM
  • Ugh!  

    And your running Terminal Services on that Server? 

    Monday, May 29, 2017 2:41 PM
  • I don't have a definitive fix, but since my RDP servers can't just be rebooted when in production I force reset the Terminal Server Service via command line. I have this issue on my 2012 R2's and never could get a true fix. You can Console into the Server fine and all the services appear fine, but when you RDP you get the black screen of death with normal bounce back out after some time.

    Maybe this helps someone out there so they don't have to reboot their server with people on it that work and those trying to connect can't;

    From a remote computer;

    C:\>tasklist /s \\<server> /svc /fi “imagename eq svchost.exe” (From the list locate the PID for TermService)

    C:\>taskkill /s \\<server> /pid <nnnn> /f

    C:\>sc \\<server> start TermService

    Good Luck and if anyone does ever get the Hot Fix or Permanent Fix please post it here :)
    Thursday, June 8, 2017 6:05 PM
  • I'm experiencing this in Windows Server 2016.

    Is there a fix?

    Wednesday, July 26, 2017 1:07 PM
  • This did not work for me on Server2016.
    The commands did restart TermService and its running under a new PID, but the black screen persists.
    Wednesday, August 2, 2017 4:26 PM
  • we are facing the "black screen upon login" issue too.
    • Proposed as answer by hfgdgf Thursday, August 31, 2017 8:43 AM
    Thursday, August 3, 2017 11:45 AM
  • We tryed setting the pagefile to a bigger size. We had 8GB, and set it to 16, then we could log in again. Hope this will help some, since i dont think this solution apply to all. 

    We could see our ram was almost maxed out, and noticed than pagefile even dynamic could not handle it. So we manually set a second pagefile, on another drive, and lord behole, our users could log in.

    Thursday, August 31, 2017 8:53 AM
  • Good suggestion, just had the issue crop up on one of our 2016 Std servers.

    Windows had dynamically allocated a 2GB page file on a server with 16GB RAM. Set it manually to 16GB and RDP immediately starting working again without a reboot.

    Thanks for sharing!

    Tuesday, September 5, 2017 10:55 AM
  • This happened to me on Windows Server 2016 the system locked up. I was not able to RDP, log in to it locally or get a response with the keyboard. The logon screen was up and the time kept sync, after pressing the power button on the Cisco C220M4 it began to gracefully shut down. After about 10 minutes of shutting down, all screen activity froze and I was forced to hard shut down the machine. I powered the server back on and saw the 4005 Windows logon process had failed. This is the first occurrence.
    Thursday, September 7, 2017 10:16 PM
  • Hey

    From MSS: (Windows Server 2016)

    Symptom: "Event ID 4005 - The Windows logon process has unexpectedly terminated"

    Resolution: Hotfix in development, expected to be released later this year.

    Thursday, October 12, 2017 4:23 PM
  • i was hoping for a release of this fix in november :/
    Wednesday, November 22, 2017 10:06 AM
  • April, 2018 :s
    Wednesday, April 25, 2018 3:48 PM
  • Try configuring GPO in: 

    Computer Configuration\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Session Host\Device and Resource Redirection

    Do not allow supported Plug and Play Device Redirection - DISABLED

    Some of local devices is trying to connect thru RDP session, causing deadlock on RDP service and timeout on WinLogon.


    • Edited by Kojo1984 Saturday, July 7, 2018 7:23 PM
    Saturday, July 7, 2018 7:22 PM
  • seems that this issue returned on RDS Server 2016.
    Running a RDS Server with current updates (mid. December 2018) and connecting with a Wyse T10 Thinclient with FW 8.4.XX creates the same issue.

    Got Event ID 36:
    Source: TerminalServices-LocalSessionManager

    Fehler beim Übergang von "CsrConnected" in Reaktion auf "EvCsrInitialized". (Fehlercode: 0x80004005)

    I've tried to set the option "Do not allow supported Plug and Play Device Redirection" to DISABLED, as mentioned above.. let's see if it helps..

    Has someone else also issue with Server 2016 and Wyse Thinclients?

    Regards
    Simon

    Tuesday, December 18, 2018 4:06 PM
  • I've found now the main reason with Wyse Thin Clients and the Black screen of death.
    DO NOT USE THE "RDP USB" Option under the general Settings. Use "TCX USB" insted.

    Otherwise, when you have RDP USB option enabled and connect a USB stick on the thin client, every new RDP connection on the Server will get the black screen until you reboot.

    • Proposed as answer by Simon Tauber Thursday, December 27, 2018 1:59 PM
    Thursday, December 27, 2018 1:59 PM
  • I can confirm we are having this exact issue!!!! Do you have any update on this? 
    Monday, March 25, 2019 5:18 PM
  • We do not use TCX and TCX is also now EOL - we cannot disbale usb re-direciton as our business have certain requirements for it. Has this issue definitely been resolved for yourself?
    Monday, March 25, 2019 5:19 PM
  • Is there a hotfix for this problem? I can't disable usb ports redirection on remote sessions.
    Friday, March 29, 2019 10:35 AM