none
RDS Servers Events 7011, 7046 - BSOD rdbss.sys RRS feed

  • Question

  • Hi All

    I have a virtualised (VMWare) RDS 2012R2 environment with 20 Session hosts spread across 6 Dell ESXI Hosts - 2 Sets of different PowerEdge Models. Over the past 4-6 weeks we have started to get multiple event 7011's followed by a 7046.

    A timeout (30000 milliseconds) was reached while waiting for a transaction response from the UmRdpService service.

    The following service has repeatedly stopped responding to service control requests: Remote Desktop Services UserMode Port Redirector

    At this point some existing connected users cant sign out and applications start to crash including explorer.exe. Trying to shutdown via the GUI just hangs and the only way to get the server back is to reset the power using vSphere console. 

    Applications on the Session Hosts are mainly MS Office 2016, Acrobat Reader, 7Zip and Webroot AV. Windows OS and applications are fully patched and up to date and Dell Firmware and drivers are fully up to date. 

    Users connect in via RemoteApp and local drives and printers are redirected into their sessions. 

    The weird thing is, like clockwork the crashes happen at the end of each day usually between 16:00 - 18:00 - To me its like a degradation symptom or perhaps its the actions of users disconnecting or logging off their session - Its affecting a couple of servers each day. 

    On top of this, it appears 7011, 7046 results in a BSOD. I have grabbed the Memory.dmp file and opened it with WinDbg. 

    Im now trying to figure out the dmp - uploaded to PasteBin here (happy to paste dmp here but didnt want to "dump" to much information in the post)

    What stands out to me is rdbss.sys

    Probably caused by : rdbss.sys ( rdbss!__RxAcquireFcb+1f3 )

    IRQL_NOT_LESS_OR_EQUAL (a)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If a kernel debugger is available get the stack backtrace.
    Arguments:
    Arg1: 0000000000000000, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, bitfield :
    bit 0 : value 0 = read operation, 1 = write operation
    bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
    Arg4: fffff80179d3ba44, address which referenced memory

    BUCKET_ID:  AV_rdbss!__RxAcquireFcb

    PRIMARY_PROBLEM_CLASS:  AV_rdbss!__RxAcquireFcb

    My rdbss.sys version - 6.3.9600.18895

    Can anyone help to try and decipher the above and suggest next/best cause of action?

    Many thanks :)


    • Edited by Tee-Eff Thursday, March 14, 2019 10:43 PM
    Thursday, March 14, 2019 10:42 PM

Answers

  • I think I have found the cause (and fix) for this......

    Opening a PDF in Adobe Acrobat Reader DC (Our Version 19.010.20098) in a RDS Session and then file-save that PDF to local C Drive \\tsclient\C\) results in a weird error followed by a 0kb file. Then upon trying to opening the local PDF you get an error that the file is in use by rdp. If you then log out of the Session Host, 7011's start to appear eventually ending in a complete server crash. This is re-creatable every time.

    The Adobe setting that fixes this is to turn off "Run in AppContainer" in the Security tab or dword via GP:

    HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown\bEnableProtectedModeAppContainer 0

    A quick read on bEnableProtectedModeAppContainer is that its some sort of sandbox that evidently has caused our RDS servers to 'flip out'!

    What a nightmare!

    Wednesday, March 20, 2019 8:49 AM

All replies

  • Hi,

     

    >BugCheck A, {0, 2, 0, fffff80179d3ba44}

    >Probably caused by : rdbss.sys ( rdbss!__RxAcquireFcb+1f3 )

    > An attempt was made to access a pageable (or completely invalid) address at an interrupt request level (IRQL) that is too high.  This is usually caused by drivers using improper addresses.

     

    Please open CMD with admin permission, then, using “sfc /scannow” to check/repair system files.

     

    Besides, please use DISM.exe to check the system files again to have an confirmation - Repair a Windows Image:

    https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/repair-a-windows-image

     

    > the crashes happen at the end of each day usually between 16:00 - 18:00

    Is there any task, scanning process which is scheduled to be running during this period of time? Also, please make sure that you have enough resources for your VMs to be running, including CPU, RAM and DISK.

     

    Best Regards, Eve Wang


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Friday, March 15, 2019 2:20 AM
    Moderator
  • Hi Eve

    sfc /scannow returns:

    Beginning verification phase of system scan.
    Verification 100% complete.

    Windows Resource Protection did not find any integrity violations.

    Dism /Online /Cleanup-Image /CheckHealth returns:

    Deployment Image Servicing and Management tool
    Version: 6.3.9600.17031

    Image Version: 6.3.9600.17031

    No component store corruption detected.
    The operation completed successfully.

    Friday, March 15, 2019 12:51 PM
  • Hi,

     

    As you had mentioned that, problem happens recently, if possible, undo the change, including new software and update installation, configuration change if there is any before problem happens and check the result.

     

    Also, if problem happens during specific time period, please capture process details, and compare it with other period time, try to find the difference.

     

    If you want to find root cause, detail dump file analyzing is necessary. I would suggest you contact Microsoft Customer Support and Services where more in-depth investigation can be done so that you would get a more satisfying explanation and solution to this issue.

     

    Global Customer Service phone numbers:

    https://support.microsoft.com/en-us/help/13948/global-customer-service-phone-numbers

     

    Best Regards,
    Eve Wang


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Monday, March 18, 2019 8:52 AM
    Moderator
  • Hi

    No I am still really struggling with this. Something really weird I have noticed is they seem to be crashing in pairs.

    I have:

    serva,servb,servc,servd,serve,serve

    serv1,serv2,serv3,serv4,ser5,serv6

    And the last time this happened, serva and serv1 crashed within 5 minutes of each other. Then a few days later servd and serv4 crashed did the same thing And then the same thing with 2 others. Is this just a coincidence?

    Tuesday, March 19, 2019 4:35 PM
  • I think I have found the cause (and fix) for this......

    Opening a PDF in Adobe Acrobat Reader DC (Our Version 19.010.20098) in a RDS Session and then file-save that PDF to local C Drive \\tsclient\C\) results in a weird error followed by a 0kb file. Then upon trying to opening the local PDF you get an error that the file is in use by rdp. If you then log out of the Session Host, 7011's start to appear eventually ending in a complete server crash. This is re-creatable every time.

    The Adobe setting that fixes this is to turn off "Run in AppContainer" in the Security tab or dword via GP:

    HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown\bEnableProtectedModeAppContainer 0

    A quick read on bEnableProtectedModeAppContainer is that its some sort of sandbox that evidently has caused our RDS servers to 'flip out'!

    What a nightmare!

    Wednesday, March 20, 2019 8:49 AM
  • Hi,

     

    Thank you for taking the time to share details. Your share might be helpful for other people who has similar problem.

     

    If there is anything else we can do for you, please feel free to post on the forum.

     

    Best Regards,

    Eve Wang


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, March 21, 2019 2:10 AM
    Moderator
  • I think I have found the cause (and fix) for this......

    Opening a PDF in Adobe Acrobat Reader DC (Our Version 19.010.20098) in a RDS Session and then file-save that PDF to local C Drive \\tsclient\C\) results in a weird error followed by a 0kb file. Then upon trying to opening the local PDF you get an error that the file is in use by rdp. If you then log out of the Session Host, 7011's start to appear eventually ending in a complete server crash. This is re-creatable every time.

    The Adobe setting that fixes this is to turn off "Run in AppContainer" in the Security tab or dword via GP:

    HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown\bEnableProtectedModeAppContainer 0

    A quick read on bEnableProtectedModeAppContainer is that its some sort of sandbox that evidently has caused our RDS servers to 'flip out'!

    What a nightmare!

    I was getting mad with this. I saw it many times in logs that when adobe hangs, the error events 7011 and ghosts users disconnected with (4) processes appears. The only fix was to restart the vm.

    I'll give this solution a try. I hope this will be the end of this nightmare. 

    Thank you.

    Tuesday, March 26, 2019 6:12 PM
  • I think I have found the cause (and fix) for this......

    Opening a PDF in Adobe Acrobat Reader DC (Our Version 19.010.20098) in a RDS Session and then file-save that PDF to local C Drive \\tsclient\C\) results in a weird error followed by a 0kb file. Then upon trying to opening the local PDF you get an error that the file is in use by rdp. If you then log out of the Session Host, 7011's start to appear eventually ending in a complete server crash. This is re-creatable every time.

    The Adobe setting that fixes this is to turn off "Run in AppContainer" in the Security tab or dword via GP:

    HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown\bEnableProtectedModeAppContainer 0

    A quick read on bEnableProtectedModeAppContainer is that its some sort of sandbox that evidently has caused our RDS servers to 'flip out'!

    What a nightmare!

    I was getting mad with this. I saw it many times in logs that when adobe hangs, the error events 7011 and ghosts users disconnected with (4) processes appears. The only fix was to restart the vm.

    I'll give this solution a try. I hope this will be the end of this nightmare. 

    Thank you.

    Confirmed, no more crashes with the deploy of the GPO disabling adobe appContainer. 
    Thank you!

    Friday, March 29, 2019 4:03 PM

  • THANK YOU!!!! This was what caused my issues. I think this automatically got enabled when adobe auto-updated. Lesson learned!
    Monday, April 15, 2019 7:00 PM
  • Im glad this has helped out a few people :)
    Wednesday, April 17, 2019 1:01 PM
  • Jesus, I only had this issue happen twice in the last 10 days, but I was going mental trying to figure it out... I do hope the Adobe DC workaround fixes it, I knew of the issue in trying to save PDFs to \\tsclient\ share but wasn't aware of the possible connection with RDS and Print Spooler stalling and hanging. I'm going to try this out, but I'm not too sure about the explanation. In case you change this setting at a user level, it is saved as the 32bit REG_DWORD named "bEnableProtectedModeAppContainer" with the value 0 in the root of the key "HKCU\Software\Adobe\Acrobat Reader\DC\Privileged\", is my interpretation correct to apply this system wide you should do the same by creating the same REG_DWORD under the root of the key "HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown\" withgout the need for any additional sub-key?

    And thanks for sharing the solution, worst case I'll uninstall Adobe Reader.

    Wednesday, April 17, 2019 3:46 PM
  • thank you so much, this one is very difficult. So sick of Adobe.
    Tuesday, May 21, 2019 6:56 PM
  • We faced a rash of these same errors and RDS 2012R2 crashes a few months ago. Our issue seems to be related to Printer drivers and we were opening Reliability History to see hundreds of crashes of the print spoiler a day which also left these orphaned ghost sessions. I don’t feel like we fully understand the issue in our case as we don’t have adobe reader, but rather Acrobat Pro 11.0.21 running on the hosts. I was unable to find the key or security settings that match, so there are potentially other sources for these issues. We have a mix of Xerox and HP printers. Using BombProf to cleanup all profiles and wiping out all printer drivers seems to have stabilized the environment for now. We are migrating to a new peri this server with all Universal Drivers that seems to be making things better for both RDSH and Workstation users.

    -Scott

    Wednesday, May 22, 2019 3:16 PM
  • Thanks for this information of great value.

    In our case, first problem appeared the 21th january 2019.
    As you said : each time it was between 17 and 18pm

    On both our Windows server 2012R2 RDS. (sometimes only one, sometimes both of them in a short interval)

    Sometimes twice a week and sometime once a month

    Very frustrating not having any clues of what was happeming (and knowing it could happen anytime)

    We are currently trying your solution and will report back.

    Thursday, May 23, 2019 3:57 PM
  • Hello,

    I have similar issue going on since weeks now and all the symptoms mention above are same in our scenario.

    On the server I have created the suggested registry key but still the issue  is reoccurring just wanted to confirm on the registry parameter do I have to create a DWORD value with "Run in AppContainer" in bEnableProtectedModeAppContainer key or just till the bEnableProtectedModeAppontainer and value it O?

    I do see the "Run in AppContainer" under preference in Adobe and I uncheck it from there as well.

    Kindly suggest the exact steps needs to be taken over here.

    Thanks,

    Aatif

    

    


    Regards, Aatif Kungle

    Tuesday, June 4, 2019 12:04 PM
  • Hello,

    just to report back that the issue didn't occur since the "adobe reader fix" has been applied.

    @AatifKungle = 

    you have to create a new DWORD value named "bEnableProtectedModeAppContainer" under "HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown"

    Value = "0" (zero)

    Sorry, I can't post a screenshot to show you our "regedit" screen (account limitation)

    Few keywords to help other people finding this thread = 

    BSOD, Windows server, 2012 RDS, Remote Desktop Session, crash, unresponsive, unable to close program, unable to log in or log out

    Best regards


    • Edited by SogaM02 Thursday, June 6, 2019 7:44 AM
    • Proposed as answer by AatifKungle Friday, June 7, 2019 6:49 PM
    • Unproposed as answer by Tee-Eff Friday, June 7, 2019 7:17 PM
    Thursday, June 6, 2019 7:41 AM
  • Thank you for the reply @SogaM02

    I applied the below solution and from last three days problem didn't reoccur.Hoping for it not to occur ever again:-)

    • <section>

      Keypath:          HKLM\SOFTWARE\Wow6432Node\Policies\Adobe\Acrobat Reader\DC\FeatureLockDown

      Value name:   bProtectedMode

      Value type:     REG_DWORD

      Value data:     0

      </section>

    Regards,

    Aatif


    Regards, Aatif Kungle

    Friday, June 7, 2019 6:52 PM
  • @SogM02, AatifKungle - Glad to hear the bEnableProtectedModeAppContainer Reg value did the trick. 
    Friday, June 7, 2019 7:19 PM
  • OMG - you are God - it works!
    you are God - it works
    Tuesday, June 25, 2019 9:21 AM
  • You're a lifesaver. It works!
    Thursday, July 4, 2019 3:17 PM
  • Two months of work thrown away. Every day with BSOD. A registry key

    One day with no errors after trick.. Thanks, thanks, thanks. 

    Tee-Eff I would like to know how you came to this solution. You are a crack. Bye bye Adobe.

    Friday, July 19, 2019 5:58 PM