none
Client Side Rendering on Terminal Servers

    Question

  • I’m having horrible printer issues on my Terminal Servers such as slow printing, slow to add printers, some applications load slowly because they are enumerating the list of printer (thanks Procmon), slow logons, etc. I'm trying to get Client Side Rendering (CSR) disabled. From what I can tell it still appears to be on. Terminal Server are Windows Server 2008 SP2 x86.

    On my print servers I have verified that all printers have Render print jobs on client computer unchecked. I have a Group Policy linked to the OU that the Terminal Servers are in that has Always render print jobs on the server set to enabled. Per http://blogs.technet.com/b/askperf/archive/2008/02/10/ws2008-client-side-rendering.aspx I have verified that on each Terminal Server in HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Printers the ForceCSREMFDespooling value is present & set to 1 so the GP is being applied.

    However I still have tons of entries in HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider\Servers\<SERVERNAME>\Printers.

    I have found KB958656 which seems to describe my problem that CSR is not being disabled. However one of the files it updates is older than a file that I have (win32spl.dll) & one is newer (printcom.dll).

    Printcom.dll in hotfix is 6.0.6001.22288 mine is 6.0.6001.18000

    Win32spl.dll in hotfix is 6.0.6001.22288 mine is 6.0.6002.18005

    Has anyone else seen this behavior & been able to get CSR disabled?

    Has anyone else had success with the KB hotfix mentioned or know whether or not installing it would causing any issues (since it has that older file)?

    Thanks in an advance to any input.


    Patrick Hoban
    http://patrickhoban.wordpress.com


    Wednesday, June 13, 2012 4:59 PM

All replies

  • Client Side Rendering IS disabled if the print job arrive at the print server in EMF format.  Pause the print share and send a job.  Open the properties of the job if RAW the data is rendered on the client, if EMF, the server will process the print job.

    The ClientSideRendering Print provider is win32spl.dll, there is not any way to disable that


    Alan Morris Windows Printing Team

    Wednesday, June 13, 2012 9:33 PM
    Answerer
  • I have paused the printer & sent a test page to it. I see the SPL & SHD files on the print server in C:\Windows\System32\spool\PRINTERS. How do you tell if the SPL file is RAW or EMF?

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Thursday, June 14, 2012 2:49 AM
  • Open the queue view or from Printmanagement Enable Extended View in Printer section.  Select job, right click, Properties. 

    The Datatype entry is what you are looking for.


    Alan Morris Windows Printing Team

    Thursday, June 14, 2012 5:50 PM
    Answerer
  • Datatype is RAW meaning CSR is enabled.

    Again I'm still focusing on the fact that there are tons of entries in HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider\Servers\<SERVERNAME>\Printers. Watching that key using Procmon shows basically non-stop writes. Which makes me believe that CSR is still enabled even though all settings are to the contrary. KB958656 would appear to resolve the issue but my concern is with the file versions.

    Which brings me back to my original questions.

    Has anyone else seen this behavior & been able to get CSR disabled?

    Has anyone else had success with the KB hotfix mentioned or know whether or not installing it would causing any issues (since it has that older file)?


    Patrick Hoban
    http://patrickhoban.wordpress.com

    Thursday, June 14, 2012 8:45 PM
  • Any other thoughts?

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Tuesday, June 19, 2012 3:31 AM
  • The spooler always reads the Print Providor registry keys, it's how the client spooler knows the share is configured for EMF spooling.

    To answer your specific question

    1) I have not seen this behavior  and Yes I can easily disable CSR.

    2) No clue on the hotfix


    Alan Morris Windows Printing Team

    Tuesday, June 19, 2012 11:07 PM
    Answerer
  • So what am I missing since everything appears to be configured correctly yet CSR not disabled?

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Wednesday, June 20, 2012 11:51 PM
  • Not sure, I just uncheck the box on the sharing tab on the print server.  It could be the driver.  What OS version is the print server? What print driver are you using? 


    Alan Morris Windows Printing Team

    Thursday, June 21, 2012 1:34 AM
    Answerer
  • Windows Server 2008 SP2 x86. HP UPD v5.2.

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Thursday, June 21, 2012 2:06 AM
  • I'll send this information to the HP driver developer I deal with.    He may know if the driver does not support server side rendering on some cases.

    Just so I have the scenario correct.

    Remote desktop to Server 2008 SP2 x86 (the client should not matter).

    Connect to a shared printer from a server running 2008 SP2 x86 sharing the HP Universal Driver v5.2 with Render Printjobs on the client unchecked.

    Send a print job to the shared printer from the remote destop session.

    Actual: Job gets to the print server in the format of the device driver (PDL);  Expected:  EMF on the print server so the rendering will occur on print server.


    Alan Morris Windows Printing Team

    Thursday, June 21, 2012 2:19 AM
    Answerer
  • Correct. Terminal Server & print server are Windows Server 2008 SP2 x86. Driver for all printer is HP UPD 5.2. Render print job on client is unchecked on all printer queues on the print server. Pausing the queue & sending print job shows RAW format. Tons of sub-keys on the Terminal Server under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider\Servers\<SERVERNAME>\Printers.

    I was expecting the job to be EMF & no sub-keys in that key.


    Patrick Hoban
    http://patrickhoban.wordpress.com

    Thursday, June 21, 2012 2:54 AM
  • Spoke with HP this morning.  The Universal driver always renders on the client to the PDL of the device.  It will not generate EMF.

    The registry key HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider\Servers\<SERVERNAME>\Printers

    should always exist when the share is configured to render on the client or the server.   The client side rendering print provider is always in use for any print connection to a Windows server.


    Alan Morris Windows Printing Team


    Thursday, June 21, 2012 6:05 PM
    Answerer
  • Do they have any documentation stating that the UPD driver does not support disabling CSR?

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Saturday, June 23, 2012 5:13 AM
  • Alan,

    Hi, We have a large RDS environment, and registry bloat of the “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider” key slows the enumeration of printers to a point where they effectively don’t work.

    After a user logs off, the keys for their printers and CSR settings remain in the above location.  This key grew to 900MB on one server over a few weeks.

    If we delete this key, and reboot the servers daily, things work OK.  But this is not a sustainable solution. 

    We need CSR disabled, or we need the OS to delete the user keys as the log out.

    è  Through your HP contacts, do you know if the new (5.5 from late June ) release of HP UPD supports disabling CSR ?

    è  Do you know if MS provides any way of these keys being deleted as users log out, so they don’t grow forever ?

    Thanks

    David Frith

    Wednesday, August 15, 2012 1:20 AM
  • Disabling Client Side Rendering will not address this issue.  The data in the registry is for the Print Provider and will exist when CSR is enabled or disabled.

    The GUID printer names should be removed when the client disconnects.  The server names and forms are retained (basically forever) so they do not have to be readded when the TS client reconnects or adds a connection to another share from the same machine.


    Alan Morris Windows Printing Team

    Friday, August 17, 2012 4:19 PM
    Answerer
  • Alan, hi, thanks for your response. We have been given some registry key settings via a support call that do now enable the printer GUID entries to be removed at client log off. This was not enabled by default, as they were persistent forever (surviving a reboot).  The value is "RemovePrintersAtLogoff"=dword:00000001 under the  “… Providers\Client Side Rendering\” key.

    However the “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering\<user SID> keys still remain.

    One <user SID> key is created for every user of the server.  As there are potentially over 3000 users who may log on to an RDS server in this farm over time.  As this key is not cleared with a spooler restart or server reboot, this key has grown to have over 3000 sub keys, which causes significant performance delays with enumerating printers.
    The issue can be viewed here

    We have a support call open, and a work around of nuking the entire CSR key (and sub-keys) and rebooting the servers every two weeks, however if you have any further suggestions that would be appreciated.

    Tuesday, August 21, 2012 8:51 AM
  • I'm going to test the "RemovePrinterAtLogoff" value & see what I get. Sounds like you & I are in a very similar situation.

    Patrick Hoban
    http://patrickhoban.wordpress.com

    Tuesday, August 21, 2012 7:34 PM
  • Well, we have just had MS support try and close the call as ‘by design’.  Which is extremely frustrating.

    Whilst I understand that ‘by design’ does not actually mean that the feature was ‘designed’ to work this way – it just means that code handles the scenarios that were proposed at specification time – it’s just amazing that no one in either the Printing or RDS product teams had thought of this scenario.

    It seems like it could be fairly common to me;

    • 2008R2 RDS server farm with lots of users
    • 2008R2 print server hosting HP printers (they have been the most popular for many years)
    • HP UPD in use – as this is all that HP support in x64, and it is certified by MS
    • Clients map printers to the print server from within session

    It looks like the CSR code just doodles crap all through the registry with no regard to how/when it is going to be cleaned up.  It really does not look like CSR can handle any medium size RDS environment at all.

    Specific areas of registry bloat I have found so far are;

    • HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering\<user SID>
    • HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering\<SERVERNAME>\Printers\<GUID>
    • HKU\.Default\Printers\DevModePerUser
    • HKU\.Default\Printers\DevModes2

    .. and I think there are others

    I’m not even 100% certain the problem is _just_ registry bloat, there seems to be some other underlying resource constraints in the spooler service when as a client – as sometimes I can just restart the spooler and get some sort of printer enumeration working – without nuking the registry… possibly there is some kind of handle leak

    But I’m pretty p..ssed in trying to reverse engineer what the bug is in the logic

    Patrick – did you have any luck ?

    Alan – other than replacing all the HP printers in our environment with another vendor’s (that support disabling CSR) is there any way we are going to get an RDS server acting as a printer client to be able to enumerate client printers for more than a few hours ?

    Thursday, September 06, 2012 7:00 AM
  • Sorry forgot about this.

    Under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering I do not see any keys with user GUIDS however it is after hours & no one is logged in. I will look tomorrow.

    HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering\<SERVERNAME>\Printers\<GUID>. I do see these however I have started rebooting my Terminal Server nightly & I think they may be clearing them up. I'll watch closer.

    HKU\.Default\Printers\DevModePerUser. I do have a few in here (maybe 100) but I think they are old. I'm deleting them to see if they come back.

    HKU\.Default\Printers\DevModes2. Empty

    I will say that one of the things I notice was that when the spooler service started consuming around 200,000KB of RAM is when we really started to notice the issue. Since we have started rebooting nightly it has gotten better. Before when we tried to add a printer in a client's session it could take up to 20 minutes to install (or just flat out never install). Now each time I have tried it took longer than I would like (think XP/2003 speeds) but not an unresonable amount (around 30 seconds).

    I also came across this post (http://blogs.technet.com/b/askperf/archive/2012/03/06/performance-issues-due-to-inactive-terminal-server-ports.aspx) referencing KB2655998 & have started running the fixit each Sunday night. In the key that it mentions on each of my Terminal Servers I had a little under 1000 subkeys (they all start with #TS)! Since you are running R2 you apparently just have to run the fixit once then install a hotfix.

    I'll update again after checking those keys during the day. I'd be curious to know if you see the same correlation between the amount of RAM being used by the spooler service & the severity of the issue. Also, if the KB article mentioned is affecting you & if so, how many of those subkeys do you see. I know I took a screenshot of a couple of them,I'll see if I can find one & post it.


    Patrick Hoban
    http://patrickhoban.wordpress.com

    Friday, September 07, 2012 3:57 AM
  • Patrick, Hi, just to clarify, the entries under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering are not GUID's but SID's of users

    The screen capture of our environment can be viewed here: http://i469.photobucket.com/albums/rr52/dfrith20/registry.jpg

    WRT the link to article KB2655998, we have had that hotfix in place for quite a while, and also run the fixit to clean the registry.  We have moved away from redirected printers (which has actually caused our issue) - and that article is more specific to redirected printers.

    I'll take note of the spoolsv.exe memory usage next time we have issues.  It certainly is some kind of resource shortage.  Some assistance to identify it (the resource shortage) from MS would be great, but as noted they want to shut down the call and blame the WHCL certified HP drivers....without even a memory dump or perfmon trace.

    I think your issue and ours are slightly different, especially given the OS differences, but clearly similar in nature

    Friday, September 07, 2012 12:25 PM
  • I have checked today & do not see any user sid entry's under that key.

    http://i1162.photobucket.com/albums/q539/PatrickHoban/e072a0d8.png

    The entries in HKU\.Default\Printers\DevModePerUser have not come back

    For reference I have 24 users on one server right now & the spooler is using 96,456KB of RAM. I installed a printer in about 10 seconds (hey better than I thought). I'll try to check towards the end of the day to see what it gets up to.

    I do have a email case open with MS about this. As it stands right now they are (of course) blaming the UPD drive. I was on PCL6 v5.2 but am in the process of upgrading to PCL6 v5.5.0 per their recommendation.


    Patrick Hoban
    http://patrickhoban.wordpress.com

    Friday, September 07, 2012 4:19 PM
  • I thought I replied to this post.

    David, what's the name of the person who you are working with in support?   

    I'll poke at my HP contact when he returns next week and find out if he knows if any recent changes with the UPD are around TS scenarios.  I don't think he likes to work with that driver but he gets dragged in on the driver debugging/development from time to time.

    I specifially utilize only the print drivers the vendors include with the product.  When they have coding errors, they can get fixed with the version of Windows they are included with.   Any error in print drivers that are not included with Windows blocks testing of the other parts of the spooler and there is not a path for code change in the OS for these drivers.  There is a team that verifies and reports faulting vendor drivers (for many different devices).  It is up to the vendor to determine how they deal with the code changes required.


    Alan Morris Windows Printing Team

    Friday, September 07, 2012 5:05 PM
    Answerer
  • Alan/Patrick,

    Hi, sorry, went away on leave for a while so didn’t respond.

    Alan – our case was REG:112081612189183.

    For someone who has been dealing with MS support for about 20 years, this was the most disappointing of any support call I’ve ever had raised – to have the call shut down, without a perfmon trace, process explorer analysis, or hang dump analysis (of spooler) – and simply blame “3<sup>rd</sup> party drivers” without any proof – is utterly deplorable.

    Anyway, for anyone’s benefit who has similar problems – trying to do direct printing from RDS – I’ve managed to get a solution working. 

    Here are the details;

    • A nightly print spooler clean-up script that;
      • Stops the spooler
      • Deletes the entire key under “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider”
      • Re-creates the key (empty) and sets the value "RemovePrintersAtLogoff"=dword:00000000
        • Note, this was vital.  MS support had recommended we set this to 1, along with some other keys (InactiveGuidPrinterAge,  InactiveGuidPrinterTrim) with specific values.  If we used these MS support recommended values, our RDS serer would not enumerate printers for more than 3-4 hours before requiring a restart of the spooler.
      • Restart the spooler
      • Map a printer (just to make sure it works)
    • Clean up the USERS\.DEFAULT\Printers key on all existing servers
      • There was heaps of crap here, the default user NTUSER.DAT was over 800MB in size
    • Modify the security on the registry, using GPO to deny SYSTEM write access as below, to stop the crap writing here again;
      • USERS\.DEFAULT\Printers
        • Deny Set value
        • Deny Create Subkey
    • Run NGREGOPT on all servers to compress the DEFAULT and SOFTWARE hives back down.
      • Even though we had deleted the crap from “Client Side Rendering Print Provider” and the DEFAULT user hive, the registry files were still large of course, and needed to be compressed to reduce paged pool usage.
      • Note, make sure no users are on the server when this is run !

    With the nightly spooler ‘refresh’ and the registry security changes, we are no longer seeing any problems.  In addition the paged pool has gone down from 5GB to 1GB – which I believe was related to the registry bloat that had occurred previously.  Cleaning up the keys and using NGREGOPT has fixed this.

    In addition, I am running a spooler check script every 30 minutes on each of the 13 servers.  This script checks how long it takes to enumerate the printers for the specific test user.  If it takes more than 20 seconds, we get an alert.

    Since I have made the changes above, we no longer have any printing problems… touch wood.. even using HPD 5.4 for most printers, and other (RICHO) 3<sup>rd</sup> party drivers.

    If anyone wants the scripts (the spooler refresh or the check script) let me know on david.frith<at>glfconsulting.com.au

    ta

    Thursday, October 04, 2012 5:31 AM
  • Has anyone tried this hotfix?

    http://support.microsoft.com/kb/2778831

    Wednesday, January 30, 2013 11:20 AM
  • Dale - wow, that sounds like the hotfix we are after

    about time Microsoft !

    Gee, makes me even more angry that our support call was closed on us by Microsoft blaming 3rd party drivers ...

    It will be on our farm within 2 weeks, and I'll try removing the nightly registry clean up - but looking at the article, it sounds 99% sure to solve the problem

    Wednesday, January 30, 2013 11:41 AM
  • I've just put in on a farm of 3 RDS Servers.

    We were getting the issue when using the name \\server\share didn't work.

    It would also sit on downloading driver for ages.

    Users are just logging back in. See how it goes.

    Dale

    Wednesday, January 30, 2013 12:04 PM
  • Looking good so far. Although I did also delete the client rendering folder manually and let it recreate it on reboot.

    Guess time will tell. :)

    Let us know how you get on.

    Wednesday, January 30, 2013 12:09 PM
  • Hmmm Users rung up this morning saying printers not appearing in their list.

    Thursday, January 31, 2013 9:28 AM
  • Getting this error when trying to map manually

    Thursday, January 31, 2013 9:39 AM
  • To get it working again I

    1) Stopped the print spooler

    2) renamed the folders in C:\WINDOWS\system32\spool\prtprocs\

    3) Started the print spooler.



    Thursday, January 31, 2013 10:19 AM
  • Hi guys,

    Sorry to resurrect an older thread but just wondering if anyone had 100% confirmed that the HP UPD will always use client side rendering? We print a lot of heavy graphics via Citrix and CSR kills the CPU. We had purchased a beefy print server, loaded up the HP UPD and turned off CSR on the queues but we also noticed that the jobs always arrive in RAW format. This is very frustrating.


    • Edited by shocko Wednesday, October 16, 2013 10:17 PM
    Wednesday, October 16, 2013 3:04 PM
  • I did find another reference that said UPD will always use CSR. I cannot for the life of me find the link. I'll post if I do.


    Patrick Hoban
    http://patrickhoban.wordpress.com

    Wednesday, October 16, 2013 8:40 PM
  • Thanks Patrick
    Wednesday, October 16, 2013 10:17 PM
  • Has anyone else tried the 2778831? Results?

    Dale - did this seem to resolve your printing and printing related issues in RDS environment?  Is it automatically cleaning up the GUIDs?

    Any updates on this situation are appreciated.

    Thanks!

    Thursday, January 16, 2014 8:04 PM
  • I found these registry entries to clean up this area every 15 minutes:


    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Print\Providers\Client Side Rendering Print Provider]
    "InactiveGuidPrinterAge"=dword:00000384
    "ActiveGuidPrinterAge"=dword:00000384
    "InactiveGuidPrinterTrim"=dword:00000384
    "RemovePrintersAtLogoff"=dword:00000001


    Anyone know about a 45 second delay in adding the first printer after the spooler service restarts?  (EDIT:  print spooler service listens on high random ports, and they were blocked by accident)


    Here's an old blog post about Client-Side Rendering:  https://blogs.technet.microsoft.com/askperf/2008/02/10/ws2008-client-side-rendering/




    • Edited by JS2010 Monday, May 21, 2018 12:28 PM
    Tuesday, May 01, 2018 8:03 PM