none
what causes windows 2008 r2 slow file & folder copy?

    Question

  • We use a new HP Proliant ML350 G6 running win2008 R2 as a file server (and AD/DNS/WSUS/SQL). Server was very robust for a week or so and suddenly started running VERY slow when copying files to/from the server. CLient computers are all XP (32-bit or 64-bit). Domain is 2003 level (we have 2 more AD/DNS servers of older type running 2003 SP2 32-bit).

     

    Our network is gigabit based (copper), 2 switches, 1 core router and gigabit gateway to internet cable connection.

    When I do file copy speed tests the 2008 R2 looks now very pale (half the speed of 2003 at max, usually 1/3 speed, it used to be twice as fast as 2003!).

    suddenly we started getting file transfer speeds like 10MB/s to/from our 2008 R2 (I think after most recent windows updates), before we were getting up to 70MB/s ... what is the problem?

     

    I disabled some 2008 global registry settings per RSS, chimneys, TCP off load and such thinking that maybe AD replication has something to do with (it should not) per this article (David Shen post):

    http://social.technet.microsoft.com/Forums/en-US/winserverfiles/thread/e55022a4-db65-4dc9-a2f1-96b7f5d8e2fa

    and the file copy bandwidth increased to maybe 13MB/s. (no more than 10-12% load on the NIC, 0% load on 12 cores of CPUs, only 6 of 12GB RAM assigned).

     

    Then I disabled TCP offload (both settings) on NIC driver and the speed went up to 17MB/s ...

     

    Now, I have all of above settings enabled on two other 2003 servers and each of them can easily do 27-30MB/s on file transferring ... what's up with 2008 R2?

    Tuesday, August 16, 2011 11:16 PM

Answers

  • identical problem to mine and also as noted in that link (and based on my own testing) problem occurs ONLY when 2008 R2 server is receiving the file:

    http://social.technet.microsoft.com/Forums/en-US/w7itpronetworking/thread/96fe7d15-2bde-451b-8a12-796682a6cea8

     

    file transfer speed in opposite direction is lightning fast (up to 100MB/s when pushing ro around 50MB/s when pulling).

     

    EDIT:

    found out that server's NIC used disabled Flow Control, so I enabled it, didn't change much anyways,

     

    Then I enabled the IPv6 protocol which I disabled a few days ago as it appeared unneeded on our LAN ... and bam! :)

    2008 R2 started receiving files at 24-25MB/s instead of 9-13MB/s ... ridiculous, especially that that's the only machine actually using IPv6 on our network ... it worked anyways, all back to normal now :)


    • Marked as answer by fenixus Friday, August 19, 2011 1:30 PM
    • Edited by fenixus Friday, August 19, 2011 1:30 PM
    Wednesday, August 17, 2011 7:07 PM

All replies

  • We use a new HP Proliant ML350 G6 running win2008 R2 as a file server (and AD/DNS/WSUS/SQL). Server was very robust for a week or so and suddenly started running VERY slow when copying files to/from the server.

    You have way to much stuff on this one server, plus SQL by default will take as much memory as possible, and given you problem takes a week, its most likely the fact SQL (and othe processes) have taken all your memory.  In general monitor the various processes memory space, and implement caps if possible.

    Suggested reading, most are for SQL 2005/Windows 2003, but still apply.

    http://networkadminkb.com/kb/Knowledge%20Base/SQL/Configuring%20SQL%202005%20Maximum%20Server%20Memory.aspx

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20view%20the%20Virtual%20Address%20Space%20currently%20in%20use%20by%20a%20process.aspx

    Finally, SQL/AD/WSUS/DNS all require a different memory configuration than File Services, thus if you optimize for these services, then file services will be slow, if you do the reverse then you may slow the other services.  Ideally, you want separate servers.

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20performance%20tune%20a%20Windows%202003%20Server.aspx

    I would look at installing ESXi on create separate virtual machines on this single virtual server.  If you have 2008 R2 Enterprise edition license you should be able to install 4 VM's running Windows 2008 without violating any license agreement.

     

     

    Wednesday, August 17, 2011 1:33 AM

  • We use a new HP Proliant ML350 G6 running win2008 R2 as a file server (and AD/DNS/WSUS/SQL). Server was very robust for a week or so and suddenly started running VERY slow when copying files to/from the server.

    You have way to much stuff on this one server, plus SQL by default will take as much memory as possible, and given you problem takes a week, its most likely the fact SQL (and othe processes) have taken all your memory.  In general monitor the various processes memory space, and implement caps if possible.

    Suggested reading, most are for SQL 2005/Windows 2003, but still apply.

    http://networkadminkb.com/kb/Knowledge%20Base/SQL/Configuring%20SQL%202005%20Maximum%20Server%20Memory.aspx

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20view%20the%20Virtual%20Address%20Space%20currently%20in%20use%20by%20a%20process.aspx

    Finally, SQL/AD/WSUS/DNS all require a different memory configuration than File Services, thus if you optimize for these services, then file services will be slow, if you do the reverse then you may slow the other services.  Ideally, you want separate servers.

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20performance%20tune%20a%20Windows%202003%20Server.aspx

    I would look at installing ESXi on create separate virtual machines on this single virtual server.  If you have 2008 R2 Enterprise edition license you should be able to install 4 VM's running Windows 2008 without violating any license agreement.

     

     

    thanks for a reply, I will read those articles.

     

    However I have to totally disagree with you. Our company always uses only 3 domain controllers and that always was that since times of NT, then 2000, 2003 and now having the first 2008 R2.

     

    WSUS doesn't harm much once all clients (around 50) have taken the updates, nor it eats any memory whatsoever.

    SQL servers we have are just 3 small instances (all are 2008 express R2), one for WSUS db, the other for Symantec Manager and last one for HP Insights Manager. none of them is big nor eats memory, and I actually pointed out that server has 12GB of RAM and uses only 6GB assigned. same about CPU load, it's close to zero and we have 2 6-core Xeon processors in that machine. network load at its peak is never higher than 12%, usually also sitting around 0%.

     

    As the matter of fact the very same setup was formerly running on 2003 R2 (older server, 2x Xeon single core, 4GB RAM, roughly 1.5TB HDD SATA RAID1) without ay trouble whatsoever (until server died, bad Motherboard, hence why we upgraded to a new one).

     

    I will look into optimizing services/resources for file server as I don't care if those all SQL instances run slow or fast, they are barely used anyways.

     

    My point is the actual file transfer speed of 2008 R2 when directly compared to other server (5 year old machine running 2003 32-bit, 4GB RAM, 2x Xeon single core, only 36GB SCSI RAID1 and hosting a massive legacy financial database), over the same network and using same test XP x64 workstation. 2008 R2 just seems unable to copy files fast as 2000/2003 can. what's up with that?

    we didn't spend $15k for a server giving us half of the current 2003 machine network file transfer speeds. it's just not right.

     

    Something is NOT right with 2008 R2! it started copying files slooooooow and it's not a problem with any resources. some setting somewhere is messing (throttling down) the network file transfers, but I don't know where and why.

     

    it's not normal that an over-loaded 5 year old 2003 server can copy files 2-3 times faster than a brand new freshly installed 2008 R2 with components faaaaar superior to the first one.


    EDIT:

    just double checked and it appears that TCP auto-tuning is still running as I never tried disabling it ... hmmm

    also found some DNS zone transfers misconfig - will fix those and re-test.

     

    maybe permanent disabling TCP auto-tuning, chimney and RSS makes it run better, will see.

    Wednesday, August 17, 2011 2:11 AM
  • My point is the actual file transfer speed of 2008 R2 when directly compared to other server (5 year old machine running 2003 32-bit, 4GB RAM, 2x Xeon single core, only 36GB SCSI RAID1 and hosting a massive legacy financial database), over the same network and using same test XP x64 workstation. 2008 R2 just seems unable to copy files fast as 2000/2003 can. what's up with that?

    Its called performance tuning, you (or your predecessor or some else on staff) may have tweaked those 2000/2003 servers over the years, ie: adding in small registry hacks to fix this issue or that issue.  Now all of that is gone, and you have to start from scratch, unless you documented those settings.  Also some enhancements have been made in 2008, now you have figure out how you applications work with those enhancements.  You are already tweaking 2008, turning things off and making changes to the default.  You probably did the same thing over the years to 2000/2003.

    You are welcome to disagree with me, its your environment.  I'm just giving you a best practice way of setting up things.  In my experience all your eggs in one basket is never a good idea, i think that's just good IT practices at a minimum.  If it works for you great.

    For performance issues, that show up days/weeks after system reboots, there are very rarely things you can immediately point to.  My approach is to prevent them from happening by designing a better environement and mitigating the risks across several servers not one.   To solve performance issues takes time, and research and loging performance counters, recording dates and times of when things start to go bad, then correlating that information.

    Good luck, it will be a long road.  If you find the magic bullet, great.  If not dig in and start the hard work.

    Wednesday, August 17, 2011 2:44 PM
  • hehe, no please, no tricks like that. I was the one installing all servers starting with 2000 and 2003 :)

    I didn't have to tweak 2003 for faster file transfers, that's the whole point here. they were very fast at time of installation already :)

    I had lots of other trouble with them like Time sync issues, file replication issues, some initial driver issues, but never a problem with the speed of file transfers across our LAN.

     

    Anyways, DNS (all zones) replication now works correctly between all 3 domain controllers (2008R2 and 2x 2003 SP2), I retested file transfer speeds and they got a tad better on 2008R2, but nothing dramatic.

     

    Once again background info:

    0) 2003 functional level domain, gigabit network environment (2 gigabit L2 switches, 1 gigabit L3 switch, 5 VLANs, gigabit gateway/firewall to WAN)

    1) w2k8r2 = primary DC = 2008 R2 (HP Proliant ML350 G6)
    running services: AD, DNS, WSUS, SQL (3 small 2008 R2 Express instances including WSUS), IIS7 (for management consoles like HPSIM, SEPM and WSUS), File server (6TB RAID50 - mostly large files), Print server (1 color laser printer)

    2) w2k3 = secondary DC = 2003 SP2 32-bit (HP Proliant ML350 G4p)
    running services: AD, DNS, WINS, DHCP, SQL (2 small 2005 Express instances including Payroll software), IIS6 (for management consoles like HPSIM, etc.), File server (147GB RAID10 - lots of small files), Print server (14 office laser printers)

    3) we also have another 2003 server (similar to the one above), but it does only 3 roles AD/DNS and old legacy financial database. this one is overwhelmed by the database users and shows similar networking speeds to 2008R2 (not that it is is related to it in any way).


    XPx64 to 2003 (number 2 in listing above) file transfer speeds:
    iso file, 897MB, around 30 seconds, between 24-27MB/s

    XPx64 to 2008 R2 (number 1 in listing above) file transfer speeds:

    1) TCP auto-tuning & Chimney Offload & RSS all turned ON
    iso file, 897MB, around 1 minute, between 9-15MB/s

    2) TCP auto-tuning OFF, Chimney offload OFF & RSS turned OFF
    iso file, 897MB, around 50 seconds, between 13-16MB/s

    I will try optimizing this 2008R2 for a File Server role and see what comes up later. I doubt it will change anything much.

     

    I have found DOZENS of threads on technet, techarena and other forums where users are complaining about low file transfer speeds of 2008R2, most of them are left unsolved. Not good news.

     

    Seems like Microsoft is repeating the "Vista success story" now in the segment of servers operating systems ... nice.


    Wednesday, August 17, 2011 3:12 PM
  • I have found DOZENS of threads on technet, techarena and other forums where users are complaining about low file transfer speeds of 2008R2, most of them are left unsolved. Not good news.

    Yeah, i see those also.  I have 6 dedicated 2008 R2 file servers doing nothing but serving files, and doing DFS replication, holding about 8 TB of files.   No slowness here, didn't make any changes to the install, other than following the best practices i laid out for you.  My practices work...for me at least.

    I really don't have much else to offer, this is a common issue some people have.  I have never had it, so i don't know what else i can suggest.

     

    Wednesday, August 17, 2011 3:26 PM
  • I have found DOZENS of threads on technet, techarena and other forums where users are complaining about low file transfer speeds of 2008R2, most of them are left unsolved. Not good news.

    Yeah, i see those also.  I have 6 dedicated 2008 R2 file servers doing nothing but serving files, and doing DFS replication, holding about 8 TB of files.   No slowness here, didn't make any changes to the install, other than following the best practices i laid out for you.  My practices work...for me at least.

    I really don't have much else to offer, this is a common issue some people have.  I have never had it, so i don't know what else i can suggest.

     

    ok, no probs. I will check settings here and there and keep lurking on forums for a solution.

    I am preparing to upgrade at least 1 more (now 2003) domain controller to 2008 (not R2 though, regular 32-bit will be only supported for in-place upgrade), and if necessary I will demote the last 2003 DC from AD/DNS role and raise the domain functional level to 2008, maybe this will help ...

     

    Currently we have no custom file replication between 2008R2 vs 2003 (non-R2) as they do NOT have a common protocol for that.

    2008R2 can only use DFS replication on custom files (FRS is only for AD, cannot be used on custom files) while 2003 (non-R2) can only use FRS for custom file replication (and do not support DFS at all)...

     

    I am sure the issue will go away sooner or later after some updates, either from HP or from MS. I'd like to have more severs to balance and offload some stuff, but this company is small and there is no budget for more servers... kinda sucks, but I am working with what I can ;)

     

    thanks for your help Gunner999, I am looking through the articles you linked. Some interesting stuff.

    Wednesday, August 17, 2011 3:56 PM
  • interesting.

     

    I have the Properties button grayed out on "File and Printer Sharing for Microsoft Windows Networks" on all 3 domain controllers (2003 and 2008R2), not sure why.

     

    anyways, I checked the registry settings of LargeCache and Size keys and all the 3 servers were actually configured as Maximize Throughoutput for File Sharing acording to this article:

    http://msdn.microsoft.com/en-us/library/ms191471%28v=sql.90%29.aspx

    http://technet.microsoft.com/en-us/library/cc784562%28WS.10%29.aspx

     

    ... weird, I have never set that... changed settings of one (not being a file server at all) to Maximize Networking Applications, will reboot it after hours.

     

    I also limited the memory usage of all instances of SQL. They were almost all (except one 2005Express instance on 2003) allowing to use ... up to 20TB of RAM ... nice default setting, LOL!

     

    Now, I am at a total loss still with 2008R2 network file transfers being throttled down to maximum of 14% Network Utilization, why is it doing it???

    2003 File Server allows to use up to 25% of Network utilization (hence double the file transfer speed) when copying a single large file ...



    Wednesday, August 17, 2011 5:33 PM
  • identical problem to mine and also as noted in that link (and based on my own testing) problem occurs ONLY when 2008 R2 server is receiving the file:

    http://social.technet.microsoft.com/Forums/en-US/w7itpronetworking/thread/96fe7d15-2bde-451b-8a12-796682a6cea8

     

    file transfer speed in opposite direction is lightning fast (up to 100MB/s when pushing ro around 50MB/s when pulling).

     

    EDIT:

    found out that server's NIC used disabled Flow Control, so I enabled it, didn't change much anyways,

     

    Then I enabled the IPv6 protocol which I disabled a few days ago as it appeared unneeded on our LAN ... and bam! :)

    2008 R2 started receiving files at 24-25MB/s instead of 9-13MB/s ... ridiculous, especially that that's the only machine actually using IPv6 on our network ... it worked anyways, all back to normal now :)


    • Marked as answer by fenixus Friday, August 19, 2011 1:30 PM
    • Edited by fenixus Friday, August 19, 2011 1:30 PM
    Wednesday, August 17, 2011 7:07 PM
  • For curiousity's sake, check the protocol binding order, you always want the most used protocol on top.  There is some negotiation that takes place when a client attempts a connection, but i don't know the technical details.  I do remember is 2003 this was also an issue, mostly when running IPX and IP on the same servers.

    On my 2008 servers the first protocol is IPv4, but i also know that several people on these forums have made reference that you should not disable ipv6 on windows 2008 for various reasons.

    Checking the protocol binging order is Step 10 from this guide

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20performance%20tune%20a%20Windows%202003%20Server.aspx

     


    Wednesday, August 17, 2011 8:49 PM
  • For curiousity's sake, check the protocol binding order, you always want the most used protocol on top.  There is some negotiation that takes place when a client attempts a connection, but i don't know the technical details.  I do remember is 2003 this was also an issue, mostly when running IPX and IP on the same servers.

    On my 2008 servers the first protocol is IPv4, but i also know that several people on these forums have made reference that you should not disable ipv6 on windows 2008 for various reasons.

    Checking the protocol binging order is Step 10 from this guide

    http://networkadminkb.com/kb/Knowledge%20Base/Windows2003/How%20to%20performance%20tune%20a%20Windows%202003%20Server.aspx

     


     

    thanks, I keep those links from you, opened in my browser tabs, they are pretty helpful in optimizing stuff.

     

    One stupid question though. How do I get to Protocol Binding Order in 2008 R2?

    in 2003 that was under Advanced Settings in Control Panel/Networking applet ... here there is no such button/option :) can't find that thing...

    Wednesday, August 17, 2011 9:06 PM
  •  

    thanks, I keep those links from you, opened in my browser tabs, they are pretty helpful in optimizing stuff.

    One stupid question though. How do I get to Protocol Binding Order in 2008 R2?

    in 2003 that was under Advanced Settings in Control Panel/Networking applet ... here there is no such button/option :) can't find that thing...

    While in the Control Panel\Networking press the (left) ALT key, then a menu will appear, select the Advance menu selection just like in 2003.

    You can turn on the menu permantly if from the menu you select, Tools...Folder Options...View, then select Always show menus.  Click Ok

     


    Wednesday, August 17, 2011 9:50 PM
  •  

    thanks, I keep those links from you, opened in my browser tabs, they are pretty helpful in optimizing stuff.

    One stupid question though. How do I get to Protocol Binding Order in 2008 R2?

    in 2003 that was under Advanced Settings in Control Panel/Networking applet ... here there is no such button/option :) can't find that thing...

    While in the Control Panel\Networking press the (left) ALT key, then a menu will appear, select the Advance menu selection just like in 2003.

    You can turn on the menu permantly if from the menu you select, Tools...Folder Options...View, then select Always show menus.  Click Ok

     


    thank you! :)
    Wednesday, August 17, 2011 10:24 PM
  • I checked the protocol binding order on w2k8r2 and it was set correctly as IPv4 on top and IPv6 as secondary. still, disabling IPv6 in LAN properties was a direct cause of slow file transfers...

     

    My best guess is that 2008 R2 was made kinda future proof and some core system services rely on IPv6 as a primary protocol regardless of binding order, and once that protocol has been disabled, the affected service falls back into some failsafe mode using IPv4.

     

    anyways, all is swift and stable as rock here now :)

     

    prepping to upgrade the other DC from 2003 to 2008 (32-bit) in a week or so (I need that custom files replication!).

    Thursday, August 18, 2011 3:08 PM