Sign in
 
HomeWindows Server 2012Windows Server 2008 R2Windows Server 2003LibraryForums
Windows Server TechCenter >
> 
> 
Windows 2008 R2 - large file copy uses all available memory and then tranfer rate decreases dramatically (20x)

Answered Windows 2008 R2 - large file copy uses all available memory and then tranfer rate decreases dramatically (20x)

  • Wednesday, March 20, 2013 11:59 PM
     
     
    Sign In to Vote
    0

    http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/3f8a80fd-914b-4fe7-8c93-b06787b03662

    I experienced the problems described in the above link, but needed a solution...

    I have recompiled DynCache on 2008R2 (WHS2011), followed the instructions with the original DynCache and replaced the .exe.  The performance isn't great (20-30MBytes/s), but it doesn't drop connection like it used to.

    Is this of use to anyone??

     

All Replies

  • Thursday, March 21, 2013 1:48 AM
     
     
    Sign In to Vote
    0

    Hey there DynCache...looks like you've done your homework on this but I must ask just to make sure.  Are you transferring over LAN?  As you know when the OS begins to control the hard disks and begin transferring data it first loads this data as cache in RAM because of its higher speed compared to the HDDs.  I know with linux you can control buffer sizes and cycle numbers which can affect the amount of RAM used during a file transfer but I have to admit ignorance with Windows Server.  Your transfer speed however is much more complicated to explain because there can be so many factors that go into play, especially when you are transferring data over a LAN.  If you think of it this way

    SATA @ 6.0Gbit/s   *** it takes 8bits to make a byte  *** 6/8 = 0.75GB/s or aprox 750MB/s  ***That's fast but...

    Our HDDs have a cache built into them and operate at a certain rpm...the larger the cache and the higher the rpm size the closer you can get to this ideal mark of 750MB/s

    Alot like data moving over ethernet data moving from HDD to RAM to another HDD needs to be checked for consistency...if you've read into the difference between udp and tcp as far as network traffic is concerned this is KINDA the case here as well and this slows things down some more.  If you notice sometimes when you transfer or copy some files you might see windows report that it will take over an hour to complete a transfer but in reality it usually only takes 10min ;) 

    I think your speeds are ok...not great like you said but your memory usage is crazy.  I would start troubleshooting with the drivers, firmware, controller and of course the OS.  Try uninstalling and reinstalling your service pack/download the latest one first, then start back on drivers controller and firmware.  Sorry I don't really have the answer for you my friend.

     
  • Thursday, March 21, 2013 8:18 AM
     
     Answered
    Sign In to Vote
    0

    Hi rootproxi, thanks for the reply.  This issue occurs whether the transfer is local or over the network (from a 7200rpm source to 5400 rpm destination), only if I am transferring between from a source that is faster than the destination though.  Before installing the DynCache service all network connections to the server would be lost at the point all the physical ram on the server was used.

    At first I thought it was a problem with a particular disk (a recert-Seagate 2Tb 5400rpm), so I bought a new 4Tb 7200rpm Hitachi and the problem seemed to go away......until I started transferring files to another 5400rpm drive (a Samsung 2Tb this time).  Then I found all these other reports and couldn't believe this sort of bug could exist in a server O/S!  If it wasn't for the hassle I'd be straight back to Linux!  I tried all the other working set "fixes", but nothing helped until I tried the DynCache service after wrestling with Visual Studio and removing the OS version checks.

    Anyway, if I can help anyone else out there that has this issue and needs a stable system, just reply to this post.  I'd put together a how-to, but there was a lot of trial and error and WDK and SDK issues as well as never having used Visual Studio before!

    Thanks.