locked
File Explorer File Enumeration Slowdown in Windows 8.1 - Why? RRS feed

  • Question

  • File enumeration in Windows 8.1 appears to be slowing down.

    Here's what I mean:

    • On a freshly rebooted system (i.e., with few files in the cache), open a File Explorer window on the root folder of drive C:.
    • Select all the files, then right-click and choose Properties.
    • TIME how long it takes to finish, and divide the number of files enumerated by the seconds it took to do the enumeration.
    • Do it again now that the data is all cached.

    I've always thought this was a decent way to gauge real world file system responsiveness, and in fact it really was a pretty good benchmark up until Windows 8.1 came along.  With Windows 8(.0) and earlier, if you saw more than 30,000 files per second counted up, you had a very good SSD setup.

    On the workstation on which I'm typing this, my Windows 7 x64 installation would enumerate about 30,000 files per second the first time, and about 50,000 files per second the second time, from the RAM cache.  I have a very responsive I/O subsystem.

    • When Windows 8 came out the numbers were about the same, if not a little better.
    • When Win 8.1 was released they dropped BIG TIME.  File enumeration was something like 3 to 1 slower.
    • Now, with the updates in past months, they appear to have dropped even further!

    Right now, with a fresh bootup I am seeing just under 6,000 files per second enumerated the first time, and a bit over 8,000 from the cached data.  This is DISMAL by comparison to earlier versions.

    Thankfully, typical file system performance - as measured by how long it takes to do real work on a few files in real applications - doesn't seem to have been affected as much, but then most applications don't enumerate lots of files.  It seems likely that this enumeration slowdown affects real operations in Explorer beyond just this file count.

    Keep in mind I can run a Windows 7 VM or Windows 8.0 VM on this same hardware and still see 30,000 files/second enumerated inside the VM.  My hardware hasn't degraded.  The older Windows systems are still just as fast.  If I run a Win 8.1 VM the 6 to 1 slowdown is obvious.

    That even the RAM-cached directory data can't be run through nearly as quickly in Win 8.1 says that the slowdown is in the file system implementation.

    This isn't a question about why MY workstation is slowing down - it's not.  It's about why WINDOWS is becoming less and less efficient out of the box.  I have measured and compared progressive slowdowns in virtual machine snapshots ranging from a fresh, out-of-box installation of Win 8 or 8.1, to one that's been run for years.  Something's becoming less efficient IN THE DESIGN.

    So...

    Why is File Explorer taking longer and longer to simply count up files in folders?

    A corollary question is:  What else is this slowdown affecting?  Explorer doesn't seem as responsive to me as it once did.

    I'd hate to think Microsoft is slipping stuff into Windows to cause it to be slower and slower since its release date, just so they can make it look like the next version is faster.

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Saturday, June 7, 2014 9:20 PM

All replies

  • A day has gone by, and apparently nearly no one even cares whether their Windows I/O subsystem is slowed down arbitrarily by Microsoft.

    It's sad, really.

    People rarely pay attention to performance, and most have come to expect Windows to "slow down" over time anyway, so why not just meet their expectations, eh Microsoft?  It will help when the next release comes out and it seems so much faster.

    I have an idea:  Avoid a lawsuit and make the file system run as fast as it should again.  And fire whomever it is that forces engineers to make it work worse on purpose.

    Thanks.

      

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Sunday, June 8, 2014 7:39 PM
  • I've measured it on many systems.  I don't think it's anything particular to my systems.

    I can only guess that on an HDD-equipped system the difference must be difficult to notice, and not that many people have SSD setups yet.

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Tuesday, June 10, 2014 3:23 AM
  • Still looking for any explanation for this, and ideally a secret "MakeEnumerationFaster" registry key to tweak to circumvent this horrendous slowdown.

    Just deleting (to the recycle bin) a set of a few hundred folders with a few thousand files takes a SIGNIFICANT amount of time now.  This on a high end system equipped with an SSD array.

    Why is the file system so slow to enumerate files now?

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Sunday, July 6, 2014 4:09 PM
  • Definitely you can use Sysinternals tools to do diagnostics. However this is very tedious task and majority of people would not care if the response is decreasing from 1 to 3 second for operation that you have described.

    I remember complaints on previous operating systems when some people said that DOS programs were faster than Windows ones. This phenomenon is accompanying operatings systems. Adding some new or "improved" functionalities diminish response of other ones. Looks like separate developers groups are not cooperating...

    I would suggest to contact people in operating system staff and ask. (Mark Russinovich is not in this team, it is in Azure one, but he may give you an answer.)

    Regards

    Milos

    Sunday, July 6, 2014 4:30 PM
  • Why is the file system so slow to enumerate files now?

    Have you tried changing the number of Properties you specify for the Details view? 

    FWIW repeating the test was unaffected by caching.  Counting the slowest, biggest drive was slowest.  But I don't know if it was ridiculously slow.  It did seem ridiculous that it couldn't seem to remember anything about a previous execution.



    Robert Aldwinckle
    ---

    Sunday, July 6, 2014 4:49 PM
  • Thanks guys.  I was beginning to wonder if anyone cares about actual performance.  ;-)

    I'm not sure how to contact operating system staff directly; Microsoft does a pretty good job of keeping its distance from users.  I was hoping maybe some of them might see my post here.

    Robert, once you have done the initial count, assuming you have RAM enough to cache everything, the second enumeration should really blaze through, and in fact that was easy to see in Win 7 and 8.  That it no longer does this leads me to believe that something fundamental has changed in Win 8.1.  It sure would be nice to know what/why.

    Roughly what kinds of counts / second do you actually see?

    It's crossed my mind that Microsoft may have added delays to the code so that users might have a chance to cancel inadvertently started operations - i.e. that laypeople might not realize that modern computers can do tens of thousands of things per second and thus not realize they've done dangerous things.  If that's the case, I'd sure love to know how to turn off the delays, as some of us in the course of doing real work really do need to be able to manipulate large trees of files with ease and quickness.

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Sunday, July 6, 2014 7:49 PM
  • By the way, based on my test just now a grepWin search of the entire drive for a nonexistent filename yields a performance figure close to what I posted for Windows 7 and 8 - roughly 30,000 files / second enumerated the first time, 40,000 files / second with the data cached.

    This tends to imply the slowdown is in File Explorer, rather than the file system itself.

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Sunday, July 6, 2014 7:57 PM
  •  the slowdown is in File Explorer, rather than the file system itself.

    Want some more proof?  And proof that your system is caching the data that is needed.  And proof that stuff may be being ignored by WE which should be counted...


    In an elevated cmd window at the desired drive root enter:

    D:\>dir/a/s | findstr "File(s) Dir(s)"

    The surprise for me is the way the pipe seems to work?   Is it really a separate process that doesn't have to wait for the completion of the first stage's output?

    Anyway, I might be willing to time that--in fact, it may be possible to automate its timing--but I am definitely not interested in hanging around waiting for the number of minutes that WE takes to be sure I get an accurate measurement--if you know what I mean.   <eg>

    And apparently that would not be a fair comparison anyway.  WE only finds 2/3 in Size of what the dir finds.  I probably need to tweak WE's Options.  Finding only 2/3 of what another procedure is finding seems suspect even if it does take an absurd amount of time longer.  The funny thing is that the number of files is almost the same.  So, what does that imply: that 1/3 of the Size was contained by some very few which could not be counted?

    I should probably also try to convert that pipeline to Powershell and see how it does.  That could at least check the dir measurements in one way or another.  TBD.

    BTW I think I am now on a faster processor than I had with my tx2 but this Surface 2 Pro even with 8 GB RAM still doesn't "scream" for me.  I have noticed that its firmware shuts down cores occasionally.  My guess is that could be due to its lack of cooling.  My tx2 always had its fans running full.  Maybe I will get a different result if I do this test in the winter than now in the middle of summer?   <eg>



    Robert Aldwinckle
    ---

    Monday, July 7, 2014 5:10 AM
  • So, just to be unambiguous, I guess you're confirming that you find Windows 8.1's File Explorer slow as well.

    You're right, the slowdown doesn't seem to be in the file system itself.  DIR enumerates nearly 30,000 files per second... 

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options

    Monday, July 7, 2014 5:54 PM
  • More info...

    On an SSD enthusiast forum I've gotten some feedback as well.  It seems an up to 30% slowdown (vs. 8 and earlier) is being seen by everyone.  But not an 80% slowdown.

    However, an important clue:  The folks who tested there are using the Intel RST storage driver, which introduces low level caching that's integrated with the file system caching.

    I wonder whether Microsoft has deliberately changed File Explorer to NOT use the file system cache, which would explain a slowdown on systems that do not use Intel RST (I don't).  Intel RST-equipped systems by contrast, given that they employ low level caching, would still be satisfying Explorer's requests from the cache in RAM.

     

    -Noel


    Detailed how-to in my eBooks:  

    Configure The Windows 7 "To Work" Options
    Configure The Windows 8 "To Work" Options


    Monday, July 7, 2014 7:08 PM