Kernel Paged Pool Memory Use With FlexRAID, Large Files & MmSt Tag RRS feed

  • Question

  • I have a media file server running Win 7 Pro 64-bit with 8GB of RAM.  I'm using a software RAID solution called FlexRAID to run an array of 8x 2TB data drives, and 3x 2TB parity drives.  FlexRAID contains a data integrity check function called "Validate" that reads each data file and computes its checksum, then compares the checksum vs. a stored value to see if there is any change.

    When performing this Validate task, FlexRAID reads from all 8 drives simultaneously.  My data files are optical disc backups, stored as Matroska (.mkv) files, each approx. 25-35GB in size.  When reading the disks, my I/O throughput averages 1100-1300 MB/s.

    Unfortunately, the Validate task cannot complete because of what looks like a kernel paged pool memory leak.  The cause of the leak is unknown.  I am running Process Explorer and no processes are showing abnormally high memory use or handles count.  Using RAMMap, I see that the system is mapping the .mkv files into paged pool memory, but not releasing it.  Poolmon shows the MmSt tag as the cause, but this is Windows' own memory manager, and the page file is being managed by Windows.

    My kernel paged virtual use get to ridiculously high levels (40GB+), and the system becomes unstable.  I get "low virtual memory" messages from the system.  For an unknown reason, disk activity eventually stops and the Validate task doesn't complete.

    I have tried the following troubleshooting steps:

    • Re-installing FlexRAID
    • Re-installing Windows
    • Un-installing Microsoft Security Essentials
    • Running a bare-bones system with only required drivers installed
    • Running FlexRAID in Safe mode (won't run)
    • Scanning all drives for surface errors (none found)
    • Upgrading to faster CPU & motherboard
    • Upgrading to 32GB of RAM
    • Adjusting registry settings for PoolUsageMaximum and PagedPoolSize, per KB312362
    • Installed Haali Media Splitter, to allow .mkv thumbnail generation

    Since the leak is tied to MmSt, I have to conclude it's a possible bug in how Windows manages heavy I/O with large files.  Is there a way to dig deeper into MmSt, to see the true cause?  Or, would a kernel dump help analyze the problem?  I'm not really an IT pro, just a consumer with a bit of tech knowledge, so I would need help to debug the kernel dump.

    EDIT: Forgot to mention, the leak occurs immediately upon starting the Validate task, and is very fast -- it only takes a few hours to reach such high levels (not days).

    • Edited by Matsutake1 Saturday, May 4, 2013 3:58 PM
    Saturday, May 4, 2013 3:51 PM