none
System Drive filling up on DPM server, hidden files RRS feed

  • Question

  • Free space on the C:\ keeps disappearing but cannot find the culprit.

    I'm running DPM 2007 (v. 2.0.8844.0) using the included SQL instance, on Windows Storage Server 2003 SP2.

    DPM is installed on the C:\ drive which is a 50 GB partition.

    Directory statistics (WinDirStat) on the C:\ drive show Windows, Program Files, page file (fixed size), SQL, etc. using 16 GB of space (including hidden files) and 1.5 GB of free space, leaving 32 GB of 'unknown' disk usage.  What is utilizing this 32 GB of disk and how can I recover at least some of it?

    I am using short term storage to disk and long term storage to tape.

    The hardware requirements note: "Additionally, before archiving data to tape, DPM copies the file catalog to a DPM temporary installation location; therefore, we recommend that the volume on which DPM is installed contains 2–3 GB of free space."  Could this space be orphaned temporary file catalogs, or should these be in the C:\Program Files\Microsoft DPM\DPM\Volumes\ folders?

    I've been chasing this lack of free space for several weeks, removing what I know I can safely delete from the server, getting it back up to about 5 GB free a couple times, but it quickly evaporates.

    Are there any cleanup operations that I should be running routinely to clear DPM temporary files or recover free space.  I've run a 'Shrink Database' task on the DPM database, but that had little effect.

    Any help in identifying what is using this hidden 32 GB of disk space and how to safely free it up would be greatly appreciated.

     

    Monday, September 26, 2011 9:20 PM

Answers

  • Hi,

     

    OK, that's good.  So to see where the "mystery files" are, you can leverage NTBackup.exe and use it as a magnifying glass to see what folders are taking up all the disk space.   Run ntbackup, and select the c: drive, then uncheck the C:\Program Files\Microsoft DPM\DPM\Volumes folder (since those will all be mountpoints and don't consume space on C:) - then select a network share as desination.   Let DPM calculate how much data it will backup, that should give you accurate count of files and size to be backed up which should include the 32GB of mystery files.  You can let the backup run and look at the backup log for the list of files backed up, or just poke around and let ntbackup calculated data size for subset of folders you selected.    You can compare DIR /S /A for the same subset of folders and see what difference there is.   DIR /S /A will only count files you have access to, but ntbackup will count ALL files since it has the backup rights to all files.   Good luck.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by Scooter79 Wednesday, September 28, 2011 5:57 PM
    Tuesday, September 27, 2011 4:40 PM
    Moderator

All replies

  • Hi,

    You can safely delete all the *.errlog files located in C:\Program Files\Microsoft DPM\DPM folder.    Also check to see if there are any dangling files under C:\Program Files\Microsoft DPM\DPM\Temp\MTA folder.

    Also check out this old article, still applies to Windows 2003.

    303079 How to locate and correct disk space problems on NTFS volumes
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;303079


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, September 26, 2011 10:48 PM
    Moderator
  • Mike,

    Thank you for the info on the *.errlog files.  I was unsure about these.  I deleted all but the *curr.errlog and regained about 400 MB.

    The 303079 article was also helpful and may have given me some insight on where the 'hidden' 32 GB of disk is being used.  I ran a DIR /A /S on the volume and it happened to be while a replica was being created.  The DIR cmd was scrolling all the files from the volume being replicated as files within C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System\D-de4ca400-a448-11dc-8214-806e6f6e6963\ mounted volume.  So I assume that the 32 GB of disk space has been reserved by DPM for temporary replica creation, otherwise there would be no space for these files.  So my next question is, how do I control it's size from continuing to grow and how can I reduce it's current size back down so I have some working free space on this volume?

     

    Tuesday, September 27, 2011 1:59 PM
  • Hi,

    The path  C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System\D-de4ca400-a448-11dc-8214-806e6f6e6963 is mountpoint and although it is using the C: namespace, no data is stored physically on C: (UNLESS) the mountpoint is broken, which I suspect is your problem.

    If you do a DIR from a command prompt from the directory C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System - does the directory D-de4ca400-a448-11dc-8214-806e6f6e6963 show as <junction> - or is it a normal folder <DIR> ?

    If the latter, then you will need to perfrom ONE of the following:

    A) Stop protection of that datasource and delete the replica, Delete all the files and folders under the directory D-de4ca400-a448-11dc-8214-806e6f6e6963 so it's an empty folder, then reprotect the datasource.

    B) Delete all the files and folders under the directory D-de4ca400-a448-11dc-8214-806e6f6e6963 so it's an empty folder, then run DPMSYNC -SYNC so DPM recreates the mountpoint.  (This will cause all replicas to become inconsistent and you need to run CC to get back in the green.

    C) Delete all the files and folders under the directory D-de4ca400-a448-11dc-8214-806e6f6e6963 so it's an empty folder, then locate the GUID for the replica and manually remake the mountpoint using mountvol.   You can run mountvol and see if a volume "*** Has no mountpoints ***".

    IE:   mountvol "C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System\D-de4ca400-a448-11dc-8214-806e6f6e6963"  \\?\Volume(GUID}


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, September 27, 2011 3:37 PM
    Moderator
  • Mike,

    These mount points are correctly reporting as Junctions:

    C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System>dir
     Volume in drive C has no label.
     Volume Serial Number is 606D-5324

     Directory of C:\Program Files\Microsoft DPM\DPM\Volumes\Replica\<ServerName>\File System

    09/27/2011  11:00 AM    <DIR>          .
    09/27/2011  11:00 AM    <DIR>          ..
    09/27/2011  11:00 AM    <JUNCTION>     C-de4ca3ff-a448-11dc-8214-806e6f6e6963
    09/27/2011  11:00 AM    <JUNCTION>     D-de4ca400-a448-11dc-8214-806e6f6e6963
                   0 File(s)              0 bytes
                   4 Dir(s)   3,289,078,784 bytes free

    Tuesday, September 27, 2011 4:05 PM
  • Hi,

     

    OK, that's good.  So to see where the "mystery files" are, you can leverage NTBackup.exe and use it as a magnifying glass to see what folders are taking up all the disk space.   Run ntbackup, and select the c: drive, then uncheck the C:\Program Files\Microsoft DPM\DPM\Volumes folder (since those will all be mountpoints and don't consume space on C:) - then select a network share as desination.   Let DPM calculate how much data it will backup, that should give you accurate count of files and size to be backed up which should include the 32GB of mystery files.  You can let the backup run and look at the backup log for the list of files backed up, or just poke around and let ntbackup calculated data size for subset of folders you selected.    You can compare DIR /S /A for the same subset of folders and see what difference there is.   DIR /S /A will only count files you have access to, but ntbackup will count ALL files since it has the backup rights to all files.   Good luck.


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by Scooter79 Wednesday, September 28, 2011 5:57 PM
    Tuesday, September 27, 2011 4:40 PM
    Moderator
  • It looks like the culprit is C:\Windows\System32\$vds$.log
    Wednesday, September 28, 2011 5:09 PM
  • Interesting - see if any of these help ya.

     

    924726 The size of the $vds$.log file grows quickly when a RAW hard disk is connected to a Windows Server 2003-based computer
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;924726

    924733 The size of the $vds$.log file grows quickly after you remove a RAW file system storage device from a Windows Server 2003-based computer
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;924733

    842011 How to turn on logging for the VDS in Windows Server 2003 and in Windows Server 2008 and 2008 R2
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;842011


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, September 28, 2011 5:19 PM
    Moderator
  • Mike,

    Thanks, these articles are helpful.

    I do not have Content Indexing turned on on this server, but, I do add and remove sets of external hard drives every week that are acting as virtual tape for long term storage (using Firestreamer).  I imagine VDS is logging a lot with this activity.

    I assume I can savely delete the existing $vds$.log file.  Since Content Indexing is turned off, but I'm still getting this large VDS log file, I'm wondering what would be the negatives to turning off logging for VDS?

     

    Wednesday, September 28, 2011 6:05 PM
  • Hi,

    You can safely turn off VDS logging, I don't believe it's on by default, perhaps you enabled it to troubleshoot another issue and forgot to turn it off.  Anyhow, I'm glad the "mystery is solved".


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, September 28, 2011 6:14 PM
    Moderator