none
Event Log Source: Disk Event ID: 51 An error was detected on device \Device\Harddisk3\DR76 during a paging operation.

    Question

  • Ok, so maybe at first glance this isn't a DPM issue. On the other, it looks like it is occuring during DPM backup cycles.

    There doesn't seem to be any good explanations to figure out which drive these errors are referring to. Its happening on a DPM machine with a 4 drive RAID 10 array and a 4 drive RAID 5 storage pool array. Customer never connects a USB drive.

    It seems to be coincidental with Source: volsnap Event ID: 66 (The reverting of volume \\?\Volume{GUID} to the state of a previous shadow copy is complete.) and Source: FiltrManager Event ID 3 (Filter Manager failed to attach to volume '\Device\HarddiskVolume121'.  This volume will be unavailable for filtering until a reboot.  The final status was 0xc03a001c.)

    I did notice some VHDs being mounted and dismounted in Disk Management (light blue icon) around the time of the Event. This is likely a WBAdmin System State Backup initialited by DPM.

    1. What is the best way of determining the actual hard drive in question for \Device\Harddisk3\DRxx and Device\HarddiskVolumexx?

    2. Is there anyway Microsoft can get these messages changed to report additional info such as the drive label, letter, size, registry keys, etc?

    I also reviewed KB982210 and tried to install the hot fix but the message said it was not applicable to this system. There were indeed many entries in the registry, especially in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\DeviceClasses and the SYSTEM hive had hit 27MB.

    This machine originally did go through some reinstalls of DPM including a domain name change. Perhaps they could be orphaned from back then?

    I assume this means that KB982210 has already been installed with a standard DPM or Windows update.

    I did not run the devnodeclean.x64 (have an aversion to running 3rd party tools on customers' systems).

    Any tips to start us off?

    Thanks,

    Bob.

    Thursday, February 16, 2012 12:34 AM

Answers

  • Hi

    The only information I have for the error 0xc03a001c = "The requested operation cannot be performed on the virtual disk in its current state".   I see that this was only reported once a very long time ago and it only occured for Guests running Windows 2003.  If the backups were done in a saved state, then the erors went away.  

    Unfortunatly,  I don't have anything more on those events, but they all deal will VSS.   They will need to be investigated by the Windows team, so you might want to post a question in the Windows forum to see if they can help you further.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    • Marked as answer by BobH2 Tuesday, February 28, 2012 8:15 PM
    Tuesday, February 28, 2012 4:14 PM
    Moderator

All replies

  • Hi,

    Is the DPM server installed on a system that has hyper-V running on it ?


    Just for grins, try installing this hotifx, it contains the newest version of the same binary (Vhdmp.sys) that was included in the other fix that will not install.

    2662288 A differencing VHD file that is created by using Microsoft iSCSI Software Target 3.3 becomes corrupted when you mount or attach it in Windows Server 2008 R2 or in Windows 7
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2662288


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Saturday, February 18, 2012 12:13 AM
    Moderator
  • Mike,

    Installed the hotfix. We'll keep an eye on it for a few days and see if it helps.

    Yes, DPM and Hyper-V.

    Thanks Mike.

    Bob.


    • Edited by BobH2 Monday, February 20, 2012 1:20 AM
    Monday, February 20, 2012 1:19 AM
  • Mike,
    I didn't notice that the machine had to be rebooted after installing 2662288. This I finally did on Saturday when everyone was off.

    It did not fix the problem.

    What I am seeing are a series of:

    1. volsnap Event ID 64 Volume "\\?\Volume{GUID} is being reverted to the state of a previous shadow copy."
    2. Filter Manager Event ID 3 "Filter Manager failed to attach to volume '\Device\HarddiskVolume48'.  This volume will be unavailable for filtering until a reboot.  The final status was 0xc03a001c."
    3. Disk Event ID 51 "An error was detected on device \Device\Harddisk3\DR11 during a paging operation."

    Sometimes the volsnap events are not there.
    Sometimes the Disk Events are not there.

    Some of the Flter manager disk names are in the format "\Device\Harddisk3\DR19" and some are in the format "\Device\HarddiskVolume56".

    This problem also seems to be related to http://social.technet.microsoft.com/Forums/en-US/windowsbackup/thread/2a434be6-9b61-4f7d-a74b-b7707d30c655 except it is initiated by DPM when the VM's are getting backed up.

    Bob.


    • Edited by BobH2 Sunday, February 26, 2012 6:38 PM
    Sunday, February 26, 2012 6:38 PM
  • Besides being nag messages, are they causing any problems, I suspect they are benign.  

    Are your Guests causing these events running Windows 2003 ?

    The Volsnap 64 and 66 are informational, we mound the GUEST VHD's and revert the snapshots and those are normal.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, February 27, 2012 6:11 PM
    Moderator
  • Mike,

    I guess that is part of the problem, we don't really know what the messages mean or if they are important. 66 and 64 are informational - no problem.

    Warning Event 51 "An error was detected on device \Device\Harddisk3\DR27 during a paging operation." is not clear. What should we look for to know if it is a problem? Which disk is this? 

    Error Event 3 "Filter Manager failed to attach to volume '\Device\HarddiskVolume66'.  This volume will be unavailable for filtering until a reboot.  The final status was 0xc03a001c.".  This sounds more ominous. I'm not sure I understand "filtering". It appears to be a way for software modules to trap into system calls like antivirus or VSS. What would be impacted?

    There are other posts that seem to relate this to KB982210 from 2010. My customers' SYSTEM hives range from about 17MB to 27MB and some seem to have a lot of entries in HKLM\SYSTEM\CurrentControlSet\Enum\STORAGE. Could orphaned entries have a role in this? 982210 is not installed but wouldn't the fix have worked its way into automatic updates by now?

    Sorry for all the questions, but I would like to make sure these system are completely stable and won't give us any surprises.

    Thanks,
    Bob.

    Tuesday, February 28, 2012 12:32 PM
  • Hi

    The only information I have for the error 0xc03a001c = "The requested operation cannot be performed on the virtual disk in its current state".   I see that this was only reported once a very long time ago and it only occured for Guests running Windows 2003.  If the backups were done in a saved state, then the erors went away.  

    Unfortunatly,  I don't have anything more on those events, but they all deal will VSS.   They will need to be investigated by the Windows team, so you might want to post a question in the Windows forum to see if they can help you further.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    • Marked as answer by BobH2 Tuesday, February 28, 2012 8:15 PM
    Tuesday, February 28, 2012 4:14 PM
    Moderator
  • Mike,

    I'm seeing more errors now including VDS Basic Provider Event ID 1 "Unexpected failure. Error Code 490@01010004" (are they ever EXpected?).

    I agree. Will move on to the VSS forums.

    Thanks,
    Bob.

    Tuesday, February 28, 2012 8:15 PM
  • Hi Bob,

    I'm not sure where you are seeing those VDS errors, but I only found when I queried for it.

    979391 VDS Basic Provider event ID 1 is logged on a Hyper-V guest operating system
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;979391


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, February 28, 2012 8:31 PM
    Moderator
  • Mike,

    They are inside the Virtual Machines in the System log. Actually they have been happening all the time and are supposed to be benign. I've just got so many events that i am trying to follow through I got mixed up for a moment. I'm trying to catalog them and tie them together if I can. For example:

    2012-02-28 06:26:25
    Service Control Manager Event ID 7034
    The DPMRA service terminated unexpectedly.  It has done this 1 time(s).

    2012-02-28 06:26:19
    Application error Event ID: 1000
    Faulting application name: DPMRA.exe, version: 3.0.7707.0, time stamp: 0x4d24b811
    Faulting module name: KERNELBASE.dll, version: 6.1.7601.17651, time stamp: 0x4e21213c
    Exception code: 0x809909c3
    Fault offset: 0x000000000000cacd
    Faulting process id: 0x928
    Faulting application start time: 0x01ccf5a388ea622b
    Faulting application path: C:\Program Files\Microsoft DPM\DPM\bin\DPMRA.exe
    Faulting module path: C:\Windows\system32\KERNELBASE.dll
    Report Id: 08bf2869-61ff-11e1-b304-0025903d5521

    2012-02-28 06:24:49
    Service Control Manager Event ID 7034
    The Volume Shadow Copy service terminated unexpectedly.  It has done this 1 time(s).

    2012-02-28 06:20:49
    VSS Event ID 24
    Volume Shadow Copy Service Warning: The Volume Shadow Copy Service is shutting down and is experiencing delay 
    while waiting for in-progress calls to complete. 

    Then also:

    2012-02-28 08:07:03
    VHDMP Event ID: 3
    Failed to surface VHD \Device\HarddiskVolumeShadowCopy1078\XXXX\XXXX-C-Drive.vhd. Error status 0xC0000061.

    Stuff's going on but I'm not sure what yet. I think I have the anti-virus exclusions right and the antivirus scans are between 1AM and 3AM.

    Still hunting through though...

    Bob.

    Tuesday, February 28, 2012 10:59 PM