none
Backup fails for a Hyper-V guest with VSS Writer failures using DPM 2012 R2 - Hyper-V guest has Oracle application installed, RRS feed

  • Question

  • I am trying to backup a VM which before it had Oracle installed backed up with no problems.  It is a Windows 2008 R2 server which sites on an 8 node Hyper-V 2012 R2 cluster (with CSVs).  I am using DPM 2012 R2 to run the backups which have been successful for the last few weeks but then Oracle was installed and the backups have failed since then.

    The job fails and a large number of VSS writers go into a 'Failed' state with the 'Last Error' showing as 'Timed out'.  I then get 4 popups appear each referring to a 'temporary' drive which appears briefly in Disk Management with a RAW file system.  These pops say "You need to format the disk in drive X: before you can use it.  Do you want to format it?" (where X: is replaced by the drive letter assigned to each 'temporary' drive).

    The System event log is populated with a large number of warnings with Event 51, Disk, stating "An error was detected on device \Device\Harddisk<number>\DR<number> during a paging operation".  There are also a few warnings for Ntfs (eventid 57) stating that "The system failed to flush data to the transaction log.  Corruption may occur."

    Prior to these warnings there are 5 other warnings for partmgr (eventid 58) stating "The disk signature of disk <number x> is equal to the disk signatuire of disk <number y>" and 4 errors (eventid 1, VDS Basic Provider), "Unexpected failure.  Error code: 490@01010004".

    There is a script which is run to stop the Oracle application on the server and if this is run then the backups will complete successfully.  We have been troubleshooting this by running a certain amount of the script and seeing which part affects the backup and it seems that if the Weblogic (wls_reports) service is stopped then the backup will succeed but if it is running then the backup will fail and the above symptoms occur.

    Another point which may help is that there is a pre-production server which resides on a Windows 2008 R2 Hyper-V standalone server, has the same scripts and installation of Oracle but backs up without any issues.

    I have experienced VSS writer failures before with VM backups but I have not seen this before.  It is not intermittent and I can find no work around to alleviate the problem of having no backup (except stopping this service or shutting the server down, but as it is a production server this is not practical).

    If anyone has experienced this before or have any suggestions / advice it would be much appreciated.

    Thanks

    Chris

    Thursday, March 12, 2015 4:50 PM

All replies

  • Hi Chris

    I have exactly the same issue.

    2008 R2 Server running Oracle

    6 node 2012 R2 Hyper-V Cluster with CSV

    Exact same errors and popup "format disk" message.

    DPM2012 R2

    I also have some other VM's on the same cluster which do back up through DPM

    The only difference I can see is the server which has the issue has a legacy network adapter.

    Production server so cannot make any changes until later but will add new adapter and report back.

    Interesting about the Weblogic service, may test this also.

    Cheers

    Kev.

    Wednesday, April 8, 2015 9:37 AM
  • Not sure if this will help either of you but I was having the same "Format Disk" problem.  I found that if I disabled the "System Center Management" service the backups would succeed.  Now to figure out why.

    BTW, I'm not running Oracle, just SCOM. 


    Thursday, May 7, 2015 3:51 PM