locked
WSB Backup Errors (using Hyper-V writer) RRS feed

  • Question

  • I have a 2008 SP2 server with Hyper-V.  All VMs reside on 3 RAID1 volumes (3 or 4 VMs per volume).  16GB RAM installed.  Backup target is a USB 2.0 1TB drive hanging off of the server.  The server itself has roughly 2 - 3GB free RAM.  Running a Core install.  Total of 11 VMs, 2 are always offline during the backups.  10 2003 SP2 x86 VMs and 1 2000 SP4 x86 VM.  The 2 offline VMs are 2003 SP2 x86.
    The target disk has a capacity of 931.49GB and there is 528.80GB free.  26 copies are on disk.

    I'm receiving these errors, sometimes, not all of the time.  For example, I had successful backups from the 13th through the 17th, but the 18th through today (the 20th) have failed.  I have had previous failures as well.  I did upgrade the 2008 server to SP2 on the evening of the 17th.  Backups occur at 2:30 AM, so the failure on the 17th was while the server was still running SP1 (but I had a hotfix or two surrounding WSB & Hyper-V backups installed at that time).  The performance options are configured for "Always perform a full backup".  For some reason the backup disk appears in this list (as "Full backup").  Here are the errors (I only copied one of each, but some errors have multiple notices in the Event Log):

    Log Name:      System
    Source:        PlugPlayManager
    Date:          7/19/2009 2:37:39 AM
    Event ID:      12
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    The device 'Storage miniport driver' (VMBUS\{7264bc2a-6caf-4730-8626-2b34210b1ba8}\1&3189fc23&0&{7264bc2a-6caf-4730-8626-2b34210b1ba8}) disappeared from the system without first being prepared for removal.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="PlugPlayManager" />
        <EventID Qualifiers="49152">12</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:37:39.000Z" />
        <EventRecordID>20773</EventRecordID>
        <Channel>System</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>Storage miniport driver</Data>
        <Data>VMBUS\{7264bc2a-6caf-4730-8626-2b34210b1ba8}\1&amp;3189fc23&amp;0&amp;{7264bc2a-6caf-4730-8626-2b34210b1ba8}</Data>
        <Binary>00000000</Binary>
      </EventData>
    </Event>
    
    
    
    Log Name:      System
    Source:        volmgr
    Date:          7/19/2009 2:40:28 AM
    Event ID:      57
    Task Category: (2)
    Level:         Warning
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    The system failed to flush data to the transaction log. Corruption may occur.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="volmgr" />
        <EventID Qualifiers="32772">57</EventID>
        <Level>3</Level>
        <Task>2</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:40:28.826Z" />
        <EventRecordID>20806</EventRecordID>
        <Channel>System</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>\Device\HarddiskVolume18</Data>
        <Binary>00000000010000000200000039000480000000000E0000C000000000000000000000000000000000</Binary>
      </EventData>
    </Event>
    
    
    
    
    Log Name:      System
    Source:        volsnap
    Date:          7/19/2009 2:40:28 AM
    Event ID:      27
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    The shadow copies of volume \\?\Volume{d00c5356-2c66-11de-9245-001b7842a95c} were aborted during detection because a critical control file could not be opened.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="volsnap" />
        <EventID Qualifiers="49158">27</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:40:28.826Z" />
        <EventRecordID>20807</EventRecordID>
        <Channel>System</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>
        </Data>
        <Data>\\?\Volume{d00c5356-2c66-11de-9245-001b7842a95c}</Data>
        <Binary>0000000002003000000000001B0006C0030000000E0000C01B000000000000000000000000000000</Binary>
      </EventData>
    </Event>
    
    
    Log Name:      System
    Source:        Ntfs
    Date:          7/19/2009 2:41:25 AM
    Event ID:      137
    Task Category: (2)
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    The default transaction resource manager on volume \\?\Volume{d00c5379-2c66-11de-9245-001b7842a95c} encountered a non-retryable error and could not start.  The data contains the error code.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Ntfs" />
        <EventID Qualifiers="49156">137</EventID>
        <Level>2</Level>
        <Task>2</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:41:25.173Z" />
        <EventRecordID>20907</EventRecordID>
        <Channel>System</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>
        </Data>
        <Data>\\?\Volume{d00c5379-2c66-11de-9245-001b7842a95c}</Data>
        <Binary>1C0004000200300002000000890004C000000000850100C000000000000000000000000000000000850100C0</Binary>
      </EventData>
    </Event>
    
    
    Log Name:      Application
    Source:        Microsoft-Windows-Backup
    Date:          7/19/2009 2:41:25 AM
    Event ID:      521
    Task Category: None
    Level:         Error
    Keywords:      
    User:          SYSTEM
    Computer:      hv.domain.local
    Description:
    Backup started at '7/19/2009 9:31:23 AM' failed as Volume Shadow copy operation failed for backup volumes with following error code '2155348001'. Please rerun backup once issue is resolved.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Backup" Guid="{1db28f2e-8f80-4027-8c5a-a11f7f10f62d}" />
        <EventID>521</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:41:25.688Z" />
        <EventRecordID>3348</EventRecordID>
        <Correlation />
        <Execution ProcessID="4728" ThreadID="4548" />
        <Channel>Application</Channel>
        <Computer>hv.domain.local</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data Name="BackupTime">2009-07-19T09:31:23.263Z</Data>
        <Data Name="ErrorCode">2155348001</Data>
        <Data Name="ErrorMessage">%%2155348001</Data>
      </EventData>
    </Event>
    
    
    
    Log Name:      System
    Source:        storvsp
    Date:          7/19/2009 2:41:26 AM
    Event ID:      1
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    The Storage VSP failed to flush a drive mounted in loopback mode. 
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="storvsp" />
        <EventID Qualifiers="49158">1</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:41:26.733Z" />
        <EventRecordID>20915</EventRecordID>
        <Channel>System</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>\Device\STORVSP</Data>
        <Binary>000000000100300000000000010006C000000000850100C000000000000000000000000000000000</Binary>
      </EventData>
    </Event>
    
    
    
    Log Name:      Application
    Source:        VSS
    Date:          7/19/2009 2:41:27 AM
    Event ID:      12305
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      hv.domain.local
    Description:
    Volume Shadow Copy Service error: Volume/disk not connected or not found. Error context: CreateFileW(\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy229,0xc0000000,0x00000003,...). 
    
    Operation:
       Processing PostFinalCommitSnapshots
    
    Context:
       Execution Context: System Provider
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="VSS" />
        <EventID Qualifiers="0">12305</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:41:27.000Z" />
        <EventRecordID>3349</EventRecordID>
        <Channel>Application</Channel>
        <Computer>hv.domain.local</Computer>
        <Security />
      </System>
      <EventData>
        <Data>CreateFileW(\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy229,0xc0000000,0x00000003,...)</Data>
        <Data>
    
    Operation:
       Processing PostFinalCommitSnapshots
    
    Context:
       Execution Context: System Provider</Data>
        <Binary>2D20436F64653A20494E434943484C4830303030303232322D2043616C6C3A2053505251534E504330303030303132352D205049443A202030303030363930382D205449443A202030303030313633322D20434D443A2020433A5C57696E646F77735C53797374656D33325C737663686F73742E657865202D6B2073777072762D20557365723A204E5420415554484F524954595C53595354454D20202020202D205369643A2020532D312D352D3138</Binary>
      </EventData>
    </Event>
    
    
    
    Log Name:      Microsoft-Windows-Hyper-V-VMMS-Admin
    Source:        Microsoft-Windows-Hyper-V-VMMS
    Date:          7/19/2009 2:41:27 AM
    Event ID:      10103
    Task Category: None
    Level:         Error
    Keywords:      
    User:          SYSTEM
    Computer:      hv.domain.local
    Description:
    Failed to revert to VSS snapshot on the virtual hard disk '\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy230\vms\2003VM\2003VM.vhd' of the virtual machine '2003VM'. (Virtual machine ID FF1A13F4-01E7-4B48-9E52-D9687E1F313F)
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Hyper-V-VMMS" Guid="{6066f867-7ca1-4418-85fd-36e3f9c0600c}" />
        <EventID>10103</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2009-07-19T09:41:27.560Z" />
        <EventRecordID>68</EventRecordID>
        <Correlation />
        <Execution ProcessID="2136" ThreadID="4648" />
        <Channel>Microsoft-Windows-Hyper-V-VMMS-Admin</Channel>
        <Computer>hv.domain.local</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <UserData>
        <VmlEventLog xmlns:auto-ns2="http://schemas.microsoft.com/win/2004/08/events" xmlns="http://www.microsoft.com/Windows/Virtualization/Events">
          <VmName>2003VM</VmName>
          <VmId>FF1A13F4-01E7-4B48-9E52-D9687E1F313F</VmId>
          <Parameter0>\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy230\vms\2003VM\2003VM.vhd</Parameter0>
        </VmlEventLog>
      </UserData>
    </Event> 





    http://sharepoint.nauplius.net

    Monday, July 20, 2009 1:53 PM

Answers

  • Eric,

    While this is hardly ideal, I was able to get this working unattended by creating the following schedule:

    • 1:45am - Shutdown all virtual machines using a scheduled task
    • 2:00am - Start backup process
    • 2:15am - Startup all virtual machines using a scheduled task

    This works even if the backup is still running @ 2:15am.  I was going to try tightening up the interval, but since it's working and 30 minutes of downtime isn't critical for us, I've just left it be.

    Best,
    Doug

    • Proposed as answer by eeacosta Wednesday, February 17, 2010 1:40 PM
    • Marked as answer by Vincent Hu Wednesday, May 5, 2010 6:15 AM
    Tuesday, February 16, 2010 2:29 PM

All replies

  • The backup failure is because of a shadow copy creation timing out after 10 minutes.

    Most probably this issue is because of misbehaving Hyper-V writer. I'm assigning this question to the appropriate forum so that someone could help you out.

    Thanks,
    Sriram [MSFT]

    Thursday, September 24, 2009 6:49 AM
  • Hi Trevor - I'm getting a similar error.  Did you get it resolved?
    Sunday, December 6, 2009 2:54 PM
  • Have exactely the same problem.

    Have seen than when we stop a VM with a 500GByte dynamic fixed sized disk before the backup, the problem does not occur.
    Is it possible to increase the time out for the shadow copy creation?

    Franz

    Wednesday, December 16, 2009 1:12 PM
  • Sriram - Has this issue been resolved yet?  We have to shutdown all our VMs before starting a backups or the backups fail.  Ironically, we can restart the VMs right after the backup starts and it seems to work just fine.  To be clear here...the sequence is:
    • Shutdown VMs
    • Start backup
    • Startup VMs --- backup is still running
    • Backup completes successfully
    Thanks for any info.
    Friday, January 29, 2010 8:06 PM
  • Im having the same problem. have anyone resolved this?? 
    Tuesday, February 9, 2010 2:50 PM
  • Hi everyone!


    According to the Microsoft you can safely ignore this message.

    "Event ID 12 is logged in the System log when you back up or compact a Hyper-V virtual hard disk on a Windows Server 2008-based computer"

    http://support.microsoft.com/kb/958669


    Hope this helps as I was having the same issue and I found that it isn't a threat to my Hyper-V servers. I also did check my logs and it was easy to see that it happens only during backup procedure of Hyper-V virtual machines that are currently running.



    Kind regards,
    Matjaz Pavel
    Friday, February 12, 2010 8:56 AM
  • Hi Matjaz Pavel,

    Thanks for sharing this information, but it didn't apply to my troubles.  In my case, the backups failed to complete, so ignoring an error in the logs is not really prudent.  :)  The *only* way we've been able to get WBS to backup is to follow the procedure I mentioned earlier. 

    Best,
    Doug
    Monday, February 15, 2010 4:13 PM
  • Hey Doug Im having exactly the same problem. I noticed that if I restart de VM's before the backup starts running also it works. Of course this is not a solution. If you or anyone find out how to work this out please let us know. I really need to let this unattended.

    Regards,
    Eric
    Tuesday, February 16, 2010 1:26 PM
  • Eric,

    While this is hardly ideal, I was able to get this working unattended by creating the following schedule:

    • 1:45am - Shutdown all virtual machines using a scheduled task
    • 2:00am - Start backup process
    • 2:15am - Startup all virtual machines using a scheduled task

    This works even if the backup is still running @ 2:15am.  I was going to try tightening up the interval, but since it's working and 30 minutes of downtime isn't critical for us, I've just left it be.

    Best,
    Doug

    • Proposed as answer by eeacosta Wednesday, February 17, 2010 1:40 PM
    • Marked as answer by Vincent Hu Wednesday, May 5, 2010 6:15 AM
    Tuesday, February 16, 2010 2:29 PM
  • The Exact same thing is happening to me. I have a virtual server with 4 running VMs. The only way a backup will work is if I put every VM into a saved state and try the backup. I get alternating errors when trying to run WSB:

    Log Name:      Application
    Source:        Microsoft-Windows-Backup
    Date:          2/16/2010 9:18:19 PM
    Event ID:      521
    Task Category: None
    Level:         Error
    Keywords:     
    User:          SYSTEM
    Computer:      CCB-HYPERV2.ccb.rutgers.edu
    Description:
    The backup operation that started at '‎2010‎-‎02‎-‎17T02:08:19.309000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348001'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Backup" Guid="{1DB28F2E-8F80-4027-8C5A-A11F7F10F62D}" />
        <EventID>521</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2010-02-17T02:18:19.835489900Z" />
        <EventRecordID>2052</EventRecordID>
        <Correlation />
        <Execution ProcessID="4024" ThreadID="1960" />
        <Channel>Application</Channel>
        <Computer>CCB-HYPERV2.ccb.rutgers.edu</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data Name="BackupTime">2010-02-17T02:08:19.309000000Z</Data>
        <Data Name="ErrorCode">2155348001</Data>
        <Data Name="ErrorMessage">%%2155348001</Data>
      </EventData>
    </Event>


    Log Name:      Application
    Source:        Microsoft-Windows-Backup
    Date:          2/16/2010 8:43:17 PM
    Event ID:      521
    Task Category: None
    Level:         Error
    Keywords:     
    User:          SYSTEM
    Computer:      CCB-HYPERV2.ccb.rutgers.edu
    Description:
    The backup operation that started at '‎2010‎-‎02‎-‎17T01:37:42.352000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Backup" Guid="{1DB28F2E-8F80-4027-8C5A-A11F7F10F62D}" />
        <EventID>521</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2010-02-17T01:43:17.652087900Z" />
        <EventRecordID>2045</EventRecordID>
        <Correlation />
        <Execution ProcessID="4024" ThreadID="3012" />
        <Channel>Application</Channel>
        <Computer>CCB-HYPERV2.ccb.rutgers.edu</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data Name="BackupTime">2010-02-17T01:37:42.352000000Z</Data>
        <Data Name="ErrorCode">2155348129</Data>
        <Data Name="ErrorMessage">%%2155348129</Data>
      </EventData>
    </Event>

    Log Name:      Application
    Source:        VSS
    Date:          2/16/2010 8:42:45 PM
    Event ID:      12322
    Task Category: None
    Level:         Information
    Keywords:      Classic
    User:          N/A
    Computer:      CCB-HYPERV2.ccb.rutgers.edu
    Description:
    Revert operation for volume \\?\Volume{e60a5582-bf93-11de-a919-002481e15acf}\ has begun.  Volume is being reverted to the shadow copy with id {befc4f9c-3afd-40d1-81ea-6781194f3c79}.

    Operation:
       Revert a Shadow Copy

    Context:
       Execution Context: Coordinator
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="VSS" />
        <EventID Qualifiers="0">12322</EventID>
        <Level>4</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2010-02-17T01:42:45.000000000Z" />
        <EventRecordID>2044</EventRecordID>
        <Channel>Application</Channel>
        <Computer>CCB-HYPERV2.ccb.rutgers.edu</Computer>
        <Security />
      </System>
      <EventData>
        <Data>\\?\Volume{e60a5582-bf93-11de-a919-002481e15acf}\</Data>
        <Data>{befc4f9c-3afd-40d1-81ea-6781194f3c79}</Data>
        <Data>

    Operation:
       Revert a Shadow Copy

    Context:
       Execution Context: Coordinator</Data>
        <Binary>2D20436F64653A20434F52434F4F524330303030313532302D2043616C6C3A20434F52434F4F524330303030313436392D205049443A202030303030343039322D205449443A202030303030333138302D20434D443A2020433A5C57696E646F77735C73797374656D33325C76737376632E6578652020202D20557365723A204E616D653A204E5420415554484F524954595C53595354454D2C205349443A532D312D352D313820</Binary>
      </EventData>
    </Event>

    Wednesday, February 17, 2010 3:37 AM
  • Thanks Doug, that is a good workaround for now. Lets see if Microsoft do something about it.
    Thanks again for the idea.
    Wednesday, February 17, 2010 1:42 PM
  • Has anyone come up with any ideas as to why this is happening? Is it related to the harddrive speeds? Is running 7200 rpm sata drives on a hyper-v server a bad idea? I only have 4 VMs on a 2x Intel quad box. But am unable to perform the backup because the VSS keeps timing out. That means the harddrive can't handle the process of creating the snapshots for backup?

    Monday, March 1, 2010 9:49 PM
  • I suggested a work-around above.  Have you tried that?  Did it work?
    Tuesday, March 2, 2010 6:17 PM
  • Yes, if I shutdown all of the virtual machines I can get a successful backup. But this is hardly an ideal solution.
    Wednesday, March 10, 2010 8:49 PM
  • I agree completely.  It is not an ideal solution, and I am quite frustrated that it is necessary.

    However, I think there is value in providing this work-around in a public forum:
    • It gives people a work-around.  After all, inconvenient backups are better than failed backups.
    • It isolates the problem to a specific group within Microsoft.  Somewhere, somebody who manages the Hyper-V group...specifically the folks responsible for the VSS writer...should be pounding someone on the head and saying "Hey...why doesn't this work?"
    • It shines a bright glaring light on the problem, and the more people (like you) that stand up and say "Yes, this is a problem"...the more likely it will get fixed.
    So...thanks for adding your comments.  :)
    Thursday, March 11, 2010 1:20 AM
  • FWIW, I never found a solution to this issue. Most of the time, the backup would just work, but I still encountered a few days here and there that it would fail.
    http://sharepoint.nauplius.net
    Thursday, March 11, 2010 5:13 AM
  • Eric,

    While this is hardly ideal, I was able to get this working unattended by creating the following schedule:

    • 1:45am - Shutdown all virtual machines using a scheduled task
    • 2:00am - Start backup process
    • 2:15am - Startup all virtual machines using a scheduled task

    This works even if the backup is still running @ 2:15am.  I was going to try tightening up the interval, but since it's working and 30 minutes of downtime isn't critical for us, I've just left it be.

    Best,
    Doug

    Doug,

    How do you creat this schedule?

    Thanks!

    Mike

    Monday, March 22, 2010 9:19 PM
  • Mike,

    First, I installed a powershell script that I found on Codeplex: http://capilla.codeplex.com/releases/view/29002

    If you haven't worked with powershell, be prepared for a little learning curve.  This ain't you Daddy's cmd.exe.

    After that, it was easy.  I created a Event Viewer Task (go to Server Manager -> Configuration -> Task Scheduler -> Task Scheduler Library).  In the task, I added one action for each virtual machine...like this:

    Start a program:  powershell .\hvcmd.ps1 "shutdown vmachine1"

    Next, you'll want to test the crud out of this.  After you get it working, create another one to start things up with actions like this:

    Start a program:  powershell .\hvcmd.ps1 "start vmachine1"

    Hope this helps!

    Doug

    Tuesday, March 23, 2010 11:26 AM
  • My server worked flawlessly for over a month.  Then, it started failing every few days.  It rapidly progressed to the point where it didn't work at all. 

    I really wish Microsoft would get this fixed.  It hardly inspires confidence in Hyper-V.

    Tuesday, March 23, 2010 11:32 AM
  • Hi Doug, I was in the same situation. I was able to resolve my issue re-creating the virtual machines that were giving me problems. For example I ran a P2V conversion of the VM having problems. I used SCVMM. Now it is working fine. I hope this helps.
    • Proposed as answer by Doug Clutter Saturday, July 31, 2010 1:15 PM
    Wednesday, March 24, 2010 3:47 PM
  • Thanks for the suggestion eeacosta.  I've been trying to find the time to do this, but no luck yet.  Will post back when I have some results.
    Monday, April 12, 2010 11:51 AM
  • We're having the exact same issue with two servers running Windows Server 2008 R2, one running 4 HyperV VM's, the other running 9 VM's.  After 168 days of continuous good backups on the former, they now fail.

    More disturbing is I have a passthrough drive set up on two of the VM's on this server that require more frequent backups, and those now have started failing as well (windows backup running within the VM itself, to a dedicated physical drive passed through by the host machine).

    The other server (running 9 VM's) fails when it tries to back up the main hyperV host, but incremental backups still work (for now) on a passthrough drive from within one of the more critical VM's.

    Hope Microsoft fixes this ASAP, I can no longer even get SINGLE backups to run on the former machine, let alone scheduled, even if I shut down the VM's prior to starting!  So for the time being, I have to manually copy the VHD's off of the host to another machine, after shutting the VM's down.  Very tedious, as I don't like to have to come in to the office at night and sit for two hours while these manual backups run.

     

    Thursday, June 10, 2010 7:47 PM
  • Hey guys, lots of good info here.  I'm running into the same issue as well, only a single VM running on a 2008 Standard server and it appears to be killing every other backup with the Shadow Copy timeout error.  Have there been any new fixes for this since the last post?
    Wednesday, July 28, 2010 4:51 PM
  • Sorry Alex nothing new to report; it is still broke.

    I can report that it isn't something buggered on the host OS.  I had to completely rebuild the host server last weekend.  After that, I spun up the same VHDs as VMs.  Didn't help a bit.  Same errors...same resolution.

    This leaves me wondering if something could be amiss with my Win2003 guests.  eeacosta suggested that rebuilding the guests with SCVMM would fix the problem...but I don't have SCVMM installed and it seems like overkill to fix the problem. 

    I might try DISK2VHD.  Good luck!

    Thursday, July 29, 2010 6:05 PM
  • Alex,

    Based on the suggestion by eeacosta, I rebuilt my VHD files and I'm very, very happy to report this has resolved the problem.  I'm not sure why, but it has worked for both a Full Backup and an Incremental Backup.  Here's what I did:

    1. Log in to each guest OS
    2. Ran DISK2VHD (found on SysInternals.com which is now a Microsoft site) to create a new VHD.  I suggest storing the VHD in the same folder as your current VHD but do NOT overwrite any existing VHDs.  The path I used was \\MyHost\C$\Users\Public\Documents\Hyper-V\Virtual hard disks.  Also, I did NOT choose the "Setup for Virtual PC option"...because it didn't seem to apply to Hyper-V.
    3. Log in to the host OS
    4. Delete each VM
    5. Delete the old VHDs --- I'm assuming you have backups here!
    6. Create a new VM with the new VHD
    7. Login to the new VM - let it find and install new hardware
    8. Mount the Integration Services CD and run it

    That's it!  Hope it works for you.

    Huge thanks to eeacosta for suggesting this approach!  You really made my day!

    ---------------------------

    UPDATE 8/3/10

    ---------------------------

    Backups have failed the last 2 nights with the same old error....Creation of the shared protection point timed out.

    Backups seem like a fairly critical issue, so, c'mon Microsoft, let's get this fixed. 

    • Edited by Doug Clutter Tuesday, August 3, 2010 2:29 PM Update
    Friday, July 30, 2010 1:36 PM
  • See the post at http://social.technet.microsoft.com/Forums/en-US/windowsbackup/thread/7add923f-fad1-47ab-b02c-ee0fd073cb7e There is a fix here that worked for me.
    Thursday, November 4, 2010 4:09 AM
  •  

    This is how we FIXED this problem and there are many fixes listed here:

    a) Our Fix:

    - Login to the server where Windows Server Backup runs

    - Check the Event Viewer Logs under Application Services -> Backup  (Operations)

    - If you have Hyper-V running, then go to the Hyper-V section under Application Services -> Hyper-V VMS (Operations).

    - In that log, it will state which VM is causing the error.

    - Login to that VM and check the Event Viewer logs. In our case it was SQLWriter causing it.

    - Solution is simple, batch a simple script on that VM and add it to Task Scheduler to stop SQLWriter few minutes before the backup starts and then start it up few minutes afterwards. (net stop sqlwriter , net start sqlwriter)

    - If you have a local SQL server on the server you're doing Windows backup, then this may need to be done.

     

    b) Other fixes reported by users:

    - from command prompt type: diskpart

    - from command prompt type: automount  and check if automount is enabled/disabled, it need to be enabled

     

    c) Another fix (not recommended in my opinion) only works if you have Hyper-V installed:

    - Disable VSS Writer for Hyper-V using the following:

    goto registry on the server running windows backup: 

    1. hklm\software\microsoft\windowsNT\currentversion\spp
    2. Add a new key as type reg_Multi_SZ
    3. Name for new key: DisabledWriters
    4. Value for new key: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de} (remember that yours may be different - run a vssadmin list writers to determine your value for the key)
    5. net stop wbengine
    6. net start wbengine

     

    I thought to combine all fixes in one as it takes alot of time just searching and finding tons of different topics with 10s posts until you see the fix. Good luck!

     

    Tamouh
    *** Run at your own risk, I don't take responsibility for opinions expressed here ***
    Monday, July 25, 2011 5:07 AM
  • I found this thread when, unfortunately, my environment produced the same error message.  Although your workaround is viable, I uncovered additional information in my case:

    +The problem existed in a guest, the errors in WSB, etc. were simply the host notifying.

    +I examined each guest.  I found one guest in particular was running an AV scan at the EXACT same time as my VSS backup of that guest.

    +My speculation is- the AV scan was chewing up too much of the guest's resources.  I moved the start of my VSS backup to several hours earlier, and my problem disappeared!

    I offer these points by way of clarification.

    Jason Hoskins

    Friday, August 17, 2012 6:35 PM