none
Server 2008 sp2 freezes during DPM 2010 volume shadow backup RRS feed

  • Question

  • Hi All,

    We have a hyperv guest server 2008 sp2 that freezes during DPM 2010 volume shadow backup.
    I presume this is when backing up SQL databases. There are no errors in the event logs.
    The sequense of entries in the System event are as follows up until the server freezes.

    1) The DPMRA service entered the running state.
    2) The Volume Shadow Copy service entered the running state. 
    3) DCOM  started the service swprv with arguments "" in order to run the server:
    {65EE1DBA-8FF4-4A58-AC1C-3470EE2F376A}
    4) The Microsoft Software Shadow Copy Provider service entered the running state.

    After this the entries stop and the new entries are from after reboot.                                                               
    You cannot send control-alt-delete or connect to the server in any way.
    Only hard reboot gets it going again.This is the only server this is happening to.

    Please advise if anybody has experienced this and how they resolved.
    Maybe I require a Hotfix.

    Friday, August 17, 2012 11:59 AM

All replies

  • Hi

    It could be that the guest machine runs out of resources when running the backup and then hangs completely.

    Are you able to ping the machine at all? Are their any other jobs running like SQL Maintenance jobs that run in the background possibly clashing with the DPM backup?

    Are your Hyper-V servers up to date and do you have the latest integration components installed?

    Friday, August 17, 2012 12:27 PM
  • Thanks.

    Does not seem to be a resource issue. The freezing happens at night when the backup schedule kicks in and nobody is working. Last time it happened was last night (Sunday) with no users even logged on.

    Server completely freezes and you cannot ping or connect. Need to shutdown and startup again.
    I checked and there is no maintenenace jobs runnig in the background.

    I believe hyperv and integration services are up to date as these are all relatively new setups in the last few months. Hosts are fully patched. How can I check and confirm the latest integration services are installed?

    When this happened the first few times I thought the problem might be with the host server and moved the guest to a more powerfull server with more resources. Problem seems to be with Volume Shadow Copy as system logs stop when volume shadow copy enters a running state.

    At them moment I am trying to work out if it is backup of a specific member causing the problem or random. Unfortunately the problem is intermittent so cannot reproduce everytime, but it does seem to become much more frequent now.

    Monday, August 20, 2012 6:21 AM
  • Hi

    Here is a script to check what version of the integrations services are installed:

    http://blogs.msdn.com/b/robertvi/archive/2010/10/11/a-script-to-check-the-integration-services-version-on-hyper-v-host-and-guests.aspx

    Some interesting reading regarding VSS and the server freezing:

    http://garysgambit.blogspot.com/2010/03/2008-server-freeze-hyper-v-or-volume.html

    Monday, August 20, 2012 7:11 AM
  • Thanks for the script, but integration Services seem to be up to date 6.1.7601.17514

    I have read that article you suggested before but did not feel it applied to me. I am going to give it another look though and make sure I cover all the bases.

    thanks!

    Monday, August 20, 2012 6:05 PM
  • What I can see after a quick look again is that this Hotfix to this problem is for only 2008 R2 and my problem is 2008 sp2. Think this was the original reason for not following through with this link.

    Still going to look into this a bit further though.

    Monday, August 20, 2012 6:18 PM
  • Hi

    Maybe you also want to ask the question in the Hyper-V section, perhaps they can assist you further?

    Kind Regards

    DareDevil57

    Wednesday, August 22, 2012 5:13 AM
  • Thanks,

    This does not seem like a hyperv issue as the agent is installed inside the guest, so it treats the guest as a physical machine.

    Update: I have unistalled agent and are now protecting the server with dpm 2012 with the latest agent. Problem is still there if not worse. Every time a backup now runs the server freezes and it does not seem to be just related to the SQL backups but also the file backups.

    Any advise will be much appreciated. Seems my next step will have to be reving dpm from this server and use alternate backup, which messes with my whole centralized dpm backup and monitoring. Hope to rather resolve the issue. 

    Wednesday, September 5, 2012 6:12 AM
  • This looks similar to what I'm seeing.

    DPM 2010, there's one backup set (for me a file server disk) that every time I try to run the initial replica on it the server hangs and needs to be rebooted by iLO. It doesn't just die suddenly, first the data stream on the backup stops then the OS becomes less responsive but there is no resource issue. trying to open event view will cause a few things to lock up then over a few mins the server is complete froze. like the disk drives have been locked.

    Suspecting McAfee, I added in all the exclusions, that didn't help so I added the process exclusions which are done by setting dpmra and csc to low risk and that didn't help either. I could reproduce it just by kicking off a backup for this one file servers drive so it's easy to test with.

    Tonight, I had some permissions in EPO to let me stop the scanning completely and disable the on-access scan and for the first time it worked!

    There is definitely an issue between DPM and McAfee beyond what is on MS's web page for AV checks.

    I don't have a workaround yet other than stopping the AV completely... Something to follow up on next week. For the moment I made some progress though.

    Sunday, September 16, 2012 12:29 PM
  • This was an AV issue

    I needed to find a way in mcafee to exclude not only the file locations but the processes had to be registered as Low Risk in the McAfee consol

    • Proposed as answer by Dup Lawyer Wednesday, February 27, 2013 7:07 AM
    Wednesday, February 27, 2013 7:06 AM
  • In my case it was not the AV as I uninstalled AV completely and still had the problem. In the end I opted to uninstall dpm agent and just backup the hyperv guest. We will pull backups through ILR  if needed. Users stopped using SQL and moved to pervasive application so that took care of SQL backups.

    Biggest issue I had was to convert single partition server from dynamic to basic disk for ILR to work (not having to pay for software).



    • Edited by Dirk Slabbert Friday, March 29, 2013 9:47 AM spelling
    Friday, March 29, 2013 9:45 AM
  • Dirk, did you ever resolve this, or just change backup software? I'm seeing very similar problems with DPM 2012 laptop backups and McAfee anti-virus. Tried the recommended exclusions etc.

    Dup Lawyer, are you saying you resolved this issue by excluding all the relevant processes from being scanned, but also registering ALL the relevant processes as LOW RISK?

    "I needed to find a way in mcafee to exclude not only the file locations but the processes had to be registered as Low Risk in the McAfee console".

    Thanks!

    Monday, November 24, 2014 4:26 PM