none
Windows Server Backup alway crash on some given servers RRS feed

  • Question

  • Hi All,

    I'm using SCDPM 2012 R2 RU6  to protect several servers. protected servers are VMs hosted on VMwareand have 3 drives: C:, D. for applications and E: for swap file.

    I have configured 1 protection group with volume backup scheduled on 6 pm and BMR backup on 8 pm. Protected Servers are unning Windows Server 2008 R2 and 2012

    the issue is that BMR fail on some of the servers. if I replace BMR bya System State backup then it works fine.

    on the protected server event log shows that

    1/ WSb has failed because of network issues,

    2/ Windows Error Reporting event indicate that WSB has crashed

    Log Name:      Application
    Source:        Microsoft-Windows-Backup
    Date:          14/07/2015 21:22:10
    Event ID:      517
    Task Category: None
    Level:         Error
    Keywords:      
    User:          SYSTEM
    Computer:      computer.company.com
    Description:
    The backup operation that started at '2015-07-15T01:17:56.504790200Z' has failed with following error code '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-Backup" Guid="{1DB28F2E-8F80-4027-8C5A-A11F7F10F62D}" />
        <EventID>517</EventID>
        <Version>1</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x8000000000000000</Keywords>
        <TimeCreated SystemTime="2015-07-15T01:22:10.992739900Z" />
        <EventRecordID>39313</EventRecordID>
        <Correlation />
        <Execution ProcessID="5944" ThreadID="2704" />
        <Channel>Application</Channel>
        <Computer>computer.company.com</Computer>
        <Security UserID="S-1-5-18" />
      </System>
      <EventData>
        <Data Name="BackupTime">2015-07-15T01:17:56.504790200Z</Data>
        <Data Name="ErrorCode">0x8078015b</Data>
        <Data Name="ErrorMessage">%%2155348315</Data>
      </EventData>
    </Event>

    This e-mail, including attachments, is intended for the person(s) or company named and may contain confidential and/or legally privileged information.
    Unauthorized disclosure, copying or use of this information may be unlawful and is prohibited. If you are not the intended recipient, please delete this message and notify the sender.
    All incoming and outgoing e-mail messages are stored in the Swiss Re Electronic Message Repository.

    If you do not wish the retention of potentially private e-mails by Swiss Re, we strongly advise you not to use the Swiss Re e-mail account for any private, non-business related communications.

    Log Name:      Application
    Source:        Windows Error Reporting
    Date:          14/07/2015 21:22:11
    Event ID:      1001
    Task Category: None
    Level:         Information
    Keywords:      Classic
    User:          N/A
    Computer:      computer.company.com
    Description:
    Fault bucket , type 0
    Event Name: Windows Server Backup Error
    Response: Not available
    Cab Id: 0

    Problem signature:
    P1: 4
    P2: 1
    P3: 0x8078015b
    P4: 0x8007003b
    P5: 0x8078015b
    P6: 0x00000000
    P7: 0x8078015b
    P8: 0x00000000
    P9:
    P10:

    Attached files:

    These files may be available here:
    C:\ProgramData\Microsoft\Windows\WER\ReportQueue\NonCritical_4_409ec6bd2146c21ada927c7f5969cbe324d2b3_21acb088

    Analysis symbol:
    Rechecking for solution: 0
    Report Id: ea201e75-2a8f-11e5-9409-0050568640ee
    Report Status: 4
    Hashed bucket:
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Windows Error Reporting" />
        <EventID Qualifiers="0">1001</EventID>
        <Level>4</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2015-07-15T01:22:11.000000000Z" />
        <EventRecordID>39312</EventRecordID>
        <Channel>Application</Channel>
        <Computer>computer.company.com</Computer>
        <Security />
      </System>
      <EventData>
        <Data>
        </Data>
        <Data>0</Data>
        <Data>Windows Server Backup Error</Data>
        <Data>Not available</Data>
        <Data>0</Data>
        <Data>4</Data>
        <Data>1</Data>
        <Data>0x8078015b</Data>
        <Data>0x8007003b</Data>
        <Data>0x8078015b</Data>
        <Data>0x00000000</Data>
        <Data>0x8078015b</Data>
        <Data>0x00000000</Data>
        <Data>
        </Data>
        <Data>
        </Data>
        <Data>
        </Data>
        <Data>C:\ProgramData\Microsoft\Windows\WER\ReportQueue\NonCritical_4_409ec6bd2146c21ada927c7f5969cbe324d2b3_21acb088</Data>
        <Data>
        </Data>
        <Data>0</Data>
        <Data>ea201e75-2a8f-11e5-9409-0050568640ee</Data>
        <Data>4</Data>
        <Data>
        </Data>
      </EventData>
    </Event>

    This e-mail, including attachments, is intended for the person(s) or company named and may contain confidential and/or legally privileged information.
    Unauthorized disclosure, copying or use of this information may be unlawful and is prohibited. If you are not the intended recipient, please delete this message and notify the sender.
    All incoming and outgoing e-mail messages are stored in the Swiss Re Electronic Message Repository.

    If you do not wish the retention of potentially private e-mails by Swiss Re, we strongly advise you not to use the Swiss Re e-mail account for any private, non-business related communications.


    Cheers

    Wednesday, July 15, 2015 11:32 AM

All replies

  • Hi,

    The main difference between SystemState (SS) backup that works and BMR backup that fails is the data transfer methods are different.

    SystemState - DPMRA on the protected server initiates a system state backup using Windows Server Backup (WSB) to write the backup to C:\WindowsImageBackup.  Once WSB reports a successful SS backup, DPMRA copies the c:\Windowsimagebackup contents to the DPM replica volume.

    BMR - DPMRA on the protected server initiates a BMR using Windows Server Backup (WSB) to write the BMR backup directly to the DPM Server replica volume via a share. 

    The errorcode '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder) is showing that WSB had trouble, either network communications/ permissions (look at anti-virus) or some kind of disk IO problem.

    You should start by looking at network configuration / connectivity / Anti-virus and maybe take a network trace while WSB BMR backup is in progress to see where the IO failure is occuring. 


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.



    Wednesday, July 15, 2015 6:05 PM
    Moderator
  • Hi Mike,

    Thanks for your answer.

    Here are some information that may clarify the issue:

      • The issue has been observed on Windows server 2012 only so far. No 2008 R2 are concerned
      • some server used to have successfull backup thaen started to fail every day (not a permission or configuration issue)
      • failing server fail even if the backup job is trigered manually from DPM server (seems not to be an IO issue)
      • For a given server the BMR backup will almost always fail after transferring the same amount of data. Some servers fail after less than 1 GB some others fails after few GB (no permission issue)
      • The AV on the DPM server is already configured to exclude the root folder containing mounting points

      In all cases this would not explain why WSB crashes ? it should logh an error but not crash !

      Regards


    Cheers

    Thursday, July 16, 2015 11:18 AM
  • Hi,

    Please try running a manual BMR backup on the effected protected server (PS).

    1) Set up a network share on a remote machine that has lots of free space: \\server\bmrshare

    2) From an administrative command prompt on the PS, type:

                  wbadmin.exe start backup -allcritical -backuptarget:\\server\bmrshare

    This should show you the list of volumes included in the BMR backup and ask "Do you want to start the backup operation?. - Type Y to continue..

    3) Observer if the backup is successful or also crashes.  Under the \\server\BMRshare share check the size of the files under the WindowsImageBackup. 


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, July 16, 2015 2:08 PM
    Moderator
  • Hi Mike,

    the backup is doing swell so far: 6 GB backed up and still going. Previously it does not go beyond 366 MB.

    if it succeed what can be concluded ?

    Regards


    Cheers

    Thursday, July 16, 2015 2:21 PM
  • Hi,

    Next step will be to perform a BMR backup to the DPM replica volume via the BMR share using SYSTEM account context. 

    First - On the DPM Server get the path to the replica volume for the BMR backup for the same protected server by clicking on the "path to replica" while viewing data sources under the protection group.  Then type "NET SHARE" from an administrative command prompt.  The BMR share will just be a GUID but match it with the path from the above.

    Next - in the protected server perform the following:

    BMR volume on the DPM server is really locked down from a security standpoint, however DPMRA runs in system context so you need to do the same for this test.

    1)      Download psexec.exe from www.sysinternals.com
    2)      Run PSEXEC -s cmd.exe   (this will switch window to system context) 
    3)      Afterwards - type WHOAMI it should return:

                      nt authority\system

    4)  NET VIEW \\DMB_SERVER   - this should list the shares and match the share names seen on the DPM Server.

    5) Run the BMR backup manually:

        C:\>wbadmin start backup -allcritical -backuptarget:\\DPM_SERVER\BMR_SHARE_GUID

    6)  See if that has troubles completing.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.


    Thursday, July 16, 2015 2:41 PM
    Moderator
  • Hi Mike,

    I picked up the share name from previously failing backup, Backup location field.

    The backup has started hanged s short period at the usual value, 366 MB then continued.

    Now it succeeded ?

    Regards


    Cheers

    Thursday, July 16, 2015 4:09 PM
  • Hi,

    OK - now start a new DPM BMR backup for the same protected server.  There is no difference between what you just did with the manual backup using system account and what DPMRA executes during a scheduled backup.  DPM is simply acting as a scheduler and is depository for the same wbadmin.exe commands you ran manually.   


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, July 16, 2015 4:14 PM
    Moderator
  • Hi,

    I started a consistency check that has passed the critical ammount of transfer. will keep you informed.

    if the current one succeed, how could you explain that more than 22 consecutive backup failed while this current one succeed ?

    Regards


    Cheers

    Thursday, July 16, 2015 4:56 PM
  • Hi,

    The consistency check worked well. But the planned backup failed !

    Regards


    Cheers

    Friday, July 17, 2015 8:03 AM
  • Hi,

    I tried the same procedure with another failing server:

    • the backup to a share using my account worked well
    • using system account exhibited the same behaviour: fail after transferring 17 GB

    Any idea ?


    Cheers

    Friday, July 17, 2015 9:11 PM
  • Hi

    I confirm that every test using my account works whereas uding System Account fail !


    Cheers

    Friday, July 17, 2015 9:47 PM
  • Hi,

    Are you using the same share for both tests ?  IE:  User backup up to \\server\share and System account backup to same \\server\share ?


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, July 20, 2015 7:00 PM
    Moderator
  • Hi,

    Definitely no: for my account it is \\server\testbackup and for system account it is the GUID like share that you mentionned in your previous post.


    Cheers

    Monday, July 20, 2015 8:03 PM
  • My backups on a Windows Server 2016 server stopped working recently. I think it may perhaps have been because of an update that changed something in the backup process that made it incompatible with the prior backups.

    I moved the WindowsImageBackup folder to another folder temporarily and on the next backups, it performed normally. Later I removed the old backup. It seems to be working fine now.

    I hope this was helpful to someone.

    Cheers

    Friday, January 26, 2018 2:04 PM
  • Hello,

    Not sure if it's related, but I'm starting to have the same issue with DPM 2016 installed on a Windows Server 2016. They are all patched with January Monthly Updates.

    The BMR protected servers are physical Windows Server 2012. Some of them have the Windows Updates from January 2018, not all.

    All of them have the same error :

    '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.)

    Never had this issue before (using DPM since year 2014), it appears after theses monthly updates.

    I will try to remove these last updates from my DPM servers.

    Thursday, February 1, 2018 8:04 AM
  • Did you ever find a solution? I'm having the same issue on MS Azure Backup Server (which is DPM 2016) on a 2016 server. Some servers just will not take a BMR backup no matter what I try. 

    Friday, October 5, 2018 4:09 PM
  • Just the latest update from me... I "upgraded" my MABS (based on DPM 2016) From MABS v2 to MABS v3 (based on DPM 2016 build 1807). This included a new version of the remote agent. This did seem to fix the issues with some backups randomly failing with the original poster's error. However, the reporting module in this build is completely broken. I have a new thread on that here:

    https://social.technet.microsoft.com/Forums/en-US/4d590fc0-d618-4460-88f1-63c04376db9a/mabs-v3-based-on-dpm-2016-build-1807?forum=dataprotectionmanager

    So it fixes one problem but creates another. 

    Additionally, my "upgrade" did not go smoothly. Post "upgrade" none of my scheduled jobs were actually running, and the reporting was broken. The failed jobs were choking up all kinds of errors about invalid SQL job names and the link. I had to completely uninstall DPM/MABS, completely uninstall all SQL components, DELETE THE MABS AND SQL DIRECTORIES (or the reinstall blows up) and then reinstall MABS v3 fresh and rebuild all jobs from scratch. This IS DESTRUCTIVE to your existing backup set. I found that out the hard way. So I would recommend a side by side migration as opposed to the "upgrade".

    I have worked around the completely broken reporting in DPM by getting the recovery point reports directly from SQL Reporting services itself. I've subscribed to and scheduled them there. It works, but it's less than ideal. Reporting in DPM has been broken forever. See:

    https://blogs.technet.microsoft.com/dpm/2013/12/05/attempts-to-schedule-a-mailed-report-in-dpm-2012-fail-with-reporting-services-server-cannot-connect-to-the-dpm-database/

    You would think they would have this fixed by now. The above did fix reporting in every issue of DPM prior to this version. The problem/bug appears to be a different issue this time. 

    DPM/MABS continues to be half-baked for sure. 

    Tuesday, December 11, 2018 3:27 PM