none
DPM encountered a critical error while performing an I/O operation on the tape - HP MSL4048 & Ultrium 4 RRS feed

  • Question

  • Hi

    We have recently moved DPM platform from Windows 2003 R2 x32 and DPM 2007 x32 to Windows 2008 R2 x64 and DPM 2010 x64 (all windows updates installed). The new server is built upon HP Proliant DL380 G6, 2x MSA20 + 1x MSA60, approx 25Tb storage. We also have an HP MSL4048 G3 library and HP Ultrium 4 tape drive.

    All devices have latest drivers and firmware installed - for firmware the library is on HP version 7.20 and the tape drive is on HP version B56W/MSL G3. The drivers for the library are hpuln64.sys version 1.9.0.0 dated 03/06/2009 and the tape driver is hplto.sys version 1.0.6.1 dated 14/05/2009

    Protection Groups are split via application set (1x SQL, 1x Exchange, 1x System State, 6x Volume etc). Data co-location is enabled.

    When backing up large files (Exchange databases or SQL databases at >300Gb for example), DPM fails after 2 minutes and 0.13Mb written to tape. The error is always similar:

    Event log error:

    The back up to tape job failed for the following reason: (ID: 3311)
    DPM encountered a critical error while performing an I/O operation on the tape SQL Server Protection Group-LT-1Weeks-Copy0-00000001 (Barcode - ) in Drive Hewlett Packard LTO Ultrium-4 drive. (ID: 30101)

    DPM detailed information error:

    DPM encountered a critical error while performing and I/O operation on the tape SQL Server Protection Group-LT-1Weeks-Copy0-00000001 (Barcode - ) in Drive Hewlett Packard LTO Ultrium-4 drive. (ID: 30101) Details: The request could not be performed because of an I/O device error (Ox8007045D))

    However (and this is a BIG however) - small backup jobs (under 1Gb or thereabouts) do complete successfully and are written to tape ok. I am also able to manually mount and read tapes, erase tapes successfully within DPM2010.

    I have run HP Library and Tape tools and performed all available hardware and tape tests which complete without error.

    I have searched and read through every single similar thread on the net including applying "BufferQueueSize"=dword:00000002 and "BusyRetryCount"=dword:000000c8 registry fixes which resolved the problem for others, but to no avail.

    This tape library performed flawlessly under DPM2007

    Any help greatly appreciated as I am now beginning to tear my hair out :(

    Many thanks

    Saturday, August 14, 2010 9:27 AM

Answers

  • Just wanted to report back as this issue is now fully resolved

    We ended up changing all SCSI cables for brand new ones and also changing the SCSI terminator at the tape drive, so it is likely to be a cable fault which is really odd as all we did was unplug the working MSA's, library and tape drive from one DPM server and move them to the new DPM server.

    All cables were fully labelled during the move and they were all plugged in exactly the same ports so I have no idea what went wrong there, but such is life!

    Thanks for your assistance Mike :)

    • Marked as answer by evilowluk Friday, August 20, 2010 7:20 AM
    Friday, August 20, 2010 7:20 AM

All replies

  • Try replacing the tape driver from HP (hplto.sys) with the native windows drivers via device manager. Manually choose the driver from the LTO family and then restart and make sure DPM still recognizes the tape drives in the library.  You can try an ad-hock backup by right-clicking a large data source and choose to backup to tape. 


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, August 14, 2010 3:50 PM
    Moderator
  • Thanks Mike.

    I tried what you suggested and it loaded an MS driver dated 2006 but following the reboot, the tape drive was no longer recognised by DPM at al (the library was still recognised). To get it back again I have had to revert back to the hplto.sys driver

     

     

    Saturday, August 14, 2010 4:59 PM
  • OK - Please work with HP, the fact that the libary "worked with dpm 2007" makes be believe this is still a driver problem.  DPM 2010 does not write to the tape drive any differently than DPM 2007.  The things that have changed are the server hardware, the operating system and the x64 tape drivers from HP.   

    Since it's all HP hardware, drivers, they would be in the best postion to duplicate your problem in-house and troubleshoot the issue. 


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Saturday, August 14, 2010 6:03 PM
    Moderator
  • Thanks, I will raise a call with them shortly. Unfortunately I already know what will be said and that is the library and driver works fine and passes all the HP Library and Tape Tools tests without error, so I am going to be stuck between HP and MS both pointing fingers at each other on this one. Fantastic :(

    Here is a HCL for tape libraries and drivers for DPM 2007: http://technet.microsoft.com/en-us/systemcenter/dm/cc678583.aspx

    Please could you link me to the relevant DPM 2010 article as I cannot find it?

    Monday, August 16, 2010 7:36 AM
  • Please check to make sure the tape drive (library) / controller / driver combination is supported by the vendor.

    FYI -We have seen instances where the "HP insight Agent service" would cause backups to fail with the same error - please disable or uninstall all the HP insight services and see if that helps.

     

    Report back whatever you find resolves the issue.

     


    Regards, Mike J [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, August 17, 2010 1:15 AM
    Moderator
  • Just wanted to report back as this issue is now fully resolved

    We ended up changing all SCSI cables for brand new ones and also changing the SCSI terminator at the tape drive, so it is likely to be a cable fault which is really odd as all we did was unplug the working MSA's, library and tape drive from one DPM server and move them to the new DPM server.

    All cables were fully labelled during the move and they were all plugged in exactly the same ports so I have no idea what went wrong there, but such is life!

    Thanks for your assistance Mike :)

    • Marked as answer by evilowluk Friday, August 20, 2010 7:20 AM
    Friday, August 20, 2010 7:20 AM