none
Slow DPM Hyper-V CSV backup using iSCSI target RRS feed

  • Question

  • I am running into performance issues when I backup CSV backed VMs; my disk IO falls from 12MB/s to .2MB/S From reading on this forums, I think the problem is that I am using the recently released iSCSI target and I don’t have a hardware snapshot provider. Is there a way to effective backup VMs where I don’t have a hardware snapshot provider? Inside the MS iSCSI target there is the option for a snapshot. This takes only a few seconds to process. Is there an scripting process to make these available for DPM?
    Tuesday, May 24, 2011 12:35 AM

Answers

  • If the above suggested answer does not help in resolving the thread please re-open it.
    --------------------------------------------------------------------------------
    Regards, Michael V [MSFT] - This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, January 19, 2012 6:28 PM
    Moderator

All replies

  • Hi,

    You will need to fix the poor disk performance for the csv storage - DPM is going to create a snaphot of the csv (using h/w or s/w vss provider) then mount the snapshot and copy changed blocks from the snapshot.  DPM cannot use iscsi target snapshots for backup purposes.

    Please make sure your cluster networking is optimized to handle redirected mode access especially when you are using iscsi attached storage.

    DPM 2010 Hyper-V protection Configuring cluster networks for CSV redirected access
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2473194

     


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, May 24, 2011 6:28 PM
    Moderator
  • Hello Mike, 

    Thank you for your reply. I don't see any performance issues except when running a DPM backup on the CSV backed VM. 

    I was looking at that article earlier today. I have 4 dedicated 1GB links (all on different subnets) setup on the cluster -- Heartbeat, Live-Migration, CSV_redirect, and server management (additionally there are 2 iSCSI connections and connections made available to the Hyper-V VMs).

    When i set a CSV storage node to redirect mode, the CSV_redirect network trafic goes to about 8% but the VMs are working well. (Inside the VMs I am testing with IOmeter and get 10-20MB/Sec disk IO without stressing the iSCSI target drives or seeing disk queuing.)

    When i try to backup the VM using DPM the IO drops to .2 MB/Sec while the CSV node is in "backup in progress redirected access" mode.  (When this first starts, the CSV_redirect network has a small 2% blip then runs at near 0%)

    Thank you again for your time.

    Scott
    Tuesday, May 24, 2011 11:47 PM
  •  

    Hi,

    The only thing different between putting the CSV in redirected mode manually and when a DPM backup does it, is that we make a snapshot of the CSV after it gets put in redirected mode.  If you run vssadmin list shadows on the cluster owning the guest while initiating a backup, you should see the snapshot arrive.  If you open process monitor, you can then see the dpmra process read from the snapshot once it's mounted.   You can use performance monitor to see when CSV I/O drops during this process.  It should drop momentarily (no more that 10 seconds) during snapshot creation, then pick up again after the snapshot is created.   


    Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, May 25, 2011 6:10 PM
    Moderator
  • Hi,

    Please check you NIC bindings on your Hyper-V cluster hosts. You need to ensure that the public (domain) network is first in the binding order on all hosts. This should be the NIC that the DPM backup data uses. If that does not resolve the issue try disabling TCP chimney on both the DPM and HYPER-V host server. I had the same problem and this fixed it for me. 


    Microsoft Partner
    • Proposed as answer by rEMOTE_eVENT Tuesday, June 21, 2011 11:56 AM
    Wednesday, June 1, 2011 11:09 PM
  • If the above suggested answer does not help in resolving the thread please re-open it.
    --------------------------------------------------------------------------------
    Regards, Michael V [MSFT] - This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, January 19, 2012 6:28 PM
    Moderator
  • Hi Scott.

    I have exactly the same problem.

    Did you solve the problem?

    Regads
    MHem

    Friday, November 23, 2012 8:39 PM