none
iSCSI Software Target and Differencing Disks

    Question

  • I've been playing with iSCSI Software Target lately and find the differencing disk support a little odd.

    First, when I create a differencing disk, I prefer to make the parent disk read only to protect against accidentally modifying the parent VHD and invalidating all of the differencing disks.  This is often best practice and recommended.

    However, if I try to add a differencing disk against a read-only parent disk to iSCSI Software Target as a device, I get an Access Denied error.  Making the parent VHD writable resolves the issue immediately.  It seems that iSCSI Software Target wants to lock the parent disk also, but in its own way that requires it to be writable.  Is there a reason for that?

    Now, let's take a simple scenario.  I take a parent VHD of Windows Server 2008 R2 and create 10 differencing disks.  Each is attached as a device, assigned to a target, and the machines are booted and are running.  Suddenly I realize that I need an 11th replica of this same server.  However, when I use DISKPART to create to create another differencing disk from the parent, an error message occurs.

    "DiskPart has encountered an error: The process cannot access the file because it is being used by another process."

    If I remove all of the differencing disk devices from iSCSI Software Target, then I can create the 11th differencing disk.

    Unfortunately, this means that in order to add the 11th server, I *MUST* shut down the other 10.

    This seems completely contrary to what would seem to be the primary scenario for differencing disks and iSCSI Software Target, i.e. the provisioning of targets from a standard image as needed.

    Is it really supposed to be this convoluted?  Is there anything that can be done to make this work more intuitively and, well, useful?

    Thanks in advance.

     

    Tuesday, July 5, 2011 5:37 AM

Answers

All replies

  • To answer first question

    1. However, if I try to add a differencing disk against a read-only parent disk to iSCSI Software Target as a device, I get an Access Denied error.  Making the parent VHD writable resolves the issue immediately.  It seems that iSCSI Software Target wants to lock the parent disk also, but in its own way that requires it to be writable.  Is there a reason for that?

     

    VHD formats need metadata to be maintained on both parent and differencing disk to make sure that every parent has knowledge of its child and vice-versa. So parent vhd must be writable before you can create a differencing disk for it.

     

    2. If I remove all of the differencing disk devices from iSCSI Software Target, then I can create the 11th differencing disk.

    Are you creating all earlier differencing vhd with diskpart only and all of them are succeeding?

     

    Do let me know

    Regards

    Satish

     

    Tuesday, July 5, 2011 7:29 AM
  • VHD formats need metadata to be maintained on both parent and differencing disk to make sure that every parent has knowledge of its child and vice-versa. So parent vhd must be writable before you can create a differencing disk for it.

    Are you creating all earlier differencing vhd with diskpart only and all of them are succeeding?

    How can the parent VHD be aware of the child VHD's through metadata if the last updated date/time stamp of the parent VHD never changes.  Also, how could a differencing disk ever be created if the parent VHD were read only?  I'm not seeing any evidence of the behavior that you describe about parent VHD's being aware of their child VHD's through metadata, which would presumably be stored within the parent VHD itself.

    I create all differencing disks with DiskPart, and they all succeed (as long as the parent isn't locked by iSCSI Software Target as I decribed).  I know there are other ways to create child VHD's, such as through WMI, but I have no reason to believe that both don't do exactly the same thing.

    Thanks in advance!

    Brandon

    Tuesday, July 5, 2011 2:54 PM
  • Hi Bradon,

    You were right. Parent disk should be readonly. Otherwise there can be issues while write is going to both parent and differencing which is bad.

     

    I tried your scenario in my local setup. I was able to create child VHD while the parent was readonly.

     

    Do the following

    1) Make Parent VHD Readonly.

    2) Detach Parent VHD using diskpart.

    3) Now Create Child VHD. 

     

    This should succeed. Do let me know if you have any concern

    You can get more detail here

    http://technet.microsoft.com/en-us/library/cc720381(WS.10).aspx

     

    Regarding limitation of 10 I am checking with my colleague. Give us sometime to respond back,

    Regards

    Satish

    Tuesday, July 5, 2011 6:23 PM
  • Hi Bradon,

    You were right. Parent disk should be readonly. Otherwise there can be issues while write is going to both parent and differencing which is bad.

     

    I tried your scenario in my local setup. I was able to create child VHD while the parent was readonly.

     

    Do the following

    1) Make Parent VHD Readonly.

    2) Detach Parent VHD using diskpart.

    3) Now Create Child VHD. 

     

    This should succeed. Do let me know if you have any concern

    You can get more detail here

    http://technet.microsoft.com/en-us/library/cc720381(WS.10).aspx

     

    Regarding limitation of 10 I am checking with my colleague. Give us sometime to respond back,

    Regards

    Satish


    I think you might have missed my original observation.

    I want to make the parent VHD read-only (because it's a best practice and recommended.. and it's smart), and I can create child VHD's with a read-only parent VHD.

    However, I find that I can not attach a child VHD to iSCSI Software Target while it's parent VHD is read-only.  I must uncheck Read Only in order for that to work.

    Furthermore, I find that once I have added my multiple child VHD's to iSCSI Software Target as devices, that I can no longer create ANOTHER child VHD from that parent VHD, because iSCSI Software Target has locked the parent.  In my example, I used 10, but it is an arbitrary number.  I can create as many child VHD's as I want, but once I add one of them to iSCSI Software Target as a device, the parent is now locked in a way that prevents additional child VHD's from being created.  There is no limitation of 10 inherent to the VHD stack.


    Tuesday, July 5, 2011 9:37 PM
  • I tried another scenario with iSCSI Software Target that also does not work without explanation.

    First I created a fixed parent.vhd with a child.vhd and grandchild.vhd.

    Parent.vhd

      +  child.vhd

            + grandchild.vhd

    I can attach the grandchild.vhd in diskpart, but when I try to add it as a device in iSCSI Software Target, i get the following error.

    "The wizard was unable to import one or more virtual disks.  The operation failed with error code 0x80070032.  The request is not supported.."

    Clearly iSCSI isn't using the standard Windows Server 2008 R2 VHD stack and is doing some custom and non-standard things with VHDs assigned as devices.

    Wednesday, July 6, 2011 4:29 AM
  • Well it looks like there's no real solution to the problem.

    My goal is to use iSCSI Software Target as storage for a large collection of machines that are essentially booting off the same OS image.  The base OS image, or "golden" image, will not change, but the individual machines must be able to personalize, or specialize, by writing back to storage without affecting the base image, hence the need for the child differencing disks.  

    As far as I'm concerned, this is the only reason that iSCSI Software Target would implement differencing disk support in the first place.

    The inability to add new differencing disks to a parent while the parent is locked by iSCSI Software Target is a real "what were they thinking" situation.

    The only workaround I can come up with is to produce in advance a large pool of differencing disks.  For example, if I have 50 machines in an image group, go ahead and create 100 differencing disks in advance before putting the first one into service.  Then if the need arises to add additional machines, or to service existing machines (something blows up and a machine must be reset back to the base image), I have spares already available to create new or maintenance existing targets.

    Thanks.

    • Marked as answer by brandon1234 Wednesday, July 6, 2011 12:53 PM
    • Unmarked as answer by brandon1234 Wednesday, July 6, 2011 2:33 PM
    Wednesday, July 6, 2011 12:53 PM
  • it turns out there is a way.  Since iSCSI Software Target locks the parent VHD, it stands to reason that only iSCSI Software Target is capable of creating additional child/differencing VHD's once the parent is added as a device (directly or indirectly).

    But, this feature is poorly documented and exposed.  It is not available through the UI or powershell library.  Instead, it appears the WMI provider is the only means of getting iSCSI Software Target to create child VHD's for you.

    Thanks to Jose Barreto - MSFT for this solution.

    http://blogs.technet.com/b/josebda/archive/2010/10/04/using-powershell-and-the-the-iscsi-target-3-3-wmi-classes-to-create-a-differencing-vhd-for-os-boot.aspx

     

    • Marked as answer by brandon1234 Wednesday, July 6, 2011 2:33 PM
    Wednesday, July 6, 2011 2:33 PM
  • Brandon,

    Looks like you have already found the answer.

    Your secnario for iSCSI boot is supported with the iSCSI Software Target. Make the golden image as the base VHD, and create differencing VHDs for each machine to customize. The way to create differencing VHD is to use WMI. you can find the script sample here:

    http://gallery.technet.microsoft.com/scriptcenter/f2a28ec4-9aa6-4b27-80a8-1282296b2e4f

    Regarding to making the parent VHD disk readonly question, the iSCSI Target will require it read-write, and internally, the target will access the parent VHD readonly. You should not use diskpart to create new diff VHDs, with WMI, you can create additional diffs while others are running online.

    One of the post you mentioned about parent-> child -> grandchild, the iSCSI target supports one level diff, i.e. parent-> child.

    Hope this helps.

    Wednesday, July 6, 2011 3:30 PM