none
Storage Spaces Direct Tiers/Cache

    Question

  • Hi Everyone,

    I am hoping someone out there can better explain some of the configuration options in S2D.  I have a test lab setup with 4 servers each with 2x 256GB SSD and 2x 1TB HDD.  I am attempting to do a mirror over these following steps here: https://technet.microsoft.com/en-us/library/mt126109.aspx

    What does not quite make sense to me is what options should be used.  I would think just creating a Virtual Disk with a SSD tier and HDD tier would always write to SSD first (thus avoiding cache) and move hot data to SSD for read operations (again avoiding cache).  No matter what I do it does not seem to want to let me configure this:

    New-Volume: Not Supported
    Extended Information: There are not enough eligible physical resources in the storage pool to create the specified virtual disk configuration.
    
    Recommended Actions: 
    -Choose a combination of FaultDomainAwareness and NumberOfDataCopies (or PhysicalDiskRedundancy) supported by the storage pool.
    -Chose a value of for NumberOfColumns that is less than or equal to the number of physical disks in the storage fault domain selected for the virtual disk.

    I can try with or without tiers, new-VirtualDisk does the same thing.  I am able to create a vdisk/volume through the clustering GUI, but only one without tiers.

    I guess the questions I have at this point:
    Why would new-volume be failing?
    Would using all SSDs as Journal and just creating a mirror space from the HDDs really use the SSDs effectively?

    Wednesday, January 13, 2016 3:19 PM

All replies

  • Hi Thildemar,

    Thanks for your post.

    >I would think just creating a Virtual Disk with a SSD tier and HDD tier would always write to SSD first (thus avoiding cache) and move hot data to SSD for read operations (again avoiding cache).  

    According to your description, it seems you need the functional Storage Spaces write-back cache. This capability is already in Windows Server 2012 R2 and Windows 8.1. And for simple and mirrored spaces, a per-space persistent cache can be configured which uses available SSD capacity in the pool to stage these small write bursts, which are then destaged to the HDDs in a more optimal sequential manner.

    You could refer to the articles below.

    https://msdn.microsoft.com/en-us/library/windows/hardware/dn567634%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396#wbc

    http://blogs.technet.com/b/askpfeplat/archive/2013/10/21/storage-spaces-how-to-configure-storage-tiers-with-windows-server-2012-r2.aspx

    Here are the requirements for 2012R2 WBC.

    https://technet.microsoft.com/en-us/library/dn387076.aspx?f=255&MSPPError=-2147217396

    For the error when creating new-volume, it seems there is insufficient capacity to create the the desired volume.

    And please refer to the blog shared by Clausjor, it may releated to at least four nodes to do parity (aka erasure coding)

    http://blogs.technet.com/b/clausjor/archive/2015/11/19/storage-spaces-direct-in-technical-preview-4.aspx

    Best Regards,

    Mary Dong


    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, January 14, 2016 7:56 AM
    Moderator
  • Hi Mary,

    Thanks for the info, but I think we are missing something.

    Write caching is great, but looking at how S2D handles tiers in TP4 it would seem that the write cache becomes redundant.  If all writes are already written to the hot tier as shown in that article by Clausjor then why would we need write cache or journal at all (assuming your hot tier is already SSD)?  Would we not just want to configure the SSD tier and HDD tier and let the system do it's thing?

    The volume error is not related to capacity or node count.  In my setup I have 4 nodes, each with 2x 256GB SSD (total 8 or 2TB) and 2x 1TB HDD (total 8 or 8 TB).  I was attempting to create a volume with 100GB SSD tier and 900GB HDD tier using 2 copy mirror.  Should be plenty of space and nodes for that operation.  If you look at the comments on that clausjor post it would seem I am not the only one hitting that error.  I also got the same error after rebuilding the lab from scratch on VMs with the same configuration.

    Thursday, January 14, 2016 3:26 PM
  • Here are the commands I am attempting along with a listing of the relevant setup:

    PS C:\WINDOWS\system32> Enter-PSSession s2dnode1
    [s2dnode1]: PS C:\Users\user\Documents> Get-StorageTier
    
    
    ObjectId               : {1}\\S2DTestCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StorageTier.ObjectId="{
                             36f73fd8-7dc6-4d6c-8b8f-c1e054bdc4d6}:ST:{07f2ca16-e559-428b-b24f-c641cca5b66b}{1e2aeb46-57c6-
                             43d4-861c-45df09c4bed2}"
    PassThroughClass       :
    PassThroughIds         :
    PassThroughNamespace   :
    PassThroughServer      :
    UniqueId               : {1e2aeb46-57c6-43d4-861c-45df09c4bed2}
    Description            :
    FriendlyName           : Microsoft_MRT_Template_SSD_Mirror
    Interleave             : 262144
    MediaType              : SSD
    NumberOfColumns        : Auto
    PhysicalDiskRedundancy : 1
    ResiliencySettingName  : Mirror
    Size                   : 0
    PSComputerName         :
    
    ObjectId               : {1}\\S2DTestCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StorageTier.ObjectId="{
                             36f73fd8-7dc6-4d6c-8b8f-c1e054bdc4d6}:ST:{07f2ca16-e559-428b-b24f-c641cca5b66b}{4ae4a4d7-fc6c-
                             40b5-8b8c-6b6e9bd3595c}"
    PassThroughClass       :
    PassThroughIds         :
    PassThroughNamespace   :
    PassThroughServer      :
    UniqueId               : {4ae4a4d7-fc6c-40b5-8b8c-6b6e9bd3595c}
    Description            :
    FriendlyName           : Microsoft_MRT_Template_HDD_Mirror
    Interleave             : 262144
    MediaType              : HDD
    NumberOfColumns        : Auto
    PhysicalDiskRedundancy : 1
    ResiliencySettingName  : Mirror
    Size                   : 0
    PSComputerName         :
    
    ObjectId               : {1}\\S2DTestCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StorageTier.ObjectId="{
                             36f73fd8-7dc6-4d6c-8b8f-c1e054bdc4d6}:ST:{07f2ca16-e559-428b-b24f-c641cca5b66b}{65589f3f-709c-
                             442b-847d-21a56c263a9d}"
    PassThroughClass       :
    PassThroughIds         :
    PassThroughNamespace   :
    PassThroughServer      :
    UniqueId               : {65589f3f-709c-442b-847d-21a56c263a9d}
    Description            :
    FriendlyName           : SSDTier
    Interleave             : 262144
    MediaType              : SSD
    NumberOfColumns        : 2
    PhysicalDiskRedundancy : 1
    ResiliencySettingName  : Mirror
    Size                   : 0
    PSComputerName         :
    
    ObjectId               : {1}\\S2DTestCluster\root/Microsoft/Windows/Storage/Providers_v2\SPACES_StorageTier.ObjectId="{
                             36f73fd8-7dc6-4d6c-8b8f-c1e054bdc4d6}:ST:{07f2ca16-e559-428b-b24f-c641cca5b66b}{b4bddd37-7576-
                             4208-a16c-325a1bf9dec9}"
    PassThroughClass       :
    PassThroughIds         :
    PassThroughNamespace   :
    PassThroughServer      :
    UniqueId               : {b4bddd37-7576-4208-a16c-325a1bf9dec9}
    Description            :
    FriendlyName           : HDDTier
    Interleave             : 262144
    MediaType              : HDD
    NumberOfColumns        : 4
    PhysicalDiskRedundancy : 1
    ResiliencySettingName  : Mirror
    Size                   : 0
    PSComputerName         :
    
    
    
    [s2dnode1]: PS C:\Users\user\Documents> Get-StoragePool
    
    FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
    ------------ ----------------- ------------ ------------ ----------
    Primordial   OK                Healthy      True         False
    S2D          OK                Healthy      False        False
    Primordial   OK                Healthy      True         False
    
    
    [s2dnode1]: PS C:\Users\user\Documents> Get-StoragePool s2d|Get-PhysicalDisk |ft FriendlyName, Size, CanPool, MediaTy
    pe, Usage, EnclosureNumber
    
    FriendlyName               Size CanPool MediaType Usage       EnclosureNumber
    ------------               ---- ------- --------- -----       ---------------
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk  272998858752   False SSD       Auto-Select
    Msft Virtual Disk 1071862775808   False HDD       Auto-Select
    
    
    [s2dnode1]: PS C:\Users\user\Documents> $ssdt = Get-StorageTier ssdtier
    [s2dnode1]: PS C:\Users\user\Documents> $hddt = Get-StorageTier hddtier
    [s2dnode1]: PS C:\Users\user\Documents> New-Volume -StoragePoolFriendlyName s2d -FriendlyName s2dtiered -FileSystem C
    SVFS_ReFS -StorageTiers $ssdt,$hddt -StorageTierSizes 100GB,500GB
    New-Volume : Not Supported
    Extended information: There are not enough eligible physical resources in the storage pool to create the specified
    virtual disk configuration.
    Recommended Actions:
    - Choose a combination of FaultDomainAwareness and NumberOfDataCopies (or PhysicalDiskRedundancy) supported by the
    storage pool.
    - Choose a value for NumberOfColumns that is less than or equal to the number of physical disks in the storage fault
    domain selected for the virtual disk.
    Activity ID: {1228f54e-4a47-4669-a41b-bc079f3a96b6}
    At line:1 char:1
    + New-Volume -StoragePoolFriendlyName s2d -FriendlyName s2dtiered -File ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (StorageWMI:ROOT/Microsoft/..._StorageCmdlets) [New-Volume], CimExcept
       ion
        + FullyQualifiedErrorId : StorageWMI 1,New-Volume
    

    Thursday, January 14, 2016 5:18 PM
  • More info:

    I tried creating some disks in the failover cluster GUI.  None of them will actually create (same error as above), but interestingly the free space calculations are way off for everything except double resiliency settings (3 way mirroror double parity).  For example this is what i get when I go through the wizard requesting a tiered disk, both tiers mirrored, both with two-way mirror, same disks/pool/etc from above (Note the tiny free space numbers):

    If I re-run with three-way mirror the calculations are more what I would expect, but the disk creation still fails.

    Thursday, January 14, 2016 8:19 PM
  • Hi,

    I have exactly the same Issue (but with a smaller testlab),

    I tried to rename the disks after HP_StoragePool but before HP_TieredSpace and set the –MediaType for all disks, but i still get the error (with, or without tiering). I also tried other configurations (2*ssd, 2*hdd with mediatype tag) and smaller sizes.

    My test Lab is a HP 360 G7 with one small SATA-SSD 64GB and seven SAS HDD 300GB

    It does not work in GUI nor in Powershell, here my test commands:

    New-VirtualDisk -StoragePoolFriendlyName “HP_StoragePool” -FriendlyName "HP_TieredSpace" -StorageTiers @($ssd_tier, $hdd_tier) -StorageTierSizes @(50GB, 1660GB) -ResiliencySettingName simple -WriteCacheSize 2GB
    
    
    New-VirtualDisk -StoragePoolFriendlyName HP_StoragePool -FriendlyName HP_NonTieredSpace -Size 1660GB -ResiliencySettingName Parity -ProvisioningType Fixed -WriteCacheSize 40GB
    

    I am doing this in Hyper-V Server 2016 TP4 from a USB stick-boot, what was itself hard to setup. (ESXi is easy as cake)

    If disabeling the raid controller in bios (I dont have another card) will not help, I guess I will have to use ESXi.

    Maybe I try the same thing on Hyper-V Server 2012R2, just for fun.

    (I need deep Virtualization, and Hyper-V in Hyper-V does not work, but Hyper-V in iESXI does. Without that i cannot run my Mobile Device Debug Targets; and physical ones, USB within Hyper-V, good luck with that.)

    My excact error message:

    >> New-VirtualDisk -StoragePoolFriendlyName HP_StoragePool -FriendlyName HP_NonTieredSpace -Size 1660GB -ResiliencySetti
    ngName Parity -ProvisioningType Fixed -WriteCacheSize 40GB
    New-VirtualDisk : Failed to run CIM method CreateVirtualDisk on the MSFT_StoragePool (ObjectId =
    "{1}\\HP2016\root/Microsoft/Windows/Stor...) CIM object.  CIM array cannot contain null elements.
    Parameter name: value
    At line:1 char:1
    + New-VirtualDisk -StoragePoolFriendlyName “HP_StoragePool” -FriendlyNa ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (MSFT_StoragePoo...indows/Stor...):CimInstance) [New-VirtualDisk], CimJobE
       xception
        + FullyQualifiedErrorId : CimJob_ArgumentException,New-VirtualDisk
    
    New-VirtualDisk : Not Supported
    Extended information: There are not enough eligible physical resources in the storage pool to create the specified
    virtual disk configuration.
    Recommended Actions:
    - Choose a combination of FaultDomainAwareness and NumberOfDataCopies (or PhysicalDiskRedundancy) supported by the
    storage pool.
    - Choose a value for NumberOfColumns that is less than or equal to the number of physical disks in the storage fault
    domain selected for the virtual disk.
    Activity ID: {76068b30-ca04-43b7-91c2-816d37aac570}
    At line:2 char:1
    + New-VirtualDisk -StoragePoolFriendlyName HP_StoragePool -FriendlyName ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (StorageWMI:ROOT/Microsoft/...SFT_StoragePool) [New-VirtualDisk], CimE
       xception
        + FullyQualifiedErrorId : StorageWMI 1,New-VirtualDisk

    (one small SSD 64GB Sata + seven HDD 300GB SAS)
    Tuesday, March 08, 2016 9:50 PM
  • Sometimes a few hours of sleep do help.

    I dont know excactly why, but now it seems to work for me. 

    I created a virtual tiered drive, with, like 50 (of 60) GB SSD and 1600 (of 1660) GB HDD.

    That worked (yesterday it did not, but i did a server restart this morning)

    After thex I maxed the sizes (in the GUI on virtual drives), and had the result i wanted.

    Time to test other things.

    Wednesday, March 09, 2016 9:39 AM
  • Honestly the best piece of advice on this page.  Same issue but with 2016 Essentials with very little to go on. Added all my disks to the pool and slept on it. Was able to create a VHD in the morning doing the same thing as the night before (tiered 2x mirrored). Go figure.  <g class="gr_ gr_385 gr-alert gr_gramm gr_run_anim Grammar multiReplace" data-gr-id="385" id="385">Only</g> thing I did differently was to make each size of the tiered options smaller. Had two 256 gig SSD but entered "200", Rounded down to the nearest hundredth gig also on the platters.



    Staple Bench Computers

    Friday, November 18, 2016 6:04 PM
  • There is definitely still some cases where doing the exact same thing with S2D produced two results when creating disks/volumes.  The release build has been a lot better, but still could use some gui vs powershell cleanup.
    Friday, November 18, 2016 6:32 PM
  • I got mine working with but  the disk does not show the columns and it aslo work in the gui if you change it to those settings again still no columns shown or number of data copies and says provisioning unknow

    #The first one does not work
    
    $ssd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName SSD_Tier -MediaType SSD
    $hdd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName HDD_Tier -MediaType HDD
    $vd1 = New-VirtualDisk -StoragePoolFriendlyName Zodiac -FriendlyName Zodiac -StorageTiers @($ssd_tier, $hdd_tier) -StorageTierSizes @(1.86TB, 8.18TB) --ResiliencySettingName Mirror -NumberOfDataCopies 2 -NumberOfColumns 3 -WriteCacheSize 5GB
    
    #This worked and created the file however 
    $ssd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName SSD_Tier -MediaType SSD
    $hdd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName HDD_Tier -MediaType HDD
    $vd1 = New-VirtualDisk -StoragePoolFriendlyName Zodiac -FriendlyName Zodiac -StorageTiers @($ssd_tier, $hdd_tier) -StorageTierSizes @(1TB, 8TB) -ResiliencySettingName Mirror -NumberOfDataCopies 2 -NumberOfColumns 3 -WriteCacheSize 5GB




    Medric Castle

    Wednesday, February 01, 2017 8:05 PM
  • I tried it a few different way and it all does the same thing so you cant use all of the SSD's or the HDD's which seems strange so i am not sure if i want to keep my data on this crap. to many unknowns as to why this does not work as it should or doc to explaine it properly
    $ssd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName SSD_Tier -MediaType SSD
    $hdd_tier = New-StorageTier -StoragePoolFriendlyName Zodiac -FriendlyName HDD_Tier -MediaType HDD
    $vd1 = New-VirtualDisk -StoragePoolFriendlyName Zodiac -FriendlyName Zodiac -StorageTiers @($ssd_tier, $hdd_tier) -StorageTierSizes @(1TB, 8TB) -ResiliencySettingName Mirror -ProvisioningType Fixed -NumberOfDataCopies 2 -NumberOfColumns 3 -WriteCacheSize 24GB


    Medric Castle

    Wednesday, February 01, 2017 8:32 PM