none
Server 2016 Data Center Hyper-V cluster with S2D questions

    Question

  • Hi,

    I have two identical physical servers: 8 core Xeon with HT, 96 GB RAM, 2 x OS SSDs in RAID 1, 4 x 4 TB HDs, 2 x 180 GB SSDs, dual 10GbE NICs, and 6 x 1 Gbps NICs. Each server has Server 2016 Data Center with Desktop Experience installed on the RAID 1 SSDs.

    I want to configure this hardware into a Hyper-V cluster that does fail-over whilst also using S2D as the data storage mechanism, and I have some questions. The 4 x 4 TB will form the storage area for S2D on each server with the 2 x 180 GB SSDs being the S2D cache. These 6 HDs are configured as pass-through in each server's BIOS.

    1) Does the Server 2016 Data Center OS that are acting as the Hyper-V hosts have to be attached to a domain to do Hyper-V clustering? I really want to set this up in the lab before getting it to the customer and then attaching the Hyper-V guests to the domain. But I'd rather leave the Hyper-V hosts off the domain.

    2) Am I right in thinking I can achieve what I want with Hyper-V clustering and S2D? I imagine the S2D will give me storage space on each physical server through the Hyper-V host, and also replicate itself over to the other server. So when I have my Hyper-V guests stored on the S2D space I will have a resilient infrastructure that can quickly deal with a physical server going down (the dead server's data will be on the other server's S2D space, and the Hyper-V host easily be able to spin up the VMs that were running on the dead server).

    3) When setting this up, should I be configuring the Hyper-V clustering first, or the S2D first?

    4) I have 2 x 10 GbE NICs on each server, and a switch with 4 x 10 GbE ports on. Should I use both 10 GbE ports on the Hyper-V hosts for S2D replication. or one for S2D replication and one for something else like RDMA?

    There's not that much good documentation out there atm. A lot of stuff refers to technical previews of Server 2016 which has since changed. A lot talks about CSV but not S2D. The stuff that talks about S2D doesn't talk about Hyper-V failover, etc.

    Thank you for reading.

    Thursday, January 19, 2017 9:35 PM

All replies

  • "8 core Xeon with HT, 96 GB RAM, 2 x OS SSDs in RAID 1, 4 x 4 TB HDs, 2 x 180 GB SSDs, dual 10GbE NICs, and 6 x 1 Gbps NICs. "

    You do not mention how the HDs and SSDs are connected.  You mention RAID 1 for the OS SSDs.  The HDs and SSDs must be connected via an HBA.  The CANNOT be connected via a RAID controller.

    "Does the Server 2016 Data Center OS that are acting as the Hyper-V hosts have to be attached to a domain to do Hyper-V clustering? "

    Technically, no.  Windows Server 2016 introduced the concept of workgroup clustering.  But, if you want to be able to easily manage the cluster, it is HIGHLY recommended to have it as a member of a domain.  All the management tools are designed to work in a domain out of the box.  You have to do a lot of work to manage a cluster in a workgroup environment.

    "Am I right in thinking I can achieve what I want with Hyper-V clustering and S2D?"

    Basically you are correct.  With a two-node configuration, S2D will automatically configure the HDs into a mirror across both systems.  So it is not 'replicating' itself by the definition of replication, but data is being mirrored, which you could consider a type of replication.  But I try to avoid saying those two are the same thing.

    "A lot talks about CSV but not S2D. "

    You will use CSV on top of S2D.

    "There's not that much good documentation out there "

    Planning Considerations:

    https://technet.microsoft.com/windows-server-docs/storage/storage-spaces/plan-volumes

    How to Deploy:

    https://technet.microsoft.com/windows-server-docs/storage/storage-spaces/create-volumes

    https://technet.microsoft.com/windows-server-docs/storage/storage-spaces/understand-the-cache?f=255&MSPPError=-2147217396

    Step-by-step

    https://technet.microsoft.com/en-us/windows-server-docs/storage/storage-spaces/hyper-converged-solution-using-storage-spaces-direct


    . : | : . : | : . tim


    Thursday, January 19, 2017 11:16 PM
  • best starting point is always to use a certified stack and don't play lego if you want to use it for production

    you also need to double check your Hardware with Storage Spaces Direct hardware requirements

    https://technet.microsoft.com/en-us/windows-server-docs/storage/storage-spaces/storage-spaces-direct-hardware-requirements

    - don't use 1 GB NICs

    - the size of SSDs is not working, caching needs to be at least 10% of the capacity tier

    - guessing 180 GB SSD is not a enterprise SSD means will be dead pretty soon

    https://blogs.technet.microsoft.com/filecab/2016/11/18/dont-do-it-consumer-ssd/

    also watch the Ignite Sessions around S2D to get a better view.

    cheers

    Udo

    Friday, January 20, 2017 12:12 PM
  • Well, Udo is right.

    Most probably your S2D cluster will not work properly unless you have a suitable hardware configuration. Furthermore, a 2-node S2D cluster is quite useless for production since does not have all the tasty features that are possible starting with 4+ hosts. The worthiest thing is that if you lose a single drive in each server at the same moment of time you will loose all the data on your S2D storage array since there is no RAID at the bottom and thus no redundancy. 

    In your configuration, I would recommend a more hardware agnostic solution capable of working on top of hardware RAID like HP VSA https://h20392.www2.hpe.com/portal/swdepot/displayProductInfo.do?productNumber=VSA1TB-S or StarWind Virtual SAN https://www.starwindsoftware.com/starwind-virtual-san that do almost the same that S2D but have above-mentioned benefits. 

    • Proposed as answer by Alex Bykovskyi Monday, January 23, 2017 9:27 AM
    Friday, January 20, 2017 6:37 PM
  • You do not mention how the HDs and SSDs are connected.  You mention RAID 1 for the OS SSDs.  The HDs and SSDs must be connected via an HBA.  The CANNOT be connected via a RAID controller.

    Good catch. I had set up the PERC H730 RAID controller as pass-through, but this isn't good enough. After setting it to HBA, I could no longer configure the Hyper-V OS 180 GB SSDs as RAID1 (makes sense), so after a call to Dell it appears that even setting the PERC H730 into HBA mode isn't good enough for S2D on Server 2016 and they're shipping me the RAID cards they should have spec'd when I was talking to them about this..

    best starting point is always to use a certified stack and don't play lego if you want to use it for production

    I have two new Dell T630s which Dell say are up to the job so I won't argue with them... (yet..). They were told of the S2D requirements so I expect the servers they sold me are going to be up to the job, along with the two switches that both have 4 10gbps ports

    the size of SSDs is not working, caching needs to be at least 10% of the capacity tier

    As I understand it (happy to be corrected) the SSDs in an S2D configuration are not for caching in the traditional sense. They will form the performance tier, so any hot S2D slabs get moved (not copied) to the SSDs and actually live there (rather than being copied from the slow HDs to the SSDs and then any changes made to the slabs on the SSDs getting written back to the slow HD tier). Is the 10% thing actually enforced like in VMWare? But you pointed out a mistake I made, the two 480 GB SSDs are for the S2D cache, and the two 180 GB SSDs are for RAID1 for the Hyper-V host OS.

    guessing 180 GB SSD is not a enterprise SSD means will be dead pretty soon

    They're Intel SSDs with on-board battery backup and (afaik) are considered enterprise SSDs. I do plan to double-check this tho (same as the 480 GB SSDs). Interesting link about the consumer SSDs - thanks!

    don't use 1 GB NICs

    The two Dell T630s have dual 10 Gb fibre ports, and the two Dell switches both have 4 x 10 Gb fibre ports so aiui this is good enough for a 2 node S2D setup. Your point is well taken tho.

    a 2-node S2D cluster is quite useless for production since does not have all the tasty features that are possible starting with 4+ hosts. The worthiest thing is that if you lose a single drive in each server at the same moment of time you will loose all the data on your S2D storage array

    This is well noted. Whilst there is still a chance of data loss and downtime, this solution is significantly better than the current infrastructure (single aging SBS server) and I am now more aware of its shortcomings.

    Tim, Udo, Taras, thank you for taking time to reply and learn me some stuff :) Once I get these new RAID cards that can do proper HBA for S2D I am looking forward to setting up my first Hyper-V S2D solution!




    Monday, January 23, 2017 6:50 PM
  • "it appears that even setting the PERC H730 into HBA mode isn't good enough for S2D on Server 2016 and they're shipping me the RAID cards they should have spec'd when I was talking to them about this"

    Hopefully they are not shipping you RAID cards.  You need HBA.  RAID cards are not supported. Period.  At least for data drives.  If you have a separate RAID card that is used just for OS and then an HBA for data, that works.


    . : | : . : | : . tim

    Tuesday, January 24, 2017 12:08 AM
  • Hi,
    Are there any updates on the issue?
    You could mark the reply as answer if it is helpful.
    Best Regards,
    Leo

    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, February 7, 2017 1:59 AM
    Moderator