none
Server 2012 and JBOD's!

    Question

  • Hello everyone,

    I am busy contemplating using two Windows Server 2012 Servers to connect to an external JBOD using external mini-sas cables (the 8088 variety).  This is unfortunately a completely new area to me - I have never dealt with JBOD's and SAS based cabled systems.  I have always dealt with Dell iSCSI based storage - this time I am looking at a custom built, but cheaper JBOD solution using Server 2012.

    What I want is to cable the two Windows Server 2012 Servers to the JBOD in a way that will make it resilient, and fault tolerant in that if I have for example a Virtual Machine called VM-1 on Server-A, and if Server-A has a serious problem and shuts down, I would want VM-1 to move over to Server-B.  I presume that I can use Storage Spaces here...?

    Now I am not sure here, but I think that the JBOD I am looking at will require 2 SAS Expander cards as it seems that most SAS Expander cards have two SFF-8088 external ports, but only one of those ports can be used to connect to a Server as the In and Out ports are used for different things?

    I guess what I want to know is, do I need 2 SAS Expander cards in the JBOD?  What sort of SAS card do I need to install on the Windows Server 2012 Servers?  Can I achieve what I want easily enough?

    This is the JBOD I am looking at: http://www.xcase.co.uk/48-drive-jbod-p/jbod-16.htm, and it comes with the following SAS Expander card: http://www.xcase.co.uk/product-p/expander%20-ck22803.htm.

    If anyone can give any advice on this, it would be greatly appreciated :).

    Thursday, October 04, 2012 2:59 PM

Answers

  • Hello,

    A very new unmapped area but here's some information I've gathered. It's hard to give explicit hardware advice as this is all very new to us all and the OEMs are not exactly storming out of the door yet with offers specific to storage spaces.

    Hardware that has been used in several setups for testing by MSFT: 

    • RAID Inc 4U EBOD
    • Raid Inc 2U EBOD
    • DataON DNS-1640 2U-24 JBOD

    Don't forget that Storage space is designed to be used with HBAs that do NOT provide any RAID capabilities. A prime example of such a card are the LSI 9205 HBAs.
     
    I know that MSFT is working on a certification program for Storage Spaces their storage partners (the usual supects enriched with the smaller playerlisted above), but notthing public yet. I hope from October 26th on this will change but I'm not holding my breath, could be longer :-)

    For DELL I know that you can use the  MD12xx series with Storage Spaces for stand-alone configurations. For clustered configurations, you’ll need to contact Dell to be sure But as far as I know it can work but your limited to 4 bays max.Don't forget that for shared storage you'll need NL-SAS or SAS.You'll also need to check if they support clustering. Normally it's not used for that but it is possible to build a solution, at least they used the MD1220 in their labs for buiding a SOFS cluster with storage spaces. Lool here: http://channel9.msdn.com/events/TechEd/Europe/2012/WSV210

    See slide 7/8 for a 4 node cluster leveraging RDMA and a four node cluster consisting of 2*2 servers attached to the MD1200. Note, not all nodes see all clusters. They used RDMA with CSV in redirected moden to get this results. Straight out of a BUILD example I think.  Kinda funcky and you need to wrap your head around it. I do not know if you can use the MD12XX for shared storage directly, but I didn't think so. Normaly you put an MD3200s in front but than you get into HW raid controllers in the JBOD, which is a no go for Storage Spaces. Don't forget either that storage space in a cluster means redirected mode per definition => Nature of the beast and as you need storage redundancy you need to use Mirrored spaces as that's the only redundant space config currently supported for clustering AFAIK. Also see http://blogs.msdn.com/b/clustering/archive/2012/06/02/10314262.aspx

    At TechEd Europe they also mentioned the still had to work on the firmware of the MD1200 as it was not optimized for Storage Spaces (makes sense, all OEM where RAID controller focussed)

    See slide 14: Dell – Best  Storage Spaces JBOD solutions (cost and performance)Dual 6GB SAS HA controllers connected to multiple MD1200 Powervault units - Performance of Spaces appears to be non-optimul on current generation Dell MD range (pre-PR Firmware).

    Hope this helps a bit,

    Didier Van Hoye

    http://workinghardinit.wordpress.com

    Thursday, October 04, 2012 6:02 PM
  • Hi,

    Windows Server 2012 Hyper-V Replica provides failover capability to another location (datacenter, branch office, campus building, floor or rack).  It is not a replacement for high availability that's provided by a Hyper-V cluster, which with Windows Server  2012 is paving the way to continuous availability. But unless the customer knows really well what he is getting form the technology I'd be carefull to suggest this as high availability. If fast recovery is only requirement, I'd say, that's a no brainer when done right, but with regards to high availability I'd say you need to do clustering. Storage spaces is a supported option for this. While you have a lot of options now with * Live Migrations when the hos dies you'd need to go trigger a failover scenario where a cluster would give you automatic restarts of the VMs on the other node. Just trying to make that expectations match the technology.

    Best regards,

    Didier Van Hoye

    http://workinghardinit.wordpress.com

    Replica is one way ticket... Also it does not keep VM content in-sync. At least 5 minutes of recently updated content will be lost. So... To run VMs in an HA mode (no data loss) and to do guest VM clustering (no service downtime) OP needs shared storage. It's a pity Windows / Hyper-V cannot mirror HA DAS w/o third-party software. Like other hypervisors do.

    -nismo

    Saturday, October 06, 2012 6:42 PM

All replies

  • Hello,

    A very new unmapped area but here's some information I've gathered. It's hard to give explicit hardware advice as this is all very new to us all and the OEMs are not exactly storming out of the door yet with offers specific to storage spaces.

    Hardware that has been used in several setups for testing by MSFT: 

    • RAID Inc 4U EBOD
    • Raid Inc 2U EBOD
    • DataON DNS-1640 2U-24 JBOD

    Don't forget that Storage space is designed to be used with HBAs that do NOT provide any RAID capabilities. A prime example of such a card are the LSI 9205 HBAs.
     
    I know that MSFT is working on a certification program for Storage Spaces their storage partners (the usual supects enriched with the smaller playerlisted above), but notthing public yet. I hope from October 26th on this will change but I'm not holding my breath, could be longer :-)

    For DELL I know that you can use the  MD12xx series with Storage Spaces for stand-alone configurations. For clustered configurations, you’ll need to contact Dell to be sure But as far as I know it can work but your limited to 4 bays max.Don't forget that for shared storage you'll need NL-SAS or SAS.You'll also need to check if they support clustering. Normally it's not used for that but it is possible to build a solution, at least they used the MD1220 in their labs for buiding a SOFS cluster with storage spaces. Lool here: http://channel9.msdn.com/events/TechEd/Europe/2012/WSV210

    See slide 7/8 for a 4 node cluster leveraging RDMA and a four node cluster consisting of 2*2 servers attached to the MD1200. Note, not all nodes see all clusters. They used RDMA with CSV in redirected moden to get this results. Straight out of a BUILD example I think.  Kinda funcky and you need to wrap your head around it. I do not know if you can use the MD12XX for shared storage directly, but I didn't think so. Normaly you put an MD3200s in front but than you get into HW raid controllers in the JBOD, which is a no go for Storage Spaces. Don't forget either that storage space in a cluster means redirected mode per definition => Nature of the beast and as you need storage redundancy you need to use Mirrored spaces as that's the only redundant space config currently supported for clustering AFAIK. Also see http://blogs.msdn.com/b/clustering/archive/2012/06/02/10314262.aspx

    At TechEd Europe they also mentioned the still had to work on the firmware of the MD1200 as it was not optimized for Storage Spaces (makes sense, all OEM where RAID controller focussed)

    See slide 14: Dell – Best  Storage Spaces JBOD solutions (cost and performance)Dual 6GB SAS HA controllers connected to multiple MD1200 Powervault units - Performance of Spaces appears to be non-optimul on current generation Dell MD range (pre-PR Firmware).

    Hope this helps a bit,

    Didier Van Hoye

    http://workinghardinit.wordpress.com

    Thursday, October 04, 2012 6:02 PM
  • Wow, thanks for all that useful information :).  If anyone has any details on how to custom build the hardware rather than using the likes of Dell storage, then that would be great.  Would be useful to know what hardware I would need in a JBOD chassis and also what I would need in the Windows 2012 Servers to get everything working as I would want.
    Friday, October 05, 2012 8:02 AM
  • Have you considered using strictly internal disk on both servers and utilizing the features in Hyper-V replica for high availability?
    Friday, October 05, 2012 2:29 PM
  • Hi,

    Windows Server 2012 Hyper-V Replica provides failover capability to another location (datacenter, branch office, campus building, floor or rack).  It is not a replacement for high availability that's provided by a Hyper-V cluster, which with Windows Server  2012 is paving the way to continuous availability. But unless the customer knows really well what he is getting form the technology I'd be carefull to suggest this as high availability. If fast recovery is only requirement, I'd say, that's a no brainer when done right, but with regards to high availability I'd say you need to do clustering. Storage spaces is a supported option for this. While you have a lot of options now with * Live Migrations when the hos dies you'd need to go trigger a failover scenario where a cluster would give you automatic restarts of the VMs on the other node. Just trying to make that expectations match the technology.

    Best regards,

    Didier Van Hoye

    http://workinghardinit.wordpress.com

    Friday, October 05, 2012 3:17 PM
  • Yep. I didn't say Replica would replace a traditional cluster, just proposing alternate considerations should he be unfamiliar with the newer technologies. In addition, HA means different things to different people. Since it sounded like he was working with a more strict budget, Replica could allow him to save storage cost while still maintaining the mobility of his virtual machines. Granted the failover would be a manual process, it might be worth saving the cost of the external array and all the necessary cards/cables that go along with it.
    Friday, October 05, 2012 5:37 PM
  • I understand. As Storage Spaces are one way to get redundant & performant storage at flexible price points I'd say for the cost of a JBOD, the SAS controllers & the disks he might get to do both on a budget :-)  That would be neat!
    Friday, October 05, 2012 7:01 PM
  • Hi,

    Windows Server 2012 Hyper-V Replica provides failover capability to another location (datacenter, branch office, campus building, floor or rack).  It is not a replacement for high availability that's provided by a Hyper-V cluster, which with Windows Server  2012 is paving the way to continuous availability. But unless the customer knows really well what he is getting form the technology I'd be carefull to suggest this as high availability. If fast recovery is only requirement, I'd say, that's a no brainer when done right, but with regards to high availability I'd say you need to do clustering. Storage spaces is a supported option for this. While you have a lot of options now with * Live Migrations when the hos dies you'd need to go trigger a failover scenario where a cluster would give you automatic restarts of the VMs on the other node. Just trying to make that expectations match the technology.

    Best regards,

    Didier Van Hoye

    http://workinghardinit.wordpress.com

    Replica is one way ticket... Also it does not keep VM content in-sync. At least 5 minutes of recently updated content will be lost. So... To run VMs in an HA mode (no data loss) and to do guest VM clustering (no service downtime) OP needs shared storage. It's a pity Windows / Hyper-V cannot mirror HA DAS w/o third-party software. Like other hypervisors do.

    -nismo

    Saturday, October 06, 2012 6:42 PM