none
Clustered Storage Spaces - issue with SAS interposers and all SATA drives (HDD and SSD)

    Question

  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.




    • Edited by scott18ss Friday, February 22, 2013 3:47 AM Formatting
    Friday, February 22, 2013 3:41 AM

Answers

  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.

    Scott18ss,

    I can confirm the LSI Interposer's have several problems -- as far as I can tell LSI is ignoring the issues and/or only playing lip-service to solving the problem.

    We purchased 16 LSI Interposers (OEM'd by Supermicro) in the hope of using them with Crucial M4 512GB SATA SSD, as part of a Windows 2012 server cluster (providing Tier 1 cluster storage)

    We also purchased 2 new Supermicro 837E26 JBOD chassis and 2 LSI 9207-8e controllers for the cluster.

    The Interposers did not work, with the SSD or with a 1TB Seagate ES2 SATA HDD -- the controller only saw the devices as SATA, not as SAS, and Win2012 would not allow for the devices to be added to the cluster (which is correct, since the device was "SATA").

    We have tried to get support from LSI -- their answer was talk to the OEM.  Supermicro said, we don't have 9207 controllers, "we don't list the Crucial M4 as a supported SSD".  When we pointed out that the Interposer didn't work with the HDD, their response was "*you* need to upgrade the interposer FW" ("Here are the FW tools, build a linux system to do that and let us know if that solved the problem").

    Like you, I see no reason why the use of interposers is not supported.

    I am totally disappointed with the support we have gotten -- I spent a lot of money of the Interposers and the SSDs.

    Sorry to hear this but I've told you quite a time ago: using commodity hardware and industry proven software solutions (better say concepts as I'm not talking about particular vendors rather ways to solve the task) is a way to go. Relying on a "magic hardware" (SAS JBOD + SATA + expanders) and new born kludge (Clustered Storage Spaces) is begging for troubles.

    StarWind iSCSI SAN & NAS

    Friday, February 22, 2013 10:49 PM
  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.

    Scott18ss,

    I can confirm the LSI Interposer's have several problems -- as far as I can tell LSI is ignoring the issues and/or only playing lip-service to solving the problem.

    We purchased 16 LSI Interposers (OEM'd by Supermicro) in the hope of using them with Crucial M4 512GB SATA SSD, as part of a Windows 2012 server cluster (providing Tier 1 cluster storage)

    We also purchased 2 new Supermicro 837E26 JBOD chassis and 2 LSI 9207-8e controllers for the cluster.

    The Interposers did not work, with the SSD or with a 1TB Seagate ES2 SATA HDD -- the controller only saw the devices as SATA, not as SAS, and Win2012 would not allow for the devices to be added to the cluster (which is correct, since the device was "SATA").

    We have tried to get support from LSI -- their answer was talk to the OEM.  Supermicro said, we don't have 9207 controllers, "we don't list the Crucial M4 as a supported SSD".  When we pointed out that the Interposer didn't work with the HDD, their response was "*you* need to upgrade the interposer FW" ("Here are the FW tools, build a linux system to do that and let us know if that solved the problem").

    Like you, I see no reason why the use of interposers is not supported.

    I am totally disappointed with the support we have gotten -- I spent a lot of money of the Interposers and the SSDs.

    Friday, February 22, 2013 8:58 PM
  • Just wanted to post here that there is a list of approved SSP solutions in the Windows Catalog here - http://www.windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext=&bCatID=1573&cpID=0&avc=38&ava=0&avq=0&OR=1&PGS=25&ready=0

    As you can see it is not a very long list but it is a start.


    Chuck Timon Senior, Support Escalation Engineer (SEE) Windows 8 Beta Engineer Microsoft Corporation

    Monday, March 04, 2013 12:33 PM
    Moderator

All replies

  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.




    Mixed experience. Know customers running such a configs since summer 2012 and also know guys who had lock-ups until they had replaced SATA + converter thing with NL SAS drives. With NL SAS being just a bit more expensive then SATA I don't think it worth begging for troubles building unsupported setups.

    StarWind iSCSI SAN & NAS

    Friday, February 22, 2013 11:40 AM
  • No NLSAS SSD (unfortunately for us).
    I could have sworn that I saw a post by MS staff indicating that a SATA drive with SAS interposer was a supported solution. And that would make sense given that a Windows Storage Spaces certified product is selling this as a "solution".

    Friday, February 22, 2013 7:32 PM
  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.

    Scott18ss,

    I can confirm the LSI Interposer's have several problems -- as far as I can tell LSI is ignoring the issues and/or only playing lip-service to solving the problem.

    We purchased 16 LSI Interposers (OEM'd by Supermicro) in the hope of using them with Crucial M4 512GB SATA SSD, as part of a Windows 2012 server cluster (providing Tier 1 cluster storage)

    We also purchased 2 new Supermicro 837E26 JBOD chassis and 2 LSI 9207-8e controllers for the cluster.

    The Interposers did not work, with the SSD or with a 1TB Seagate ES2 SATA HDD -- the controller only saw the devices as SATA, not as SAS, and Win2012 would not allow for the devices to be added to the cluster (which is correct, since the device was "SATA").

    We have tried to get support from LSI -- their answer was talk to the OEM.  Supermicro said, we don't have 9207 controllers, "we don't list the Crucial M4 as a supported SSD".  When we pointed out that the Interposer didn't work with the HDD, their response was "*you* need to upgrade the interposer FW" ("Here are the FW tools, build a linux system to do that and let us know if that solved the problem").

    Like you, I see no reason why the use of interposers is not supported.

    I am totally disappointed with the support we have gotten -- I spent a lot of money of the Interposers and the SSDs.

    Friday, February 22, 2013 8:58 PM
  • No NLSAS SSD (unfortunately for us).
    I could have sworn that I saw a post by MS staff indicating that a SATA drive with SAS interposer was a supported solution. And that would make sense given that a Windows Storage Spaces certified product is selling this as a "solution".

    I have never seen anything from MS telling different from "SAS" so I think you need to get a confirmation from MS is this a really supported or just working config (different things obviously).

    StarWind iSCSI SAN & NAS

    Friday, February 22, 2013 10:43 PM
  • We have an OEM storage vendor telling us that it is NOT recommended at this time to use SAS interposers with any SATA drives (regardless of brand or type (HDD or SSD)) and use it with Windows Storage Spaces/Clustered Storage Spaces. This would be for a shared-SAS backend using a JBOD/HBA and Windows Storage Spaces as a scale-out file server solution. This is the setup you see referenced from Microsoft all over the place.

    Is anyone seeing any issues? They claim it's an issue with the LSI SAS interposers (LSISS9252) that all storage vendors rebrand or sell. And, apparently LSI is not seeing this as a high priority fix.

    They're alleging that Microsoft product development has told them it's an issue with the firmware on the SAS interposers (which is only upgradable by OEMs BTW) and that using these will result in drives dropping off/not showing up.

    BTW, this comes from a Windows Storage Spaces certified product...thoughts? I'm just looking for any information so we can make an informed decision.

    Scott18ss,

    I can confirm the LSI Interposer's have several problems -- as far as I can tell LSI is ignoring the issues and/or only playing lip-service to solving the problem.

    We purchased 16 LSI Interposers (OEM'd by Supermicro) in the hope of using them with Crucial M4 512GB SATA SSD, as part of a Windows 2012 server cluster (providing Tier 1 cluster storage)

    We also purchased 2 new Supermicro 837E26 JBOD chassis and 2 LSI 9207-8e controllers for the cluster.

    The Interposers did not work, with the SSD or with a 1TB Seagate ES2 SATA HDD -- the controller only saw the devices as SATA, not as SAS, and Win2012 would not allow for the devices to be added to the cluster (which is correct, since the device was "SATA").

    We have tried to get support from LSI -- their answer was talk to the OEM.  Supermicro said, we don't have 9207 controllers, "we don't list the Crucial M4 as a supported SSD".  When we pointed out that the Interposer didn't work with the HDD, their response was "*you* need to upgrade the interposer FW" ("Here are the FW tools, build a linux system to do that and let us know if that solved the problem").

    Like you, I see no reason why the use of interposers is not supported.

    I am totally disappointed with the support we have gotten -- I spent a lot of money of the Interposers and the SSDs.

    Sorry to hear this but I've told you quite a time ago: using commodity hardware and industry proven software solutions (better say concepts as I'm not talking about particular vendors rather ways to solve the task) is a way to go. Relying on a "magic hardware" (SAS JBOD + SATA + expanders) and new born kludge (Clustered Storage Spaces) is begging for troubles.

    StarWind iSCSI SAN & NAS

    Friday, February 22, 2013 10:49 PM
  • Sorry to hear this but I've told you quite a time ago: using commodity hardware and industry proven software solutions (better say concepts as I'm not talking about particular vendors rather ways to solve the task) is a way to go. Relying on a "magic hardware" (SAS JBOD + SATA + expanders) and new born kludge (Clustered Storage Spaces) is begging for troubles.


    StarWind iSCSI SAN & NAS

    Why do I find that most of your responses promote non-Microsoft (your own software) solutions?

    Yes, I can be disappointed in my choices, but I don't need you to be all "high and mighty".

    The SATA + SAS Interposer solution is SUPPOSED to the commodity hardware solution.  The fact that it isn't, is a reflection on the interposer vendor, and on the nature of support which the interposer vendor is providing.  Interposers are not non-commodity, they are simply "on the edge".


    • Edited by Sean Leyne Saturday, February 23, 2013 3:11 AM Typo
    Saturday, February 23, 2013 3:07 AM
  • Why do I find that most of your responses promote non-Microsoft (your own software) solutions?

    Yes, I can be disappointed in my choices, but I don't need you to be all "high and mighty".

    The SATA + SAS Interposer solution is SUPPOSED to the commodity hardware solution.  The fact that it isn't, is a reflection on the interposer vendor, and on the nature of support which the interposer vendor is providing.  Interposers are not non-commodity, they are simply "on the edge".


    Probably because it's a human nature to see what he (human) wants to see :) It should take somebody a couple of spare minutes to see my profile, check what I post to figure out you're providing... mendacious information.

    "Commodity" my friend is what's used by A LOT of people and manufactured by A WIDE RANGE of vendors. This creates competition and market demand and makes pragmatic buyers happy. If "something" is a very niche product it only creates an initial spike of interest to only attract a very limited amount of early adopters and visionaries like you just... to piss them off with non-maturity of an offered solution. Exactly your case! "Whole product" is not a software title or physical part it's a combined value of everything listed + support and sales force.  

    So... Good luck! Hope you'll finally manage to find out what to do with a bunch of expensive hardware you got and cannot use because your vendor does not care about you being a customer. 


    StarWind iSCSI SAN & NAS

    Saturday, February 23, 2013 9:33 AM
  • Just wanted to post here that there is a list of approved SSP solutions in the Windows Catalog here - http://www.windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext=&bCatID=1573&cpID=0&avc=38&ava=0&avq=0&OR=1&PGS=25&ready=0

    As you can see it is not a very long list but it is a start.


    Chuck Timon Senior, Support Escalation Engineer (SEE) Windows 8 Beta Engineer Microsoft Corporation

    Monday, March 04, 2013 12:33 PM
    Moderator
  • Thanks for all the reply's. It's clear that the interposers are not recommended at all and everyone should avoid at all cost until LSI updates the firmware. The price per GB of SAS SSDs is astronomical and this severely limits the price/performance we can get out of a clustered windows storage spaces solution.

    The DataON solutions were the products I was looking at as they were advertising the interposers. After speaking with them at length (William at DataON was very helpful and knowledgeable) they told me it isn't advisable to use the interposers and that they would eventually pull them for their site.

    Note - this interposer problem will also effect those of you planning on building clustered solutions on LSI's Syncro CS product (unreleased but coming soon).

    Note - be careful at the benchmarks you see with storage spaces. The clustered spaces are typically using simple spaces (which nobody sane would use in a business) to promote the crazy throughput and iops. Not a huge deal, but read the fine print. They're using SSDs as well (not a big deal) until you realize that the 12 SSDs used in their benchmark testing retail for $27K by themselves.

    I'm bummed as we wanted to truly use commodity off-the-shelf hardware (Samsung/Intel SSDs) and use a SOFS solution (with clustered Storage Spaces backend) and Infiniband (SMB Direct) frontend to create a truly high-capacity/high performance storage that could blow the doors of any SAN at 3x the price.

    Microsoft - lean on LSI to get this fixed!

    • Proposed as answer by Shahid Roofi Thursday, September 19, 2013 11:44 AM
    Tuesday, March 05, 2013 5:45 AM
  • Microsoft - lean on LSI to get this fixed!

    100% Agree!
    • Proposed as answer by Shahid Roofi Thursday, September 19, 2013 11:43 AM
    Tuesday, March 05, 2013 5:41 PM
  • Thanks for all the reply's. It's clear that the interposers are not recommended at all and everyone should avoid at all cost until LSI updates the firmware. The price per GB of SAS SSDs is astronomical and this severely limits the price/performance we can get out of a clustered windows storage spaces solution.

    The DataON solutions were the products I was looking at as they were advertising the interposers. After speaking with them at length (William at DataON was very helpful and knowledgeable) they told me it isn't advisable to use the interposers and that they would eventually pull them for their site.

    Note - this interposer problem will also effect those of you planning on building clustered solutions on LSI's Syncro CS product (unreleased but coming soon).

    Note - be careful at the benchmarks you see with storage spaces. The clustered spaces are typically using simple spaces (which nobody sane would use in a business) to promote the crazy throughput and iops. Not a huge deal, but read the fine print. They're using SSDs as well (not a big deal) until you realize that the 12 SSDs used in their benchmark testing retail for $27K by themselves.

    I'm bummed as we wanted to truly use commodity off-the-shelf hardware (Samsung/Intel SSDs) and use a SOFS solution (with clustered Storage Spaces backend) and Infiniband (SMB Direct) frontend to create a truly high-capacity/high performance storage that could blow the doors of any SAN at 3x the price.

    Microsoft - lean on LSI to get this fixed!

    I too have been looking at the DataOn DNS1640D JBOD to be used with a 3 node 2012 Cluster to create a Scale Out File Server.  I recently spoke with DataOn and received info that the interposer did have problems and that LSI was working on it, to stay tuned.  Not sure what that means and upgrading the FW on the interposer is a pain.  I have been planning on using OCZ Deneva 2 C class SATA SSDs with the interposer, but may not now.  The Talos 2 are a bit more expensive for per GB and you need the R class to get the capacitor power fail protection making it even more expensive.  At any rate I am disappointed.  I had originally asked what Storage Space config was used in testing with their presented results and it was simple stripes.  No redundancy at all.  I was also told that parity storage could not yet be clustered but MS has plans for it in the future.  Also that the only redundant type of clustered storage that could be used was mirror spaces but only with SAS, no interposer. I am not going to put an enterprise grade san on just simple stripes and rely on daily backups, that's nuts.  

    So SAS and Mirror storage spaces do cluster ok but no idea on the performance of it.   I can also say I have been using these LSI interposes from Supermicro on only the 4th gen chips/FW and have had MAJOR problems with SATA SSDs using an older LSI 2008 RAID card.  Any time a full shut down and start up happens (NOT reboot) a random number of the drives come back flashing red and can not be seen by the raid card.  This happens with single drives, RAID 1, and RAID 10 arrays I had made.  The only way to get them to show back up was to remove the drive and place it in my laptop and boot and then shut it off, then swap it back in.  It would come up green right away and sometimes begin a RAID rebuild.  I know that sounds nuts but simply removed and re seating did nothing, only the laptop boot/swap worked.  Reboots were ok but any real shut down messed this up.  At the time I was using Corsair Force 3 drives.  After the latest corsair FW and LSI card FW I was able to get it to be more stable but not 100%.  Switching to Plextor M3 Pro with latest FW made a big improvement.  Now on shutdowns and power up only from time to time does a drive come up red, and only one or two, normally a re-seat fixes it and RAIDs do not rebuild.  Much better but still a problem.  The Corsair drives would also cause ESXI 5.0 to every once in a while lock up and need a reboot.  I'm using the Plextor M3 Pro's on physical Win 2012 now and no longer have lock ups.  

    Anyways this is just my experience.  Also the LSI RAID card and SAN appliance is a supermicro and not the DataOn solution I was mentioning. I'm looking to move to the solution in the future.  So for now I think real SAS SSDs are the way to go.

    Friday, March 22, 2013 12:41 AM

  • I too have been looking at the DataOn DNS1640D JBOD to be used with a 3 node 2012 Cluster to create a Scale Out File Server.  I recently spoke with DataOn and received info that the interposer did have problems and that LSI was working on it, to stay tuned.  Not sure what that means and upgrading the FW on the interposer is a pain.  I have been planning on using OCZ Deneva 2 C class SATA SSDs with the interposer, but may not now.  The Talos 2 are a bit more expensive for per GB and you need the R class to get the capacitor power fail protection making it even more expensive.  At any rate I am disappointed.  I had originally asked what Storage Space config was used in testing with their presented results and it was simple stripes.  No redundancy at all.  I was also told that parity storage could not yet be clustered but MS has plans for it in the future.  Also that the only redundant type of clustered storage that could be used was mirror spaces but only with SAS, no interposer. I am not going to put an enterprise grade san on just simple stripes and rely on daily backups, that's nuts.  

    So SAS and Mirror storage spaces do cluster ok but no idea on the performance of it.   I can also say I have been using these LSI interposes from Supermicro on only the 4th gen chips/FW and have had MAJOR problems with SATA SSDs using an older LSI 2008 RAID card.  Any time a full shut down and start up happens (NOT reboot) a random number of the drives come back flashing red and can not be seen by the raid card.  This happens with single drives, RAID 1, and RAID 10 arrays I had made.  The only way to get them to show back up was to remove the drive and place it in my laptop and boot and then shut it off, then swap it back in.  It would come up green right away and sometimes begin a RAID rebuild.  I know that sounds nuts but simply removed and re seating did nothing, only the laptop boot/swap worked.  Reboots were ok but any real shut down messed this up.  At the time I was using Corsair Force 3 drives.  After the latest corsair FW and LSI card FW I was able to get it to be more stable but not 100%.  Switching to Plextor M3 Pro with latest FW made a big improvement.  Now on shutdowns and power up only from time to time does a drive come up red, and only one or two, normally a re-seat fixes it and RAIDs do not rebuild.  Much better but still a problem.  The Corsair drives would also cause ESXI 5.0 to every once in a while lock up and need a reboot.  I'm using the Plextor M3 Pro's on physical Win 2012 now and no longer have lock ups.  

    Anyways this is just my experience.  Also the LSI RAID card and SAN appliance is a supermicro and not the DataOn solution I was mentioning. I'm looking to move to the solution in the future.  So for now I think real SAS SSDs are the way to go.

    1) It's a "dirty little secret" we had discovered already and calculated in theory GB/sec and IOPS matching simple storage spaces only (no even basic protection, single port flash attached). See this thread and my and Mr. Leyne's posts:

    http://social.technet.microsoft.com/Forums/en-US/winserverClustering/thread/4c416edb-6382-41ff-b6f1-5215baaf3d0c

    2) Parity spaces are dog slow right now and clustering is not going to speed up them anyway. See this threads:

    http://social.technet.microsoft.com/Forums/en-US/winserver8gen/thread/64aff15f-2e34-40c6-a873-2e0da5a355d2

    http://social.technet.microsoft.com/Forums/en-US/winserver8gen/thread/79ca6d6d-cab7-4ff3-8c17-ec6ce249e641

    (and you can search for more topics like referenced ones). So I would expect people throwing tons of $$$ into flash and clustered storage spaces

    to sqeeze just a fraction of IOPS they were expecting to get.

    3) Going for dual port SAS flash drives renders implementation costs on an expensive side w/o increasing either capacity or IOPS. Paying premium just to make sure things are working is kind of a Catch 22 situation :)



    StarWind iSCSI SAN & NAS

    Saturday, March 23, 2013 6:36 PM
  • Are there any Updates on this topic? Maybe related to 2012 R2?

    Greetings

    Thursday, August 15, 2013 3:17 PM
  • Are there any Updates on this topic? Maybe related to 2012 R2?

    Greetings

    Not much... List of supported and certified SAS JBOD vendors increased from 2 or 3 to 5. You can check more here:

    http://channel9.msdn.com/Events/TechEd/NorthAmerica/2013/MDC-B218

    Also LSI had finally released their clustered RAID controllers... Just to make people understand you still need a SAS JBOD and their "clustering" features only synchronizes caches on all the nodes so is for performance rather then for architecture point of view. See:

    http://www.crn.com/news/storage/240155832/teched-lsi-microsoft-team-on-low-cost-clustered-server-storage-solution.htm

    http://www.lsi.com/downloads/Public/Syncro%20Shared%20Storage/docs/LSI_PB_SyncroCS_9286-8e.pdf

    That's pretty much all...


    StarWind iSCSI SAN & NAS

    Thursday, August 15, 2013 5:43 PM
  • Yeah, I read about the Synchro... even the LSI support recommended the Synchro - of course.

    But that was what I would have expected from the Synchro... poor.

    But, what about the Interposers? It's just the only way to go if someone want's cheap SSD power... I think about the Samsung EVO 1TB...

    Thursday, August 15, 2013 6:56 PM
  • Yeah, I read about the Synchro... even the LSI support recommended the Synchro - of course.

    But that was what I would have expected from the Synchro... poor.

    But, what about the Interposers? It's just the only way to go if someone want's cheap SSD power... I think about the Samsung EVO 1TB...

    Get SAS SSD. That's the only way to go. IMHO.

    StarWind iSCSI SAN & NAS

    Thursday, August 15, 2013 7:03 PM
  • I just can't afford it...

    But thanks for your opinion on this. :-)

    Thursday, August 15, 2013 7:41 PM
  • I just can't afford it...

    But thanks for your opinion on this. :-)

    There are third-party software packages doing sync mirror of LUNs between hosts. Except they can use SATA and don't care about any physical shared hardware between the hosts (SAS JBOD etc).

    StarWind iSCSI SAN & NAS

    Thursday, August 15, 2013 10:11 PM
  • I spoke with a JBOD vendor and they said LSI released updated firmware for the interposers that fixed the issues mentioned in this thread.  However, the JBOD vendor DOES NOT recommend using interposers for production.  We're considering purchasing some for testing and if unhappy, we'll consider SAS SSD with SAS HDD in tiering mode (available when R2 releases).  Just would be nice if MS would allow native SATA for clustering.  I don't know enough about the dual-port aspects of SAS drives, but it would seem that they could get SATA to work rather easily.  Thoughts anyone?

    Wednesday, September 11, 2013 5:52 PM
  • Just would be nice if MS would allow native SATA for clustering.  I don't know enough about the dual-port aspects of SAS drives, but it would seem that they could get SATA to work rather easily.  Thoughts anyone?

    SSDsRFUN,

    The problem is that SATA doesn't provide a redundant path (but physical and logical) for disk controllers, so the loss of a controller would result is a loss of the Storage Spaces (built on top of the HDD/SSDs).

    The only way that SATA can be used in clustering is the approach which StarWind and others have taken to use 'replication' to keep the disk operations syncronized across 2 separate (though related) nodes.

    Wednesday, September 11, 2013 6:28 PM
  • I spoke with a JBOD vendor and they said LSI released updated firmware for the interposers that fixed the issues mentioned in this thread.  However, the JBOD vendor DOES NOT recommend using interposers for production.  We're considering purchasing some for testing and if unhappy, we'll consider SAS SSD with SAS HDD in tiering mode (available when R2 releases).  Just would be nice if MS would allow native SATA for clustering.  I don't know enough about the dual-port aspects of SAS drives, but it would seem that they could get SATA to work rather easily.  Thoughts anyone?

    It's not going to happen any time soon. For a very simple reason: Microsoft with their Clustered Storage Spaces rely heavily on a SAS hardware. So they need 1) redundant data paths to the device (as they don't use other fabric to wire the data unlike say StarWind, VMware or LINBIT use Ethernet or InfiniBand running IP) 2) device capable of a lock handling (distributed lock manager is completely missing from Microsoft design) and 3) something taking care of a keeping caches coherent (no big deal for a single device but think about two physical devices on the different hypervisor nodes). Making long story short: it's not something Microsoft can enable or write on the knee very fast it's a whole different architecture compared to a Clustered Storage Spaces. They *may* enable say FC or iSCSI (with a tons of a restrictions) but not SATA. However CSS with FC or iSCSI has zero sense: SAS JBOD is a passive hardware (no CPU, no RAM, no dual or triple controllers, no NVRAM for WAFL spoofing etc etc etc) so is cheap. iSCSI and FC (this one also has expensive fabrics) are not. So replacing cheap SAS JBOD with an expensive FC to route traffic thru the Windows hosts?!? Sounds like a summer IT project to me :)

    StarWind iSCSI SAN & NAS

    Wednesday, September 11, 2013 8:38 PM
  •    Can anybody confirm if the latest firmware from LSI does fixes the issue so SATA disk can be chosen for CSS ?


    Shahid Roofi

    Thursday, September 19, 2013 11:56 AM
  •    Can anybody confirm if the latest firmware from LSI does fixes the issue so SATA disk can be chosen for CSS ?

    I'm also interested. Also looks like one of the major "pushers" of this thing - DataOn had released similar thing. Do not know is it a real one or just OEM-ed and re-badged LSI. See:

    http://www.dataonstorage.com/dataon-products/6g-sas-to-sata-interposer-card-and-kit.html

    I'm a bit pessimistic with a very few vendors actually doing something in this direction but hope eventually the whole thing would come into "working" state :)


    StarWind iSCSI SAN & NAS

    Saturday, September 21, 2013 10:42 AM
  •    Can anybody confirm if the latest firmware from LSI does fixes the issue so SATA disk can be chosen for CSS ?

    I'm also interested. Also looks like one of the major "pushers" of this thing - DataOn had released similar thing. Do not know is it a real one or just OEM-ed and re-badged LSI. See:

    http://www.dataonstorage.com/dataon-products/6g-sas-to-sata-interposer-card-and-kit.html

    I'm a bit pessimistic with a very few vendors actually doing something in this direction but hope eventually the whole thing would come into "working" state :)

    Unfortunately, LSI is doing a very poor/lousy job of supporting their products.  They don't offer direct support for the interposers, only through the OEMs which sell their tech.  So, the support chain is terrible.

    We purchased some Supermicro interposers (20 of them actually) and tried using them with several devices.  One of the devices we tried was a very standard Seagate 1TB SATA HDD, and the interposer didn't work.  It took forever to get someone to acknowledge that there was a real problem, then when they did provide a fix it was in the form of a kit (set of scripts really) which required a Linux install and specific disk controller/cabling.  (We are a 100% Windows shop).

    Saturday, September 21, 2013 5:57 PM
  • [ ... ]

    Unfortunately, LSI is doing a very poor/lousy job of supporting their products.  They don't offer direct support for the interposers, only through the OEMs which sell their tech.  So, the support chain is terrible.

    We purchased some Supermicro interposers (20 of them actually) and tried using them with several devices.  One of the devices we tried was a very standard Seagate 1TB SATA HDD, and the interposer didn't work.  It took forever to get someone to acknowledge that there was a real problem, then when they did provide a fix it was in the form of a kit (set of scripts really) which required a Linux install and specific disk controller/cabling.  (We are a 100% Windows shop).

    Nice... I referenced in the other thread, keeping in mind the price for re-badged interposer thing it makes sense to get NL SAS drive instead of a SATA + interposer. Also it's a big question how well interposers will work with a high grade SATA flash drives (where converting from SATA -> SAS makes sense from the financial point of view).

    http://social.technet.microsoft.com/Forums/windowsserver/en-US/1d8b5c12-b62b-4337-bcb3-afaeecfa593f/sas-vs-sata-in-clustered-storage-spaces-for-hyperv-cluster


    StarWind iSCSI SAN & NAS

    • Proposed as answer by Shahid Roofi Saturday, September 21, 2013 8:48 PM
    Saturday, September 21, 2013 8:15 PM
  •    That's good compromise. I wish there is proper solution for SATA to SAS bridge, owing to excellent cost and availability factor of SATA drives (both SSD and HDDs).

       That could really make the Windows Storage Spaces shake the storage paradigm, i must say.


    Shahid Roofi

    Saturday, September 21, 2013 8:51 PM
  •    That's good compromise. I wish there is proper solution for SATA to SAS bridge, owing to excellent cost and availability factor of SATA drives (both SSD and HDDs).

       That could really make the Windows Storage Spaces shake the storage paradigm, i must say.

    Any software technology that relies on some sort of a "magic hardware" to work is a dead born solution and would be killed by other software technology based on a commodity hardware eventually. So either "magic hardware" turns to become a commodity or the whole thing sunks.

    StarWind iSCSI SAN & NAS

    Sunday, September 22, 2013 12:42 PM
  • So at this point, it seems SAS is the way to go both for HDD and SSD, at least if you want to settle with only-Microsoft Storage Spaces.

    But are there any caveats on which SAS HDD/SAS models to get, or can we get any as long as they're SAS? So we don't go out and acquire a bunch of SAS disks and they won't work good either?

    Wednesday, October 09, 2013 9:17 AM
  •   SAS disks are by definition dual ports. All of them. You can buy cheaper NL SAS and that should do as well the same. You should have no problem if you are willing to buy SAS disks. Ensure that connectivity, expanders are set to connect all servers to the disk (hopefully you would already be doing that)

      The crux of above thread is/was to be able to use SATA JBOD disks as Microsoft advertises but not so true with latest LSI firmware still missing to address that.

     


    Shahid Roofi

    Wednesday, October 09, 2013 10:06 AM
  • Great that you till monitor this thread! I suspect many of our are struggling to understand how to design this Storage Spaces. I'm planning to setup this properly in the lab and test. May I ask which JBOD brands/models you've tested?

    Wednesday, October 09, 2013 10:41 AM
  • But are there any caveats on which SAS HDD/SSD models to get

    When looking at SAS SSDs you need to consider whether power failure protection matters to you -- if power is lost, it is possible that not all data in the drive cache will have been written to SSD media.

    As I understand, if you will be using "Mirrored" resiliency for *all* your Storage Spaces, this is not an issue -- as the write operation will not be considered complete until the writes have been flushed on all mirror members. 

    But if you are using "Simple " resiliency you would need to be concerned.  (I don't know about "Stripped" resiliency, haven't looked at it since initial performance reports were 'not kind' for this modality)

    Wednesday, October 09, 2013 3:47 PM
  • So at this point, it seems SAS is the way to go both for HDD and SSD, at least if you want to settle with only-Microsoft Storage Spaces.

    But are there any caveats on which SAS HDD/SAS models to get, or can we get any as long as they're SAS? So we don't go out and acquire a bunch of SAS disks and they won't work good either?

    Yes, exactly.

    Any SAS should do the trick. I would avoid NL SAS and cheap MLC SSDs however.


    StarWind iSCSI SAN & NAS

    Wednesday, October 09, 2013 7:44 PM
  •   SAS disks are by definition dual ports. All of them. You can buy cheaper NL SAS and that should do as well the same. You should have no problem if you are willing to buy SAS disks. Ensure that connectivity, expanders are set to connect all servers to the disk (hopefully you would already be doing that)

      The crux of above thread is/was to be able to use SATA JBOD disks as Microsoft advertises but not so true with latest LSI firmware still missing to address that.

     


    Shahid Roofi

    NL SAS should be avoided. There's no sense to use them as say RE WDs have better parameters and cost less.



    StarWind iSCSI SAN & NAS

    Wednesday, October 09, 2013 7:46 PM
  • When looking at SAS SSDs you need to consider whether power failure protection matters to you -- if power is lost, it is possible that not all data in the drive cache will have been written to SSD media.

    Absolutely true. That's why "white bone" array controllers do write-flush-read-chech-hash to make sure data is actually stored and not just confirmed as stored by drive. Flash drive controllers always have some intermediate (NV)RAM to keep data in before they do burn it to flash cells and it's a very big question what happens to a block that has been confirmed by flash controller as "written" but it's actually not. Yet. 

    StarWind iSCSI SAN & NAS

    Friday, October 25, 2013 10:52 AM
  • Dear Sean Leyne!

    Which interposers could you repair?

    Can I have the firmware, tools and documentation that you got from them?

    We were not lucky enough to get the fix from them yet...

    Thanks:

    DDenke.

    Monday, October 28, 2013 10:08 AM
  • We did not repair the interposers, the firmware kit is not user friendly (requires a Linux install and some other tools).  We never actually installed/ran the kit.

    As for the kit details, I don't feel confortable with distributing it -- we got it from Supermicro (SM) support and don't think we are authorized to redistribute.  I would suggest you contact SM support and get it from them.

    Monday, October 28, 2013 6:49 PM