none
REFS for a Hyper-V CSV Volume

    Question

  • I am currently running a 4 node Server 2016 Hyper-V cluster storage is provided by a DELL MD3200 SAN normal concepts for CSV volume on NTFS are in place as previously nodes where 2012R2 and 2008R2 prior. My main question is I am going to be adding another SAN DELL MD3420 and I have been wondering whether to use REFS for the file system for VM's situated on this SAN.

    I came across a Veeam article advising moving to this scenario however I would like to get the opinion of you all.

    Regards

    Chris


    MCP, MCTS, MCITP, MCITPVA, MCSA 2012, MCSE Private Cloud, MCSE Server Infrastructure

    Thursday, February 23, 2017 7:15 PM

All replies

  • in short, stay with NTFS in the SAN Config, have a look on the supported deployments

    https://technet.microsoft.com/en-us/windows-server-docs/storage/refs/refs-overview

    cheers

    Udo

    Thursday, February 23, 2017 11:16 PM
  • Hi,

    I agree with Udo.

    I suppose using NTFS is a better choice. 

    Best Regards,

    Leo


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Friday, February 24, 2017 2:03 AM
    Moderator
  • Maybe, this could also be helpful around Cluster size:

    https://blogs.technet.microsoft.com/filecab/2017/01/13/cluster-size-recommendations-for-refs-and-ntfs/

    https://www.windowspro.de/marcel-kueppers

    I write here only in private interest

    Disclaimer: This posting is provided AS IS with no warranties or guarantees, and confers no rights.

    Friday, February 24, 2017 9:43 AM
  • Hi,

    You could mark the reply as answer if it is helpful. 

    Best Regards,

    Leo


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, March 09, 2017 12:12 PM
    Moderator
  • in short, stay with NTFS in the SAN Config, have a look on the supported deployments

    https://technet.microsoft.com/en-us/windows-server-docs/storage/refs/refs-overview

    cheers

    Udo

    Hi,

    jumping on this bandwagon as well. I've read that article you mention. The date on top of the page says 2016-1-3, but maybe it's been updated in the past, I'm not sure about that. But under 'Fcuntionality' it clearly stated ReFS is supported in cluster environments. While it might be working with redirected IO (I'm gonna test that in a test environment), you seem to say it's not supported. But as far as that document goes it is.

    By the way, the reason I'm looking into this is that although we have 16Gbps SAN which sequentially almost reaches that (around 1400MB/sec sequential write) we see huge, huge IO when Hyper-V zeroes-out a vdisk when expanding or creating one. We deliberately use fixed vhdx. Creating or expanding a vdisk practically stalls all VM's on that same array. I'm not sure what Hyper-V does there, as writing for example 100GB of random data to the same array writes with somewhere between 1200-1400MB/sec and all VM's keep running just fine.

    Anyway with ReFS we shouldn't have that issue anymore as that handles creating files completely different.

    I'll set something up in my test environment and let you know.

    Tuesday, June 20, 2017 9:06 AM
  • jumping on this bandwagon as well. I've read that article you mention. The date on top of the page says 2016-1-3, but maybe it's been updated in the past, I'm not sure about that. But under 'Fcuntionality' it clearly stated ReFS is supported in cluster environments. While it might be working with redirected IO (I'm gonna test that in a test environment), you seem to say it's not supported. But as far as that document goes it is.

    Hi,

    ReFS has a lot of benefits and in my opinon best practices only to use it with CSV in 2016 (On top of Storage Spaces / Direct). But your test regarding redirected IO is correct.

    Regards,
    Marcel

    https://www.windowspro.de/marcel-kueppers

    I write here only in private interest

    Disclaimer: This posting is provided AS IS with no warranties or guarantees, and confers no rights.



    Tuesday, June 20, 2017 10:01 AM
  • Adding some great links to help you with the choice ReFS/NTFS. 

    https://blogs.technet.microsoft.com/askpfeplat/2013/01/01/windows-server-2012-does-refs-replace-ntfs-when-should-i-use-it/

    https://www.starwindsoftware.com/blog/refs-performance

    https://mangolassi.it/topic/11362/windows-administration-ntfs-and-refs-filesystems

    We are still running NTFS though.


    Best regards,

    Taras Shved

    StarWind Software

    Blog:   Twitter:   LinkedIN:  

    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.

    Tuesday, July 18, 2017 8:06 AM
  • Hello,

    Staying with NTFS for now would be the best option for now. NTFS disk are recoverable in case of failure. REFS can not be recovered as Microsoft state that REFS is self recoverable. In case of severe failure backup is the only option when it comes to recover from a REFS data drive loss.

    Regards,

    Nick.

    Wednesday, July 19, 2017 7:15 PM
  • I've not dived THAT deep into it yet, but I think it's recoverability is based partly on the metadata. However, for VHDX files that part is disabled in the first place, making it a bad choice based on the recoverability I guess as it can't detect failures in there anyway.

    But then, the metadata is not like a copy of the data, just some form of parity in this sense. When it detects a failure, it can copy the datablock from another copy, if you have build a storage spaces direct setup. We don't use that though, and I don't see us using that in the near future. We utilize proper SANs that do data scrubs constantly to somewhat prevent the same issue. Given all severe bugs we've had so far in Server 2016 as well as the atrocious cumulative patching system, meaning if a bug is in a specific fix in a cumulative you have the choice to either update everything or nothing.

    Take KB3200970 which was released in November 2016. It breaks the functionality of Hyper-V to take snapshots of virtual failover-clusters, breaking all hypervisor-level backup tools like Veeam. It's now July 2017 and still no fix, which was promised us in the June update already. In the past we'd be able to uninstall the specific patch that screwed things up and keep all other patches installed to still be secure for the rest. With 2016 that's not possible anymore. And we've had so many more bugs, in AppV for example, rendering our 2016 AppV unusable for 4 months. In our 2008R2 all we just uninstalled the specific update and we were fine again.

    I can see the advantages of cumulative updates of course, making for a lot less diversity in the field. Yet, I can't imagine the trouble when there's a bug in REFS / Storage Spaces in a specific cumulative. With the speed of MS releasing patches (for good reasons) it could bring you down for months.

    Having said all that, we've decided to stay with NTFS as we don't trust MS in any way with our data at this point. I hope that will change in the future, but for now, if you don't have storage spaces, and no 'proper' storage solution, go for NTFS. With Hyper-V, as there's no metadata on VHDX files, stay with NTFS period until they fix that.

    Thursday, July 20, 2017 6:44 AM
  • Hi Sigdarkside!

    Refs is not supported with SAN Storage Systems, for this scenario use only NTFS, you can see updated documentation with that information:

    https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview

    Monday, September 04, 2017 12:59 PM
  • This note is very new and on some sites (e.g. german) not available.

    https://www.windowspro.de/marcel-kueppers

    I write here only in private interest

    Disclaimer: This posting is provided AS IS with no warranties or guarantees, and confers no rights.

    Tuesday, September 19, 2017 6:55 AM
  • Hi Sigdarkside!

    Refs is not supported with SAN Storage Systems, for this scenario use only NTFS, you can see updated documentation with that information:

    https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview

    Wow. That's a bold statement there on that site. ReFS is not supported on SAN storage. That would render ReFS unusable in, well, let's guess, 60% of all somewhat bigger environments? At least the ones I know. My own backup solution runs Veeam on ReFS as it has a somewhat nice integration with it, and it has it's own dedicated SAN for that. That statement renders my backup solution unsupported? I can't believe that's the way MS wants ReFS to go. I know MS wants to push their thinking to as much of it's customers as possible, but we don't all want to or can use S2D.

    I'd have so much more trust in a good hardware SAN than a MS software solution for storage. I'm really curious where this will go.

    Tuesday, September 19, 2017 7:47 AM
  • The documentation has changed. But, the problem seems that if you are using ReFS for CSVs on a SAN, then all IO is redirected through the coordinator. Check https://www.vembu.com/blog/windows-server-2016-hyper-v-refs-vs-ntfs/

    MarianoC.


    • Edited by MarianoC Wednesday, August 08, 2018 1:32 PM
    Wednesday, August 08, 2018 1:31 PM