none
Parity Storage Space so slow that it's unusable

    Pregunta

  • Using the same pool comprising of multiple 1.5TB/2TB WD&Seagate drives on a Core 2 Quad System.

    Regular Storage Space shows performance over 100MB/sec

    Mirror Storage Space shows performance of around 80MB/sec

    Parity Storage Space shows performance of 80MB/sec initially but as you copy a large file the performance keeps dropping until it reaches single digits making this pool option basically unusable. Is this a bug or is it expected behaviour?

    martes, 13 de marzo de 2012 13:13

Todas las respuestas

  • I'm not benchmarking the feature but have been performing similar measurements to get a feel for what is possible and the impact of choosing thin provisioning etc. I've had some very strange results too variable to be statistically significant, but I can't work out what to expect. This variability could well be down to device drivers rather than the volume manager/software raid layer. Saying that my past experience of software RAID performance in Windows, both native and third party ( Veritas SFW ), has been poor. It has to be said storage spaces look very like SFW.

    Out of interest, how many independent spindles are in your pool? Do you see high CPU when utilising parity? If you create a pool from a subset of spindles and perform I/O on a volume in the pool do you see activity on all the spindles in your machine or just those in the pool?

    I assume "simple" layout is equivalent to RAID-0. What isn't clear is if all the disks in a pool are involved when you select mirror or parity. What are mirror and parity equivalent to in RAID levels, 10 & 50 respectively? But if there are three disks and you choose mirror as the storage layout do you get three plexes or two plexes with one spindle unused, certainly when I create a 20GB mirror in a pool with 3 physical devices I lose 40GB from the pool, does that mean there are two plexes regardless of the pool physical disk count? What if you choose mirror and the pool contains 7 physical disks, do you get a stripe or concatenation of mirrors where the submirrors are a mix of two and three plexes? The same question applies to the parity layout option and what happens when you mix virtual disk layouts within a pool?

    Why is a 1GB allocation retained on any disk which has been part of a virtual disk? I need to use the clean operation in diskpart to have the free space returned to expected. As another poster has pointed out disks which are not returned to the primordial pool prior to the OS being reinstalled have limited functionality when the existing pool is "rediscovered" by the new OS install. It seems impossible to delete such a pool and the only way I have found to return the disks to the primordial pool is to boot to a command prompt from W2K8R2 media and perform a clean on the disks from dispart.

    jueves, 15 de marzo de 2012 20:17
  • Some of your questions need to be answered by MSFT. Right now the server has 2 1.5TB drives an 1 2TB drive and a 60GB SSD as the OS drive. I created one storage pool that includes both 1.5TB drives and the 2TB drive. Within that pool I created the different types for Spaces for my testing. So, at the pool level, there is no mirror or parity settings. That happens at the space level. The CPU is a Core 2 Quad 2.5Ghz and it is never stressed by the Mirror or Parity writes.

    jueves, 15 de marzo de 2012 20:24
  • Thanks, I know some of my questions are for Microsoft, I was just putting my thoughts out there as it is impossible to quantify performance without understanding roughly what happens under the lid. With only three spindles in the pool some of my own questions aren't relevant to you.

    I've performed a similar test with data copied from a simple virtual disk in a three spindle pool to simple, mirror and parity virtual disks carved out of a separate three spindle pool. In my case the disks in question are 6*15K SAS. I've also peformed tests using iometer against iSCSI virtual disks on each of the virtual disks. I didn't experience the same behaviour as you with a parity virtual disk, my write speeds were variable throughout the tests. However, the parity pool was an order of magnitude slower than simple or mirror virtual disks so yes I'd agree that it isn't a viable choice at present.

    You'd think at least one of the cores in your system would be showing load when calculating parity but like you mine ( 2 * Opteron 4122 ) shows virtually no load when writing to parity virtual disks. Nothing new there though, this was the same in 2003/2008. It may be that the threads performing the parity calculation are artificially throttled but if that is the case it definitely needs to be tuneable in server "8" if it is to be used purely as a storage platform.

    I've stopped looking at Storage Spaces for now, it works as a technology preview and will be neat when it is production ready.

    viernes, 16 de marzo de 2012 10:38
  • I have been experimenting with Storage Spaces on my previous Windows Home Server 2011 box (Intel Xeon X3450 16gb RAM)

    I have 6 2tb drives in a thin provisioned parity storage pool.

    When copying files (1+gb files), it will do an initial burst of writing then stop completely.  I notice that in the Resource Monitor/Disk tab, it shows the file being written so I can see it is still writing.  It will start copying again once the data is flushed to the hard drives, although it takes quite a while.  I can understand this if the system is calculating parity information but my CPU is never spiked either.  It almost needs to have the copy slowed down so the system can keep up.

    Copying from a Windows 7 machine to the Windows 8 server over a gigabit network will give errors such as the file needs to be replaced and the destination file shows 0 bytes even though I am copying it for the first time.  Also copying from more than one source at the same time to the storage pool will occasionally disconnect the server from the network and cause error connecting to \\server\share messages.  Hitting retry on the copy will start it again but then stop.  This makes it difficult to do unattended file copying.

    domingo, 18 de marzo de 2012 20:03
  • I've experienced the same thing, running on a Intel Core 2 Quad (Q6600, P45 motherboard) with 5 drives in the storage pool. Striped and mirrored virtual disks work fine but as soon as I try to copy things onto the parity disk everything just goes down the drain, it seems to copy data in "bursts" and then just stop and wait for minutes (copying 50GB of data from another local disk took hours).

    martes, 27 de marzo de 2012 8:37
  • From all the evidence I can gather so far, the Parity pools get pretty good read performance (essentially N - 1 drive speed), but the writes are stuck right around 25MB/s.


    That's glacially slow considering how fast the processors are, they should easily be able to calculate parity faster than that.


    I have a feeling that Microsoft is playing it too safe and not caching any write back data. On my Intel RAID 5, the same thing happens if I disable the write cache, the speed becomes unusable.


    So, I'm waiting for the final version to see if Microsoft improves things. I believe they allow you to do things like use an SSD for the journal log. Maybe you can specify a write back cache option with the powershell commands.
    domingo, 01 de abril de 2012 17:52
  • My question is if there is Storage Pools in Windows 8 Consumer Preview and in Windows Server 8, are they both the same or is the one in Windows Server 8 beta?  I am hoping this one is beta or else many users will be disappointed with this kind of performance.
    miércoles, 04 de abril de 2012 5:25
  • I try Parity storage pool with 3 x 500gb (7200) and I have the same "slow" write issue. I have something like 20-30MB/sec. Really slow !
    lunes, 30 de abril de 2012 20:23
  • Wow, I have a spaces test setup that I'm trialling, 7 1tb drives divided into two thinly provisioned spaces, my games space which has no redundancy and the test space which has two drive redundancy.

    I get bursts of about 30m/b write which is about what I expected, then it sharply drops off after a while to about 250k/s. Quite frustrating when I'm trying to copy the contents of a 2TB drive.

    Is there a Connect space to report this too? I haven't seen any.

    lunes, 30 de abril de 2012 23:26
  • Can confirm the same. When copying large amount of data/files to parity disks, writepe rformance droups untilly fully stops or disk disappear from My Computer. With Simple disks no issues.Waiting for Server 8 RC, hope guys will fix this issue

    MCSE 2003: Messaging, Security
    MCITP: Server Administrator, Enterprise Administrator, Virtualization Administrator
    MCITP: Consumer Desktop Support, Enterprise Desktop Support, Enterprise Desktop Support Windows 7, Enterprise Desktop Administrator
    SMS/SCCM, ISA 2004/2006, CCNA Security

    viernes, 18 de mayo de 2012 16:04
  • I am getting the same issue on Server 8 Beta, I have 5x 500GB SATA disks making a storage pool of 2.27TB and a 2TB virtual disk with Thin provisioning and Parity layout, formatted in ReFS.

    Did some testing with it today and as all others have found the transfer starts off maxing out the network at around 100-110MB/s then about 1/4 of the way through almost pauses for a few seconds and the rest of the transfer continues at around 25MB/s with some copying it starts to ask if it can over write the file.

    The RC is out next week so lets see if that fixes it.

    martes, 29 de mayo de 2012 16:11
  • Anyone tried parity performance on the Release Preview yet? Just curious.
    viernes, 01 de junio de 2012 7:03
  • On Server 2012 RC, I got ~30MB/sec in an 18 minute test on an NTFS formatted parity volume across 3x 250GB 7200rpm disks. Better, but still not great.
    viernes, 01 de junio de 2012 7:59
  • Getting quite good performance on the RC with 5 drives (sizes between 750GB - 1.5TB), usually get around 50-70MB/s and it never pauses like it did in the beta where it just stopped writing to the disks for minutes.

    sábado, 02 de junio de 2012 6:06
  • I have a storage space with 23 disks over 6 controllers. They range in size from 250GB to 1TB (all 7200rpm 3.5" SATA). There's a ~10TB fixed size parity virtual disk over that space, with an NTFS volume created on that virtual disk.

    Copying from a fast array on a Server 2008 R2 machine over gigabit Ethernet, I have 1.57TB copied so far in 37 hours ~= 12 MB/second. That's really bad compared to even a single disk, let alone what you should expect from the raw capabilities of the hardware (hundreds of MB/sec). Even Windows software RAID5 gave 20MB/sec on the same hardware.

    Summary: don't expect any decent write performance from Storage Spaces - at least based off the RC.


    • Editado David Trounce lunes, 04 de junio de 2012 23:18
    • Propuesto como respuesta Mick E lunes, 10 de septiembre de 2012 20:55
    • Votado como útil Mick E lunes, 10 de septiembre de 2012 20:55
    lunes, 04 de junio de 2012 23:14
  • Hope this helps someone after some days fiddling. I was getting burst copy of up to 80Mbps on parity storage space drive and then dropping to zero before bursting again over and over. After a while I removed all files and put all the storage drives on my main 4 sata ports with the boot drive and another storage drive onto the Gigabyte motherboard chipset 2 ports separately. I then achieved a steady 10Mbps. This was still no good so then I realized in my bios I have IDE instead of AHCI selected. After this change I now achieve 30-40 Mbps writing. Not sure of read speed yet but it will be faster of course. Now I don't know if I spread another drive over the Gigabyte ports if it would affect the speed but it is logical that it wouldn't as it seems the system is storing the data in memory to ensure the data is written first SATA then switching to IDE as my drives were spread across the two types and ensuring the write is successful before saying OK give me the rest. Good Luck.          
    lunes, 10 de septiembre de 2012 20:55
  • I just bought 2 HP micro servers, as i will be replacing my HP Ultra 4 Nas boxs..

    Servers are running Windows 2012 Standard.

    Have configured Parity pool with x3 2TB drives..need to copy near 5TB - and it is slow, painfully slow.

    Initial burst is fast, then 10MB/sec..

    Microsoft MUST have a fix for this?

    Anyone?

    viernes, 09 de noviembre de 2012 20:59
  • If you have your server protected by a UPS you could run this PowerShell command:

    Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True

    It will boost parity write performance greatly.  There is a risk of data corruption if the server doesn't go down gracefully though.  You have to decide if it's worth the marginal risk.

    I went from 20MBps sustained write to 75+MBps on WD Green drives.   I've seen 120MBps write on Black drives.



    • Editado w The One w viernes, 09 de noviembre de 2012 21:52
    viernes, 09 de noviembre de 2012 21:51
  • Setting the storage pool to Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True didn't do anything to improve my storage spaces performance. It actually made it worse. Before entering this command I had a pattern of high-low-high-low throughput. Whereas high was about 50-70MBps and low about 0-25MBps and the interval between high and low is about 3-5 seconds. Now with -IsPowerProtected enabled the throughput pattern is high-low-low-low,whereas high is about 170MBps for the first 10-15 seconds and and than it stays low at about 1MB/s to 20MB/s.

    Here is the total time required to copy 8GB of data from a local SSD drive to the Storage Pool (3x 1TB 7200rpm drives)

    Total time set to $False = 3:35 minutes

    Total time set to $True = 4:01 minutes

    domingo, 25 de noviembre de 2012 16:42
  • Trying Storage Spaces for the first time on Windows 8 Pro.   3 Spindles, 2 TB each configured in a parity.    My writes are ranging between 10-15/MB/sec.   I'd rather move to a NAS over USB 2.0 if speeds continue to be this poor.     Copying back roughly 2TB of data I moved off these disks before configuring them is going to take days.   Love to hear if anyone has had any luck resolving the performance problems.

    domingo, 25 de noviembre de 2012 19:17
  • Did anybody try any block level caching software? RAM or SSD? To accelerate slow storage spaces?

    StarWind iSCSI SAN & NAS

    domingo, 25 de noviembre de 2012 19:33
  • I tried "Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True" initially without success.  But I then went into device manager/disk drives/policies and made sure all disks only had "enable write caching on the device" EXCEPT  for the one new disk drive for storage spaces that I had created called "Microsoft Storage Space Device" which I then checked the second box "turn off Window write-cache buffer flushing on the device". And since I do have an external ups for my server I am not worried about the warning for turning this off.  The problem with writes to disk pausing while the buffers flush and write to disk have all gone away.

    Twitch



    I am using a Storage pool with 3x2TB and 2x2TB disk.  The virtual disk is set to 10TB thinly provisioned, with 5 columns.
    • Editado DJ twitch miércoles, 26 de diciembre de 2012 20:24
    • Propuesto como respuesta DJ twitch miércoles, 26 de diciembre de 2012 20:24
    miércoles, 26 de diciembre de 2012 20:17
  • What sustained throughput are you getting, now that the stop-and-go behavior is gone?

    I've got a 3x 2TB pool of Seagate ST32000641AS drives, connected to Intel X58SO motherboard with an i7 XE CPU. "Enable write-caching on the device" and "Turn off write-cache buffer flushing on the device" are enabled for the Storage Space and all of the actual drives in the pool.

    I've been able to achieve an initial spike of 90-130 MB/sec. However, sustained transfers are about 40-50 MB/sec. I'm trying to figure out if this is the upper bound I'm ever going to get.

    lunes, 28 de enero de 2013 6:03
  • What sustained throughput are you getting, now that the stop-and-go behavior is gone?

    I've got a 3x 2TB pool of Seagate ST32000641AS drives, connected to Intel X58SO motherboard with an i7 XE CPU. "Enable write-caching on the device" and "Turn off write-cache buffer flushing on the device" are enabled for the Storage Space and all of the actual drives in the pool.

    I've been able to achieve an initial spike of 90-130 MB/sec. However, sustained transfers are about 40-50 MB/sec. I'm trying to figure out if this is the upper bound I'm ever going to get.

    Initial spike is becase you're filling your cache. After you're done you're doing cache flush @ background and THAT is an actual performance of your I/O subsystem. 

    StarWind iSCSI SAN & NAS

    lunes, 28 de enero de 2013 10:41
  • In case you are all still working on this, I have been too and stumbled across your thread.

    I bought 2012 Data Center, and have been running it on an upgraded box since it came out.  Server power is not a problem, but I'm getting 7.97MB/s sustained xfer using Storage Spaces + Parity (Fixed Maximum size - essentially RAID-5) + ReFS on five 2TB drives.  When copying a 3.2TB VHDX, RAM usage would go from 32GB to 99GB then performance dived. I didn't want to order my Backblaze POD until I could get this thing figured out...

    So, what I figured out is that it was writing to one drive of the five CONSTANTLY, while the others did nothing. After updating the BIOS, drivers and anything I could think of, I dismembered the Spaces a few times and went back to windows soft-RAID5... I found out that it didn't work either... Why? Caching was enabled on 4 of the 5 drives, and refused to enable on the fifth. So I disabled the cache on all drives (in Disk Manager, couldn't find it in Server Manager).  After setting up Spaces again, the drive lights are matching up beautifully, and are spreading the load.  The RAM usage is still incredibly high, but hey, it works now :)

    Specs:
    2x E5620
    144GB ECC
    8x 146GB SAS RAID-10
    5x 2TB RE4


    viernes, 01 de febrero de 2013 0:36
  • I was able to resolve my issues by moving everything over to a newly created but identical NTFS formatted partition with 3-Way MIRROR mode enabled (NOT Parity).

    After a lot of testing, I have come to the conclusion that neither ReFS, nor Parity-Based storage spaces are ready for "prime-time". I am now achieving 100-150 MB/s sustained writes and 300-600 MB/s sustained reads on a 3-way Mirror set running on 5 WD Red 3TB drives.

    I should note that I do have a UPS set up, and power settings and cache settings are set as such, but all testing was done on sustained speeds, rather than burst speeds. The caching settings only seemed to affect performance for small reads/writes, under the size of the collective cache of the drives (64MBx5).

    When running either Parity+ReFS, or Mirroring+ReFS, I never achieved writes faster than 20 MB/s, and with Parity+NTFS, writes never performed better than ~50 MB/s.

    Hopefully, Microsoft will work out the issues with ReFS and Parity under storage spaces, as the features offered are desireable, but certainly not at the performance cost being seen.

    • Propuesto como respuesta dreamwraith martes, 05 de febrero de 2013 23:14
    martes, 05 de febrero de 2013 23:13
  • Hello, I was just reading through the thread and wanted to add my findings from my home lab.

    System specs are : 4 x 2TB 5900 RPM HDD, 4GB DDR2, Core2Duo E6850, old ich9r controller.

    Original software RAID 5 array using the onboard ich9r would give me approx 90MB/s write and 250MB/s read.

    Using storage spaces (all disks assigned to one pool) and one virtual disk set in parity (one volume, NTFS), initial reads come in at 470MB/s read but only 20MB/s write.

    After using Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True suggested above, im now seeing sustained writes of 85MB/s. Tested transfer was 1.8TB of data of varying sizes.

    Perhaps this is an issue with ReFS ?

    miércoles, 24 de abril de 2013 10:26
  • does anyone know if ReFS, nor Parity-Based storage spaces performance issues been addressed in windows server 2012 R2
    martes, 04 de junio de 2013 1:59
  • I am experiencing the same issues and thus far, nothing has really helped (tried all the above).

    I have a server with 10 * 4TB drives attached in a parity storage space and although initial performance is great, after about say 5GB of data it nose dives terribly and then it's all over the shop. Initial speeds are around 1.5GB/s (not a typo) and then they drop down to under a few hundred meg a second with the occasional drop below 100.

    That's a monumental drop off. Software RAID performs exactly the same.

    martes, 04 de junio de 2013 9:06
  • Interesting. I disabled cache on all my drives (although it feels wrong to do it) and speed drop outs are definitely better. Now i drop down to about 300mb/s.

    I unfortunately have a difficult one to diagnose and work with, as I have two RAID controllers involved (a 4 port and an 8 port - hence using storage spaces to get one drive instead of hardware RAID).

    Definitely seeing an improvement with Windows drive cache off - but maybe that's because my RAID controller has its own cache?

    martes, 04 de junio de 2013 9:17
  • Interesting. I disabled cache on all my drives (although it feels wrong to do it) and speed drop outs are definitely better. Now i drop down to about 300mb/s.

    I unfortunately have a difficult one to diagnose and work with, as I have two RAID controllers involved (a 4 port and an 8 port - hence using storage spaces to get one drive instead of hardware RAID).

    Definitely seeing an improvement with Windows drive cache off - but maybe that's because my RAID controller has its own cache?

    Yes absolutely. Turn all caches OFF and you'll see lovely read-modify-write parity RAID w/o write coalescing (remember, NTFS or it's ulgy brother ReFS don't to it like WAFL does) performance being ~1/4 of your reads.

    StarWind iSCSI SAN & NAS

    martes, 04 de junio de 2013 14:46
  • I'm facing the same issue here...

    Could you explain where did you disable the write cache for your drive ? I have a storage space of 3 drives and tried to disable write cache for every one (Device Manager -> Disk drivers -> Policies -> Write-caching policy but this didn't help :( 

    I've tried to disable write-caching for the Microsoft Storage space device but it didn't allow me... :(


    sábado, 06 de julio de 2013 19:24
  • does anyone know if ReFS, nor Parity-Based storage spaces performance issues been addressed in windows server 2012 R2
    No. You really need log-structured file system or intermediate volume manager to fight parity RAIDs. ReFS and Storage Spaces don't do it.

    StarWind iSCSI SAN & NAS

    domingo, 07 de julio de 2013 14:57
  • I'm facing the same issue here...

    Could you explain where did you disable the write cache for your drive ? I have a storage space of 3 drives and tried to disable write cache for every one (Device Manager -> Disk drivers -> Policies -> Write-caching policy but this didn't help :( 

    I've tried to disable write-caching for the Microsoft Storage space device but it didn't allow me... :(


    I did it in my RAID controller BIOS.

    StarWind iSCSI SAN & NAS

    domingo, 07 de julio de 2013 14:58
  • storage pools are at their best with rows of servers, not individual disks

    first problem, fault tolerance when thing go wrong

    second problem, everybody using server grade disks? Desktop disks are not suitable.

    RAID cards are used in servers to host the disks, rack after rack of server is what the storage is best seen as

    think cloud

    1) That's called a RAIN and not RAID. Different thing. Few companies did it with a general lack of success. Think about Lefthand Networks and their issues to scale beyond 4-5 node array.

    2) There's no problem with fault tolerance. Mirrored metadata between "controller" nodes and some way to distribute actual data between "storage" nodes (erasure codes, hash etc). Again tons of companies do this. Think about NEC HYDRAstor, Hitachi HCP etc. The only problem is - these things are real Enterprise hardcore, space Windows physically don't belong to (at least yet).

    3) There's no magic behind Enterprise disks. 2x-3x times more IOPS using the same capacity and that's pretty much all. And there's nothing wrong with NL-SAS and NL-SATA spindles. People need capacity not all is hunting for IOPS. Properly implemented Logical Volume Manager of any kind should work with both. ZFS for example does...

    4) Cloud is not a panacea. Paying for transactions on routine basis is not what most of the people want to do (at least yet).


    StarWind iSCSI SAN & NAS

    domingo, 07 de julio de 2013 15:05
  • does anyone know if ReFS, nor Parity-Based storage spaces performance issues been addressed in windows server 2012 R2

    I am using some 64TB VHDX files that are formatted with ReFS as data storage. So far no problems.

    What performance numbers are you getting from parity (do you happen to use double parity?) storage spaces with WS 2012 R2?

    StarWind iSCSI SAN & NAS

    domingo, 07 de julio de 2013 20:22