Parity Storage Space so slow that it's unusable
-
Tuesday, March 13, 2012 1:13 PM
Using the same pool comprising of multiple 1.5TB/2TB WD&Seagate drives on a Core 2 Quad System.
Regular Storage Space shows performance over 100MB/sec
Mirror Storage Space shows performance of around 80MB/sec
Parity Storage Space shows performance of 80MB/sec initially but as you copy a large file the performance keeps dropping until it reaches single digits making this pool option basically unusable. Is this a bug or is it expected behaviour?
All Replies
-
Thursday, March 15, 2012 8:17 PM
I'm not benchmarking the feature but have been performing similar measurements to get a feel for what is possible and the impact of choosing thin provisioning etc. I've had some very strange results too variable to be statistically significant, but I can't work out what to expect. This variability could well be down to device drivers rather than the volume manager/software raid layer. Saying that my past experience of software RAID performance in Windows, both native and third party ( Veritas SFW ), has been poor. It has to be said storage spaces look very like SFW.
Out of interest, how many independent spindles are in your pool? Do you see high CPU when utilising parity? If you create a pool from a subset of spindles and perform I/O on a volume in the pool do you see activity on all the spindles in your machine or just those in the pool?
I assume "simple" layout is equivalent to RAID-0. What isn't clear is if all the disks in a pool are involved when you select mirror or parity. What are mirror and parity equivalent to in RAID levels, 10 & 50 respectively? But if there are three disks and you choose mirror as the storage layout do you get three plexes or two plexes with one spindle unused, certainly when I create a 20GB mirror in a pool with 3 physical devices I lose 40GB from the pool, does that mean there are two plexes regardless of the pool physical disk count? What if you choose mirror and the pool contains 7 physical disks, do you get a stripe or concatenation of mirrors where the submirrors are a mix of two and three plexes? The same question applies to the parity layout option and what happens when you mix virtual disk layouts within a pool?
Why is a 1GB allocation retained on any disk which has been part of a virtual disk? I need to use the clean operation in diskpart to have the free space returned to expected. As another poster has pointed out disks which are not returned to the primordial pool prior to the OS being reinstalled have limited functionality when the existing pool is "rediscovered" by the new OS install. It seems impossible to delete such a pool and the only way I have found to return the disks to the primordial pool is to boot to a command prompt from W2K8R2 media and perform a clean on the disks from dispart.
-
Thursday, March 15, 2012 8:24 PM
Some of your questions need to be answered by MSFT. Right now the server has 2 1.5TB drives an 1 2TB drive and a 60GB SSD as the OS drive. I created one storage pool that includes both 1.5TB drives and the 2TB drive. Within that pool I created the different types for Spaces for my testing. So, at the pool level, there is no mirror or parity settings. That happens at the space level. The CPU is a Core 2 Quad 2.5Ghz and it is never stressed by the Mirror or Parity writes.
-
Friday, March 16, 2012 10:38 AM
Thanks, I know some of my questions are for Microsoft, I was just putting my thoughts out there as it is impossible to quantify performance without understanding roughly what happens under the lid. With only three spindles in the pool some of my own questions aren't relevant to you.
I've performed a similar test with data copied from a simple virtual disk in a three spindle pool to simple, mirror and parity virtual disks carved out of a separate three spindle pool. In my case the disks in question are 6*15K SAS. I've also peformed tests using iometer against iSCSI virtual disks on each of the virtual disks. I didn't experience the same behaviour as you with a parity virtual disk, my write speeds were variable throughout the tests. However, the parity pool was an order of magnitude slower than simple or mirror virtual disks so yes I'd agree that it isn't a viable choice at present.
You'd think at least one of the cores in your system would be showing load when calculating parity but like you mine ( 2 * Opteron 4122 ) shows virtually no load when writing to parity virtual disks. Nothing new there though, this was the same in 2003/2008. It may be that the threads performing the parity calculation are artificially throttled but if that is the case it definitely needs to be tuneable in server "8" if it is to be used purely as a storage platform.
I've stopped looking at Storage Spaces for now, it works as a technology preview and will be neat when it is production ready.
-
Sunday, March 18, 2012 8:03 PM
I have been experimenting with Storage Spaces on my previous Windows Home Server 2011 box (Intel Xeon X3450 16gb RAM)
I have 6 2tb drives in a thin provisioned parity storage pool.
When copying files (1+gb files), it will do an initial burst of writing then stop completely. I notice that in the Resource Monitor/Disk tab, it shows the file being written so I can see it is still writing. It will start copying again once the data is flushed to the hard drives, although it takes quite a while. I can understand this if the system is calculating parity information but my CPU is never spiked either. It almost needs to have the copy slowed down so the system can keep up.
Copying from a Windows 7 machine to the Windows 8 server over a gigabit network will give errors such as the file needs to be replaced and the destination file shows 0 bytes even though I am copying it for the first time. Also copying from more than one source at the same time to the storage pool will occasionally disconnect the server from the network and cause error connecting to \\server\share messages. Hitting retry on the copy will start it again but then stop. This makes it difficult to do unattended file copying.
-
Tuesday, March 27, 2012 8:37 AM
I've experienced the same thing, running on a Intel Core 2 Quad (Q6600, P45 motherboard) with 5 drives in the storage pool. Striped and mirrored virtual disks work fine but as soon as I try to copy things onto the parity disk everything just goes down the drain, it seems to copy data in "bursts" and then just stop and wait for minutes (copying 50GB of data from another local disk took hours).
-
Sunday, April 01, 2012 5:52 PMFrom all the evidence I can gather so far, the Parity pools get pretty good read performance (essentially N - 1 drive speed), but the writes are stuck right around 25MB/s.
That's glacially slow considering how fast the processors are, they should easily be able to calculate parity faster than that.
I have a feeling that Microsoft is playing it too safe and not caching any write back data. On my Intel RAID 5, the same thing happens if I disable the write cache, the speed becomes unusable.
So, I'm waiting for the final version to see if Microsoft improves things. I believe they allow you to do things like use an SSD for the journal log. Maybe you can specify a write back cache option with the powershell commands. -
Wednesday, April 04, 2012 5:25 AMMy question is if there is Storage Pools in Windows 8 Consumer Preview and in Windows Server 8, are they both the same or is the one in Windows Server 8 beta? I am hoping this one is beta or else many users will be disappointed with this kind of performance.
-
Monday, April 30, 2012 8:23 PMI try Parity storage pool with 3 x 500gb (7200) and I have the same "slow" write issue. I have something like 20-30MB/sec. Really slow !
-
Monday, April 30, 2012 11:26 PM
Wow, I have a spaces test setup that I'm trialling, 7 1tb drives divided into two thinly provisioned spaces, my games space which has no redundancy and the test space which has two drive redundancy.
I get bursts of about 30m/b write which is about what I expected, then it sharply drops off after a while to about 250k/s. Quite frustrating when I'm trying to copy the contents of a 2TB drive.
Is there a Connect space to report this too? I haven't seen any.
-
Friday, May 18, 2012 4:04 PMCan confirm the same. When copying large amount of data/files to parity disks, writepe rformance droups untilly fully stops or disk disappear from My Computer. With Simple disks no issues.Waiting for Server 8 RC, hope guys will fix this issue
MCSE 2003: Messaging, Security
MCITP: Server Administrator, Enterprise Administrator, Virtualization Administrator
MCITP: Consumer Desktop Support, Enterprise Desktop Support, Enterprise Desktop Support Windows 7, Enterprise Desktop Administrator
SMS/SCCM, ISA 2004/2006, CCNA Security -
Tuesday, May 29, 2012 4:11 PM
I am getting the same issue on Server 8 Beta, I have 5x 500GB SATA disks making a storage pool of 2.27TB and a 2TB virtual disk with Thin provisioning and Parity layout, formatted in ReFS.
Did some testing with it today and as all others have found the transfer starts off maxing out the network at around 100-110MB/s then about 1/4 of the way through almost pauses for a few seconds and the rest of the transfer continues at around 25MB/s with some copying it starts to ask if it can over write the file.
The RC is out next week so lets see if that fixes it.
-
Friday, June 01, 2012 7:03 AMAnyone tried parity performance on the Release Preview yet? Just curious.
-
Friday, June 01, 2012 7:59 AMOn Server 2012 RC, I got ~30MB/sec in an 18 minute test on an NTFS formatted parity volume across 3x 250GB 7200rpm disks. Better, but still not great.
-
Saturday, June 02, 2012 6:06 AM
Getting quite good performance on the RC with 5 drives (sizes between 750GB - 1.5TB), usually get around 50-70MB/s and it never pauses like it did in the beta where it just stopped writing to the disks for minutes.
-
Monday, June 04, 2012 11:14 PM
I have a storage space with 23 disks over 6 controllers. They range in size from 250GB to 1TB (all 7200rpm 3.5" SATA). There's a ~10TB fixed size parity virtual disk over that space, with an NTFS volume created on that virtual disk.
Copying from a fast array on a Server 2008 R2 machine over gigabit Ethernet, I have 1.57TB copied so far in 37 hours ~= 12 MB/second. That's really bad compared to even a single disk, let alone what you should expect from the raw capabilities of the hardware (hundreds of MB/sec). Even Windows software RAID5 gave 20MB/sec on the same hardware.
Summary: don't expect any decent write performance from Storage Spaces - at least based off the RC.
- Edited by David Trounce Monday, June 04, 2012 11:18 PM
- Proposed As Answer by Mick E Monday, September 10, 2012 8:55 PM
- Unproposed As Answer by Mick E Monday, September 10, 2012 8:55 PM
-
Monday, September 10, 2012 8:55 PMHope this helps someone after some days fiddling. I was getting burst copy of up to 80Mbps on parity storage space drive and then dropping to zero before bursting again over and over. After a while I removed all files and put all the storage drives on my main 4 sata ports with the boot drive and another storage drive onto the Gigabyte motherboard chipset 2 ports separately. I then achieved a steady 10Mbps. This was still no good so then I realized in my bios I have IDE instead of AHCI selected. After this change I now achieve 30-40 Mbps writing. Not sure of read speed yet but it will be faster of course. Now I don't know if I spread another drive over the Gigabyte ports if it would affect the speed but it is logical that it wouldn't as it seems the system is storing the data in memory to ensure the data is written first SATA then switching to IDE as my drives were spread across the two types and ensuring the write is successful before saying OK give me the rest. Good Luck.
-
Friday, November 09, 2012 8:59 PM
I just bought 2 HP micro servers, as i will be replacing my HP Ultra 4 Nas boxs..
Servers are running Windows 2012 Standard.
Have configured Parity pool with x3 2TB drives..need to copy near 5TB - and it is slow, painfully slow.
Initial burst is fast, then 10MB/sec..
Microsoft MUST have a fix for this?
Anyone?
-
Friday, November 09, 2012 9:51 PM
If you have your server protected by a UPS you could run this PowerShell command:
Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True
It will boost parity write performance greatly. There is a risk of data corruption if the server doesn't go down gracefully though. You have to decide if it's worth the marginal risk.
I went from 20MBps sustained write to 75+MBps on WD Green drives. I've seen 120MBps write on Black drives.
- Edited by w The One w Friday, November 09, 2012 9:52 PM
-
Sunday, November 25, 2012 4:42 PM
Setting the storage pool to Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True didn't do anything to improve my storage spaces performance. It actually made it worse. Before entering this command I had a pattern of high-low-high-low throughput. Whereas high was about 50-70MBps and low about 0-25MBps and the interval between high and low is about 3-5 seconds. Now with -IsPowerProtected enabled the throughput pattern is high-low-low-low,whereas high is about 170MBps for the first 10-15 seconds and and than it stays low at about 1MB/s to 20MB/s.
Here is the total time required to copy 8GB of data from a local SSD drive to the Storage Pool (3x 1TB 7200rpm drives)
Total time set to $False = 3:35 minutes
Total time set to $True = 4:01 minutes
-
Sunday, November 25, 2012 7:17 PM
Trying Storage Spaces for the first time on Windows 8 Pro. 3 Spindles, 2 TB each configured in a parity. My writes are ranging between 10-15/MB/sec. I'd rather move to a NAS over USB 2.0 if speeds continue to be this poor. Copying back roughly 2TB of data I moved off these disks before configuring them is going to take days. Love to hear if anyone has had any luck resolving the performance problems.
-
Sunday, November 25, 2012 7:33 PMDid anybody try any block level caching software? RAM or SSD? To accelerate slow storage spaces?
-
Wednesday, December 26, 2012 8:17 PM
I tried "Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True" initially without success. But I then went into device manager/disk drives/policies and made sure all disks only had "enable write caching on the device" EXCEPT for the one new disk drive for storage spaces that I had created called "Microsoft Storage Space Device" which I then checked the second box "turn off Window write-cache buffer flushing on the device". And since I do have an external ups for my server I am not worried about the warning for turning this off. The problem with writes to disk pausing while the buffers flush and write to disk have all gone away.
Twitch
I am using a Storage pool with 3x2TB and 2x2TB disk. The virtual disk is set to 10TB thinly provisioned, with 5 columns. -
Monday, January 28, 2013 6:03 AM
What sustained throughput are you getting, now that the stop-and-go behavior is gone?
I've got a 3x 2TB pool of Seagate ST32000641AS drives, connected to Intel X58SO motherboard with an i7 XE CPU. "Enable write-caching on the device" and "Turn off write-cache buffer flushing on the device" are enabled for the Storage Space and all of the actual drives in the pool.
I've been able to achieve an initial spike of 90-130 MB/sec. However, sustained transfers are about 40-50 MB/sec. I'm trying to figure out if this is the upper bound I'm ever going to get.
-
Monday, January 28, 2013 10:41 AM
Initial spike is becase you're filling your cache. After you're done you're doing cache flush @ background and THAT is an actual performance of your I/O subsystem.What sustained throughput are you getting, now that the stop-and-go behavior is gone?
I've got a 3x 2TB pool of Seagate ST32000641AS drives, connected to Intel X58SO motherboard with an i7 XE CPU. "Enable write-caching on the device" and "Turn off write-cache buffer flushing on the device" are enabled for the Storage Space and all of the actual drives in the pool.
I've been able to achieve an initial spike of 90-130 MB/sec. However, sustained transfers are about 40-50 MB/sec. I'm trying to figure out if this is the upper bound I'm ever going to get.
-
Friday, February 01, 2013 12:36 AM
In case you are all still working on this, I have been too and stumbled across your thread.
I bought 2012 Data Center, and have been running it on an upgraded box since it came out. Server power is not a problem, but I'm getting 7.97MB/s sustained xfer using Storage Spaces + Parity (Fixed Maximum size - essentially RAID-5) + ReFS on five 2TB drives. When copying a 3.2TB VHDX, RAM usage would go from 32GB to 99GB then performance dived. I didn't want to order my Backblaze POD until I could get this thing figured out...
So, what I figured out is that it was writing to one drive of the five CONSTANTLY, while the others did nothing. After updating the BIOS, drivers and anything I could think of, I dismembered the Spaces a few times and went back to windows soft-RAID5... I found out that it didn't work either... Why? Caching was enabled on 4 of the 5 drives, and refused to enable on the fifth. So I disabled the cache on all drives (in Disk Manager, couldn't find it in Server Manager). After setting up Spaces again, the drive lights are matching up beautifully, and are spreading the load. The RAM usage is still incredibly high, but hey, it works now :)
Specs:
2x E5620
144GB ECC
8x 146GB SAS RAID-10
5x 2TB RE4
-
Tuesday, February 05, 2013 11:13 PM
I was able to resolve my issues by moving everything over to a newly created but identical NTFS formatted partition with 3-Way MIRROR mode enabled (NOT Parity).
After a lot of testing, I have come to the conclusion that neither ReFS, nor Parity-Based storage spaces are ready for "prime-time". I am now achieving 100-150 MB/s sustained writes and 300-600 MB/s sustained reads on a 3-way Mirror set running on 5 WD Red 3TB drives.I should note that I do have a UPS set up, and power settings and cache settings are set as such, but all testing was done on sustained speeds, rather than burst speeds. The caching settings only seemed to affect performance for small reads/writes, under the size of the collective cache of the drives (64MBx5).
When running either Parity+ReFS, or Mirroring+ReFS, I never achieved writes faster than 20 MB/s, and with Parity+NTFS, writes never performed better than ~50 MB/s.
Hopefully, Microsoft will work out the issues with ReFS and Parity under storage spaces, as the features offered are desireable, but certainly not at the performance cost being seen.- Proposed As Answer by dreamwraith Tuesday, February 05, 2013 11:14 PM
-
Wednesday, April 24, 2013 10:26 AM
Hello, I was just reading through the thread and wanted to add my findings from my home lab.
System specs are : 4 x 2TB 5900 RPM HDD, 4GB DDR2, Core2Duo E6850, old ich9r controller.
Original software RAID 5 array using the onboard ich9r would give me approx 90MB/s write and 250MB/s read.
Using storage spaces (all disks assigned to one pool) and one virtual disk set in parity (one volume, NTFS), initial reads come in at 470MB/s read but only 20MB/s write.
After using Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True suggested above, im now seeing sustained writes of 85MB/s. Tested transfer was 1.8TB of data of varying sizes.
Perhaps this is an issue with ReFS ?
.png)
