Expanding a RAID 5 array?

Viper007Bond

Viper007Bond

Chicken Nugget Connoisseur
STAFF MEMBER
Joined
Sep 21, 2003
Messages
31,044
Location
Portland, Oregon
Car(s)
2008 Dodge Viper, 2006 MB CLS55 AMG
I'm out of space again and rather than buying another Synology NAS, I'm going to build a HTPC and move my Plex server from my desktop to it while I'm at it. I'm going to run Windows 8 on it just because that's what I'm comfortable with and it'll be able to run all the apps I want to run.

I have some spare parts laying around that I plan on using including my old ASUS P7P55D-E PRO motherboard from 2010.

My plan is to stick an old 640GB WD Black on the Marvell controller as my OS drive and then build a RAID 5 array using the Intel controller.

To save money, I was thinking about only buying three WD Red drives to begin with (4, 5, or 6 TB, not sure yet) and then as I run out of space, adding more of the same drives to the array until I reach the 6 drive limit (number of ports the Intel controller has). This is possible without data loss, right? I assume it'll take a while to add the new drive as it has to move around the blocks but that doesn't bother me. I just want safe, reliable, and large storage.

I'll be buying a UPS for this of course.
 
Also in the event that my motherboard's Intel chipset doesn't support expansion, then I'll pick up a controller that does, assuming no one recommends against RAID 5 expansion.
 
The instruction manual for your motherboard is of the opinion that adding a drive will nuke the array... You'll probably have to either purchase a RAID card or buy all of your drives at once.
 
If it's gonna be a low-load server like an HTPC or NAS, just use software RAID of some form over hardware RAID or fakeRAID (embedded Intel ICH/PCH RAID)

If you need more ports, get a SAS controller and expanders as needed.
 
Hmm, I'm thinking maybe I use the Intel controller to make a RAID 1 for my OS (two drives) and then use the other 4 Intel ports plus the 2 Marvell ports to make a software 6 drive RAID via Windows.

 
Last edited:
I said fuck it and just blew $1560 on six 6TB hard drives. They should be here later this week. Hopefully they work well using the onboard Intel controller otherwise I'll look at doing fully software RAID.
 
Here is my current set-up which has been working for me over the last few years.

OS: FreeBSD
Filesystem/RAID: ZFS RaidZ2 (I started with raidz[ like RAID 5], but moved over to raidz2 [like RAID 6]).
Motherboard: I've used various SuperMicro Intel Atom server boards. I've recently upgraded to an A1SAi-2550F.
Hard drives: 2 x 3TB Seagate Constellation CS and 3 x 3TB WD Red drives. (I'm not at all happy with latter and will phase them out for Seagate drives).
Memory: Now I have 8GB of ECC which has proven to be enough for my 9TB array. ZFS was designed for the datacenter and expects ECC memory. Faulty memory without error checking can easily hose an array. With ECC that can't happen (the error will either be corrected automatically or the system will crash. Either way the data are safe).

So far I haven't had any issues. I've had some corruption detected while doing a scrub, but ZFS fixed it (it does this automatically when you read the file, scrubs force a read of all files).

I moved to radiz2 as I was uncomfortable running in a degraded state (no ability to correct errors) while waiting for a dead hard drive to be replaced. Raidz2 is essentially RAID 6 (you can loose a drive and still have redundancy).

ZFS will increase the size of the pool when all the drives have increased in size. I had a 6TB pool that automatically became a 9TB pool after I replaced my 2TB drives with 3TB ones.

I've stopped using hardware RAID as I've found it buggy and limiting (you're stuck with the same vendor and risk being locked into a scheme that 3Ware or Intel may abandon later on). ZFS provides better protection anyhow.

I prefer stock FreeBSD, but FreeNAS provides a snazzy web based interface. Despite being an Atom I've had no issues with my server streaming PLEX (does 1080p just fine using both the app and the web interface).

As for hard disks, I'm moving to Seagate Constellation CS disks. They are quieter than the Reds and I haven't had to RMA them (all my REDs are RMA). And yes, I bought my current disks months apart so they aren't from the same batch.
 
On the matter of having a separate array for the OS, don't bother. I use a single array on my fileserver, that way the OS is as protected as the rest of the data. I haven't looked at Windows software RAID for awhile, but FreeBSD allows booting to a ZFS RAID. The boot loader is smart enough to handle it. I wrote the boot loader to each disk so any of them can bring up the OS.
 
ZFS seems nice but there is no way I'm buying 32GB of ECC RAM. I just blew a shit ton of money on hard drives. :(

Regardless I want to run Plex Home Theater on this same box rather than having separate storage and TV machines. It's only supported on Windows and Mac OS officially and I don't really want a headless unit.
 
I'd recommend using Windows software RAID. Hardware raid is lam-o-drain-o for the reasons I've mentioned above. Unlike ZFS I don't believe it requires ECC to function well (ZFS performs a checksum every time a file is read; which is why ECC is a requirement. If the ZFS checksum database is inconsistent with the one done in memory it trusts the one in memory).
 
argatoga said:
I'd recommend using Windows software RAID. Hardware raid is lam-o-drain-o for the reasons I've mentioned above. Unlike ZFS I don't believe it requires ECC to function well (ZFS performs a checksum every time a file is read; which is why ECC is a requirement. If the ZFS checksum database is inconsistent with the one done in memory it trusts the one in memory).
Click to expand...

Windows Storage Spaces under Windows 8.1 seemed rather lame and slow in my testing but I'll do some more testing once the drives arrive. Not sure what alternative there is to that other than the weird third party snapshot RAIDs or pooling software.
 
If you are using the pc for playing video you don't need massive i/o.
 
argatoga said:
If you are using the pc for playing video you don't need massive i/o.
Click to expand...

Read speed is fine but write speed sucks. When I torrent a new file, it allocates out the whole space and can't start writing actual downloaded bits until that finishes.

I'll test later this week and see for sure. Maybe the real drives will be better.
 
It may be faster if you use ReFS (which is what I believe Microsoft uses for RAID now). Microsoft may have speed up creating zero'd files with it. I am able to make a 1GB zero file in 1 second on my file server, I wouldn't be surprised if ReFS is as fast.
 
Last edited:
argatoga said:
It may be faster if you use ReFS (which is what I believe Microsoft uses for RAID now). Microsoft may have speed up creating zero'd files with it. I am able to make a 1GB zero file in 1 second on my file server, I wouldn't be surprised if ReFS is as fast.
Click to expand...

ReFS isn't supported for parity (essentially RAID 5) within Storage Spaces. :(
 
Windows 10? Yes, it is in the preview stage, however if it supports parity it may be worth using.
 
Viper007Bond said:
ZFS seems nice but there is no way I'm buying 32GB of ECC RAM. I just blew a shit ton of money on hard drives. :(

Regardless I want to run Plex Home Theater on this same box rather than having separate storage and TV machines. It's only supported on Windows and Mac OS officially and I don't really want a headless unit.
Click to expand...

If you're willing to wait a bit and scour secod-hand gear, 32GiB of ECC DDR3 isn't particularly expensive. Motherboard/processor support is much more of a PITA, though even then not as bad as you'd think.

Honestly, if I had the space (a basement/closet) where I could stuff a few rackmount boxes, I'd probably be running a a Dell R720 with 256+GiB of DDR3.. for under $800 shipped cause second-hand as all fuck.
 
Fucking hell. Got the drives today but the motherboard is so old that the Intel controller only sees them as 1.4TB drives. I wasn't really paying attention when testing with the 2TB drives thinking it was just the normal loss.

Time for a new motherboard and CPU...
 
You must log in or register to reply here.
Top