Raid 5 guestions??

[patrickl]

Yes, I am planing on raid 6 plus 1 hotspare

Oh wow. That's some serious overhead then though. Then you have 3 disks that don't add to the total storage space.

[meismr]

If you're going to have three drives that don't add to the actual disk space why not go with 3 Raid 1 arrays.  That way if you have a double drive failure you can still have the option of sending your drive out to a recovery center to get it back.  I'm biased though.  I use Raid 1 for myself exclusively.  So if you only have a hammer everything looks like a nail

As a late post addition to a previous question;  There are a couple of good reasons to keep your OS and data on different drives, that's why the enterprise level places do it   For instance, our company recently wanted to do a large OS update (win2k server to win2k3 server)  They took out the old OS drive and put in a new OS drive with the OS already installed.  If the new OS didn't take they simply had to put the old OS drive back in. 

Another reason to separate the data and OS layers is to keep the data 'clean'.  If you have drives with data and program files then the data drives have a lot more non-data files in them...  cluttering them up.

Finally there's speed to think of.  Your OS uses one drive (usually.  I've seen some people Raid 1 their OS (separate from their data) but I think that's overkill) and the data read is bottlenecked by the drive.  If you're performing an OS task and trying to get data off the drive then you're impacting the data retrieval performance.

[squirrellydw]

I don't plan on doing raid 6 plus 1 right away, but add to it in the furtue

[patrickl]

There are a couple of good reasons to keep your OS and data on different drives, that's why the enterprise level places do it  
...
Your OS uses one drive (usually.  I've seen some people Raid 1 their OS (separate from their data) but I think that's overkill)

When you use a single disk for the OS (as opposed to a mirrored OS array) you cannot really claim "it's what enterprise level places do" anymore, because that's not what they use It would create a single point of failure and they would not do that.

The choice is between a single OS drive or an OS partition protected by RAID. In that case I'm pretty sure any enterprise level place would opt for the OS as a partition on the RAID array. Besides this isn't a enterprise environment, but a home environment. Although with RAID 6 and a hot spare ...

But indeed if you want to be able to install a new OS by swapping disks or if you are worried about performance of the server (which I personally wouldn't in the case of a file server, but still) then keeping the OS on a separate disk would be a beter idea. I don't quite get the "clutter" point.

[meismr]

Enterprise level places most certainly DO keep their data and OS on different drives.  I didn't say enterprise places keep their OS on a single drive. 

This may not be an enterprise solution, but a home solution can still emulate a professional one.  Our 'generic' file servers at work are 6 disk solutions.  4 data drives all in raid 1 and 2 OS drives in Raid 1.  These are in a clustered environment as well...  This is extreme for the home.  What isn't extreme is a paired down version of this.  2 data drives in Raid 1 and a single OS drive (with images on a disk or on the other file server).

File retrieval performance is also a pretty large worry for a file server in general.  If you're working with small files and your primary concern is data protection then it becomes less of one, but if you're editing movies or opening .net solutions...  it becomes a larger one. 

[squirrellydw]

I will be using it to access music, movies, and photos.  I am using as a media server.

[patrickl]

Enterprise level places most certainly DO keep their data and OS on different drives.  I didn't say enterprise places keep their OS on a single drive. 

And I didn't say that they don't keep their OS separate. I said that they don't do it on a single drive.

[meismr]

If this is a media server (movies, large picture files, etc.) then I would separate my OS and data.  If this is an MP3, JPG kinda media server then you should be able to put both the OS and data on the same drive array. 

Patrickl: Since we're both saying the same thing let's not derail the thread.

[patrickl]

If this is a media server (movies, large picture files, etc.) then I would separate my OS and data.  If this is an MP3, JPG kinda media server then you should be able to put both the OS and data on the same drive array. 

Why would that matter?

Patrickl: Since we're both saying the same thing let's not derail the thread.

We don't agree. You simply misread my post. You make a recomendation that goes against the principle of using a redundant array (ie introduce a non-redundant part) so I think it has relevance to the thread.

[meismr]

Fair enough, we don't agree.  I didn't misread your post.  You were wrong.  I said enterprise solutions separate their OS and data drives.  I'm right. 

If you're streaming large files over the network (like movies, large picture files, etc.) then separating your data and OS would improve performance.  If you don't then you may not have a high enough read rate and the movie will skip.

[patrickl]

Fair enough, we don't agree.  I didn't misread your post.  You were wrong.  I said enterprise solutions separate their OS and data drives.  I'm right. 

You missed the point again. You said that there is no need to mirror the OS drive. An enterprise solution never has an OS disk that isn't (at least) mirrored.

I only asked you to explain why you felt it that important to separate the OS and data that you would even destroy the redundancy of the system just to achieve this separation. That doesn't mean I disagree with separation. I just think that redundancy is the key in a redundant system, while you seem to think that ease of OS installation by swapping disks is what's more important.


Anyway, talking about a truly massive enterprise solution, see the Google drive failure analysis (PDF). I found it pretty informative. Surprising too since they state that drive temperature didn't seem to matter much.

They also say that SMART errors don't always warn ahead of disk failures/errors. Still a pretty high percentage is predicted. It's good to check the logs for those or have them e-mailed to you.

[squirrellydw]

agree to disagree or agree, I don't care.   I get the point you are both trying to make and I appreciate all your comments and suggestions.  I am going to start with 3 drives in a raid 5 then go to a raid 6 then add a hot spare.  The OS will be on a separate drive.

Any other suggestions??

[boykster]

I know I said it before, but it deserves re-stating:

buy a good power supply. 

single most important component in a large disc array, even more so than the controller.  A super expensive controller card with a crappy PSU will be less reliable than a bunch of disks in JBOD using software paired with a high quality PSU.

[patrickl]

I know I said it before, but it deserves re-stating:

buy a good power supply. 

single most important component in a large disc array, even more so than the controller.  A super expensive controller card with a crappy PSU will be less reliable than a bunch of disks in JBOD using software paired with a high quality PSU.

Indeed. I've had at least three PSU's die on me (in workstations though). Although oddly enough those were actually quality PSU's.

I ran a HP Netserver LH3000 as file RAID server and it actually had redundant power supplies. That was a cool machine. Unfortunately it made more noise (and wind) than two vacuum cleaners too 

Don't underestimate the failure rate of hard drives used in a 24/7 application. If you read the manufacturers specs you'd expect a drive failure rate of 1% (for enterprise quality disks) and 3% (for consumer grade disks). Unfortunately in real life studies (like the Google study I pointed to earlier) they report annual failure rates of 6% and up to 12%.

Also check the warranty. I always return a broken disk if it breaks within the warranty period. Keep the box that you received the disks in, because they often have strict rules about the packaging of returned disks. I found that I often get a bigger disk back than what I returned. In one case I had to pay an administration fee (Seagate). A few months ago I returned two disks that started showing SMART errors and I got new ones for those too.

Also think about wether you want to spin down the drives or not and think about using enterprise disks (which are specifically designed for 24/7 RAID solutions) as opposed to consumer grade disks (which are designed for 8/5 office hour applications).

[squirrellydw]

OK, a few more questions now.

What is JBOD
What is considered an enterprise disk
what do you mean by spin down the drives

Thanks

[patrickl]

OK, a few more questions now.

What is JBOD

Not exactly what Boykster meant there, but it's 'Just a Bunch Of Disks'. Guess he means that the whole solution is always as strong as the weakest link. That's also why I'm not in favor of a non-redundant OS disk.

What is considered an enterprise disk

Enterprise disks are designed to work in 24/7 applications. They have a higher MTBF rating (ie less likely to break) to offset the fact that they will be running a lot more in a 24/7 application than the 8/5 application of a desktop computer.

In the days of ATA (now called PATA) these only existed as SCSI disks, but these days you can buy them with a SATA or SAS interface.

Problem is that these disks are more expensive (quite a lot actually), but they should be better suited to the task. Google uses consumer grade disks though. Guess they don't think it's work the extra money.

For instance the Seagate Barracuda 7200.11 versus the Barracuda ES.2.

what do you mean by spin down the drives

When there is no disk activity you can make the disks stop spinning. This saves power, reduces heat and can prolong the life of the drive (especially a consumer disk which is built to run during office hours only).

This is also one of these areas where you will hear different opinions. Mostly depending on how old the info that the person you talk to is using. In the past, disks had to keep on running all the time because starting them up could make the motor seize and ruin the drive. The same can still happen, but it's less likely these days.

Disks are actually rated for a number of spin-down/spin up cycles too.

BTW putting the OS on an array makes it about impossible to let array spin down since the OS often writes something to disk. Since you are not putting the OS on the array spinning down the disks is something you could consider.

:edit: spell disaster

[squirrellydw]

Cool, I have both a 7200.11 and a ES drive.  I never realized what the difference was.

[boykster]

OK, a few more questions now.

What is JBOD

Not exactly what Boykster meant there, but it's 'Just a Bunch Of Disks'. Guess he means that the whole solution is always as strong as the weakest link. That's also why I'm not in favor of a non-redundant OS disk.

[/quote]

That's what I was referring to, I guess saying 'a bunch of disks in JBOD' is redundant.  I guess what I should have said was a bunch of disks in RAID0 spanning or just attached as individual drives.  All data, no redundancy.

I agree on not considering MTBF an insurance policy, and on keeping packing materials for your drives.  I now exclusively have been buying Seagate drives since they offer a 5 year warranty.  I'm in the "don't spin down" camp, but there are valid arguments on both sides of the fence on that one.

[patrickl]

I now exclusively have been buying Seagate drives since they offer a 5 year warranty.

Did you have to pay an administreation fee? I really hated that. I bought 4 Seagates. 2 of them failed within a few months and they wanted something like 50 euro administration fee.

A few months ago I sent some Maxtor drives back (to Seagate since they bought Maxtor) and I didn't have to pay.

I'm in the "don't spin down" camp, but there are valid arguments on both sides of the fence on that one.

I'm on both sides and I use both depending on the situation.

Looking at the annual failure rates that Seagate reports for using a desktop drive 8/5 versus an enterprise disk used 24/7 shows that the enterprise drive is twice as likely to fail. So a drive with almost double the MTBF is still twice as likely to die when it is used 4 times as much. That shows that at least Seagate thinks that running the drive twice as many hours ages the drive twice as much. ie spinning it down will prolong the life.

At 8 Watt a drive with 5 drives that adds up to 40 Watt. It's not spectacular, but still.

On the other hand spinning down and up too often can kill the disk too.

Spinning a drive back up can take 15 seconds. It's annoying if you have to wait for that.

So on my webservers I don't spin down and my office file server does spin down.

[boykster]

Nope, no fee in the US at least to RMA a seagate within the warranty period.  You do have to pay return shipping for a tracked carrier and have to have proper packaging. 

I generally go for an advance replacement that costs $19.99 but they ship the drive via 2-day delivery and include a pre-paid return shipping label with the required packaging.