Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Why RAID 5 Stops Working In 2009

kdawson posted more than 5 years ago | from the back-'em-up-rawhide dept.

Data Storage 803

Lally Singh recommends a ZDNet piece predicting the imminent demise of RAID 5, noting that increasing storage and non-decreasing probability of disk failure will collide in a year or so. This reader adds, "Apparently, RAID 6 isn't far behind. I'll keep the ZFS plug short. Go ZFS. There, that was it." "Disk drive capacities double every 18-24 months. We have 1 TB drives now, and in 2009 we'll have 2 TB drives. With a 7-drive RAID 5 disk failure, you'll have 6 remaining 2 TB drives. As the RAID controller is busily reading through those 6 disks to reconstruct the data from the failed drive, it is almost certain it will see an [unrecoverable read error]. So the read fails ... The message 'we can't read this RAID volume' travels up the chain of command until an error message is presented on the screen. 12 TB of your carefully protected — you thought! — data is gone. Oh, you didn't back it up to tape? Bummer!"

cancel ×

803 comments

Carefully protected? (5, Insightful)

Whiney Mac Fanboy (963289) | more than 5 years ago | (#25461453)

12 TB of your carefully protected â" you thought! â" data is gone. Oh, you didn't back it up to tape? Bummer!

If it wasn't backed up to an offsite location, then it wasn't carefully protected.

Re:Carefully protected? (3, Interesting)

rhathar (1247530) | more than 5 years ago | (#25461507)

"Safe" production data should be in a SAN environment anyways. RAID 5 on top of RAID 10 with nightly replays/screenshots and multi-tiered read/writes over an array of disks.

Re:Carefully protected? (5, Insightful)

SatanicPuppy (611928) | more than 5 years ago | (#25461583)

Yea, because we all backup 12TB of home data to an offsite location. Mine is my private evil island, and I've bioengineered flying death monkeys to carry the tapes for me. They make 11 trips a day. I'm hoping for 12 trips with the next generation of monkeys, but they're starting to want coffee breaks.

I'm sorry, but I'm getting seriously tired of people looking down from the pedestal of how it "ought" to be done, how you do it at work, how you would do it if you had 20k to blow on a backup solution, and trying to apply that to the home user. Even the tape comment in the summary is horseshit, because even exceptionally savvy home users are not going to pay for a tape drive and enough tapes to archive serious data, more less handle shipping the backups offsite professionally.

This is serious news. As it stands, the home user that actually sets up a RAID 5 raid is in the top percentile for actually giving a crap about home data. Once that becomes a non-issue, then the point has come when a reasonable backup is out of reach of 99% of private individuals. This, at the same time as more and more people are actually needing a decent solution.

RAID doesn't protect against your worst enemy (0, Redundant)

YesIAmAScript (886271) | more than 5 years ago | (#25461625)

rm -r *

Seriously, you're kidding yourself if you think RAID is protecting you.

Re:RAID doesn't protect against your worst enemy (0, Redundant)

cbreaker (561297) | more than 5 years ago | (#25461807)

Easily solved by file system snapshots.. which you should be doing for an important file server no matter what operating system you're using.

Re:RAID doesn't protect against your worst enemy (5, Insightful)

SatanicPuppy (611928) | more than 5 years ago | (#25461903)

Wow, how incite-ful. Doesn't matter what the discussion is, some geek is bound to weigh in with all the shortcomings of any idea.

Newsflash: there is no perfect backup! No method is foolproof, especially when it's bound to be boring as hell, and you've got an inevitable human factor. You get lazy moving the tapes offsite, you put off fixing a dead drive because there are 4 others, you wipe your main partition upgrading your distro and forget that your CRON rsync script uses the handy --delete flag, and BOOM wipes out your backup.

Shit happens. Pointing out what we all already know doesn't do anything helpful.

Re:RAID doesn't protect against your worst enemy (1)

ShakaUVM (157947) | more than 5 years ago | (#25461941)

>>Seriously, you're kidding yourself if you think RAID is protecting you.

It's a question of what type of faults you care about. Right now, I have a 500GB RAID0 setup to boot and play games off of, an internal 250GB hard drive that mirrors my important folders every night, an external drive that both mirrors my important documents and is used when I shuttle between my two home offices, and an FTP server in a different city that I back up most of my small and important files to (I ain't sending my Snoop Dogg discography over my DSL line). Each has a different failure mode and risk factor. The RAID0 is most vulnerable to hard drive death (though it's been running since '04 without any problems, and my HD health monitors show it in good shape), the second internal drive (which would have been a RAID5 drive, except RAID5 sucks ass on my controller) is vulnerable to theft (steal the RAID0, steal the extra drive too), the USB drive is also vulnerable to theft, though perhaps a different kind of thief (it can be stolen from my car, for example), and the FTP server is slow as shit, but relatively secure.

I'm relatively lazy about doing backups, but having automated stuff to handle all of it makes it not much of a hassle.

Re:RAID doesn't protect against your worst enemy (5, Insightful)

lucas teh geek (714343) | more than 5 years ago | (#25461981)

RAID doesn't protect against your worst enemy
rm -r *

nor is it supposed to. not being a moron seems to have protected me from "my worst enemy" just fine. RAID has protected me from random disk failures. seems to be working as designed

Re:Carefully protected? (3, Informative)

networkBoy (774728) | more than 5 years ago | (#25461679)

you know the other solution is to not use RAID5 with these big drives, or to go to RAID1, or to actually back up the data you want to save to DVD and accept a disk failure will cost you the rest.

Now, while 1TB onto DVDs seems like quite a chore (and I'll admit it's not trivial), some level of data staging can help out immensely, as well as incrementally backing up files, not trying to actually get a full drive snapshot.

Say you backup like this:
my pictures as of 21oct2008
my documents (except pictures and videos) as of 22 oct2008
etc.
while you will still lose data in a disk failure, your loss can be mitigated, especially if you only try to backup what is important. With digital cameras I would argue that home movies and pictures are the two biggest data consumers that people couldn't backup to a single dvd and that they would be genuinely distressed to lose.
-nB

Re:Carefully protected? (4, Insightful)

SatanicPuppy (611928) | more than 5 years ago | (#25461817)

Yea, but DVD is transient crap. How long will those last? A few years? You cannot rely on home-burned optical media for long term storage, and while burning 12 terabytes of information on to one set of 1446 dvds (double layer) may not seem like a big deal, having to do it every three years for the rest of your life is bound to get old.

For any serious storage you need magnetic media, and though we all hate tape, 5 year old tape is about a million times more reliable than a hard drive that hasn't been plugged in in 5 years.

So either you need tape in the sort of quantity that the private user cannot justify, or you're going to have to spring for a hefty RAID and arrange for another one like it as a backup. Offsite if you're lucky, but it's probably just going to be out in your garage/basement/tool shed.

Now, what do you do if you can't rely on RAID? No other storage is as reliable and cheap as the hard drive. ZFS and RAID-Z may solve the problem, but they may not...You can still have failures, and as hard disk sizes increase, the amount of data jeopardized by a single failure increases as well.

Re:Carefully protected? (1)

Ucklak (755284) | more than 5 years ago | (#25462023)

The CD format has been here for 27 years with no signs of leaving soon.
I suspect that the DVD format will be the same.

Your average tape drive for a home user lasts about 3 years before a newer, incompatible version comes along to replace it.
I have never seen a commercial tape drive last longer than 7 years.
YMMV

Re:Carefully protected? (2, Informative)

Fulcrum of Evil (560260) | more than 5 years ago | (#25462073)

Read the post again - he said that home burned DVDs are good for 3 years, tops. This is called media life.

Re:Carefully protected? (2, Insightful)

MBCook (132727) | more than 5 years ago | (#25462047)

Good points. While magnetic media is problematic, SSDs are going to become a very viable option for the home backup (compared to stacks of DVDs or the possible reliability of old magnetic HDs).

Re:Carefully protected? (1)

Bandman (86149) | more than 5 years ago | (#25461883)

This weakness is really only an issue once you start getting towards an immense amount of data. As the blog entry states, one failure is statistically likely during 14TB of data recover (not storage volume, data volume). So 7 1TB drives in a full RAID5 array has a 50% chance of failure during recovery if one of the drives were to fail.

Home users won't be affected, at least at this point. I imagine that by the time it does become an issue, manufacturing tech will have caught up and it'll be 14PB.

I did cover this [blogspot.com] earlier today, too.

Re:Carefully protected? (0)

Anonymous Coward | more than 5 years ago | (#25462049)

Why are the only options for the home user RAID or DVD?

I back up critical documents at home to an external hard drive once a week, and the whole system (minus documents, which are stored on a separate partition) about once a month. The rest of the time, the external hard drive is off, saving wear and tear on the drive.

If a document is really, really important, then I'll also put it on a DVD, on a secure server, or on the flash drive that I keep in my lockbox with my important documents.

I'll admit that this probably works best for me because I only work on documents on my home computer a couple of days a week (and the rest of the time I mostly use it for web browsing), but I think it's worth considering, and takes a lot less time than trying to burn large documents to DVD.

Re:Carefully protected? (1)

Linux_ho (205887) | more than 5 years ago | (#25461717)

I think I saw a 1TB external USB drive for $180 the other day. Off-site doesn't need to be difficult or expensive, and it's worth the effort if you care about your data.

Re:Carefully protected? (4, Insightful)

Whiney Mac Fanboy (963289) | more than 5 years ago | (#25461773)

Oh come on. Do you have 12TB of home data? Seriously? And if you do, it's not that hard to have another another 12TB of external USB drives at some relatives place.

I've got about 500GB of data that I care about at home & the whole lot's backed up onto a terrabyte external HDD at my Dad's. It's not that hard.

If you think raid is protecting your data, you're crazy.

Re:Carefully protected? (1)

HTH NE1 (675604) | more than 5 years ago | (#25461841)

Oh come on. Do you have 12TB of home data? Seriously? And if you do, it's not that hard to have another another 12TB of external USB drives at some relatives place.

Only if you've established the habit of buying your drives in pairs while you were amassing your data.

Sometimes having an expensive computer system doesn't mean you have money; it means you had money.

Re:Carefully protected? (1)

Free the Cowards (1280296) | more than 5 years ago | (#25462037)

Buying a computer system you cannot afford to properly use is crazy. Yes, some people are crazy, and those crazy people are going to lose data, but there's no sense in defending it.

Re:Carefully protected? (4, Funny)

DrVxD (184537) | more than 5 years ago | (#25461967)

Oh come on. Do you have 12TB of home data? Seriously? And if you do, it's not that hard to have another another 12TB of external USB drives at some relatives place.

Not all of us have relatives, you insensitive...[URE]

Re:Carefully protected? (1)

SatanicPuppy (611928) | more than 5 years ago | (#25462031)

I actually have an RRRAID...A redundant redundant redundant array of inexpensive disks. I may lose 1 raid. I may even lose 2. But I probably won't lose 3. But that solution is WAY out of reach for the average consumer, and is only possible for me because the amount of data I have on hand doesn't change very quickly.

Even 1TB is a problem, and that is within the reach of consumers these days. And if you think your external HDD is protecting your data, you're crazy. The failure on those is single point, and thats more likely on an external drive that gets moved around than on any internal drive. Beyond that, I'm sure your rotational policy is lax; everyone's is, so what you're really saying is you have some of your data backed up. Depending on how often you back up, you may only lose a month or two.

Re:Carefully protected? (1)

cbreaker (561297) | more than 5 years ago | (#25461785)

I'm with you on this, but the problem with many RAID5 systems is that you usually purchase all of the drives at once, so it increases the likelyhood of a double-disk failure since all the drives are the same age.

Mind you, I've worked in IT for over 15 years and I've never had the fortune to experience a double disk failure.

On my own system at home, I have three RAID5 sets on two servers with hardware raid SATA controllers (Accusys controllers - they're really nice!) If I were to experience a disk failure, I'll turn off the server and go get a replacement disk before turning it back on.

I also replicate all of the important data to a friend; we both have file servers, we both run Server 2003 R2 (Well, I run 2008 now) with DFS Replication, and we use an OpenVPN tunnel between us. This way, even if we had a bad disk failure we'll be okay.

In reality, you should have backups of critical data because I'm much more afraid of a fire or flood than a double-disk failure.

Re:Carefully protected? (1)

MadMorf (118601) | more than 5 years ago | (#25462087)

Mind you, I've worked in IT for over 15 years and I've never had the fortune to experience a double disk failure.

Lucky you.

I work as a TSE for one of the major storage companies. I see double disk failures at least weekly, usually more often.

This is where the difference between a home-brewed solution and a commercial solution comes in.

Two and a half years of working this kind of case at least weekly and I've never lost any data, because we have the tools to deal with these issues.

So, ideally, mirrored raid for availability. Snapshots to protect against deletion and off-site replication for DR.

No tapes needed.

Re:Carefully protected? (1)

Urza9814 (883915) | more than 5 years ago | (#25461793)

Um, they said RAID5 is dead. What about, say, RAID 1? That's still good for backing up. Or do as I do and have your OS run a nightly backup on to a different hard drive. I have 2 500GB drives. One I use, and one is so that Mandriva can record the changes every night.

Re:Carefully protected? (1)

blahplusplus (757119) | more than 5 years ago | (#25461839)

"This is serious news. As it stands, the home user that actually sets up a RAID 5 raid is in the top percentile for actually giving a crap about home data. Once that becomes a non-issue, then the point has come when a reasonable backup is out of reach of 99% of private individuals"

This is why they made DVD, Blu-ray, USB thumb drives, Flash memory (Flash memory is getting mighty big now 16-32GB), with USB thumb drives, 16-32GB SDHC flash memory cards, then their are portable hard drives that connect via USB/Firewire and personal SAN solutions starting to appear. But it's highly likely within the next 5 years flash technology will make a lot of talk about backup moot, since online can take care of storing pictures and videos (the biggest portion usually) and then that leaves the user to just backup his higher quality pics and videos (should he/she be savvy enough to begin with). I wouldn't be surprised most people use shared picture sites like Google, Flickr, photbucket, and youtube to store their videos/pictures and not worry about having to back them up. Much of what people have is disposable or highly redundant and easy to redownload.

Most important data people want to backup is not a lot of data, unless you're talking about pictures and video and this could be done with a lot less if they were savvy about it. But... the truth of the matter is things will get better as the generations go by, as more and more people grow up with technology there will be less and less of a learning curve over time.

Re:Carefully protected? (1)

Ephemeriis (315124) | more than 5 years ago | (#25461951)

I'm sorry, but I'm getting seriously tired of people looking down from the pedestal of how it "ought" to be done, how you do it at work, how you would do it if you had 20k to blow on a backup solution, and trying to apply that to the home user. Even the tape comment in the summary is horseshit, because even exceptionally savvy home users are not going to pay for a tape drive and enough tapes to archive serious data, more less handle shipping the backups offsite professionally.

This is serious news. As it stands, the home user that actually sets up a RAID 5 raid is in the top percentile for actually giving a crap about home data. Once that becomes a non-issue, then the point has come when a reasonable backup is out of reach of 99% of private individuals. This, at the same time as more and more people are actually needing a decent solution.

First off, I have a hard time seeing a home user coming up with 12 TB of data anytime soon. Sure, it's possible... But I find it unlikely. I also find it unlikely that your average home user is going to be savvy enough to even know what a RAID-5 is. We've got plenty of home users who've been keeping all their precious family photos on a single SD card with absolutely no backup at all.

And if your home user does actually have 12 TB of data, in a RAID-5, that's not going to be a cheap pile of hardware. If they've got that kind of money to spend on storage, why wouldn't they spend some money to actually protect that data?

Next up, a RAID (1, 5, 10, various combinations there-of) does not protect you from the single biggest threat to your data - user error. A RAID will not make it any easier to recover data that you've accidentally deleted.

Accidentally delete a pile of Timmy's graduation photos? Tough. Even if you've got a working RAID-5 you aren't going to have an easy time getting them back.

How 'bout your house burns down with your RAID-5 in it? Your data is still toast, even if it was on a RAID.

Anyone who cares about their data - home user or business - needs an offsite copy of it. Doesn't need to be a tape... You could dump it to a NAS and then unplug the thing and stick it in a safe deposit box. Or you can print everything out and mail it to your uncle. Or you can burn a pile of DVDs and hide them throughout the woods. But unless you've got your data offsite it is not protected.

As far as getting a backup working... These days it's pretty damn trivial.

If you don't have a ton of data you can just burn it to DVDs easily enough.

If you've got more data you could get yourself an external HDD, or a few USB flash drives, or a cheap NAS and dump the data to it.

You can also get an LTO1 drive for about $1,000 if you really wanted to go with a tape.

Re:Carefully protected? (5, Funny)

sholsinger (1131365) | more than 5 years ago | (#25461953)

Next they'll want to unionize. At that point you've lost everything.

Re:Carefully protected? (1)

Vu1turEMaN (1270774) | more than 5 years ago | (#25462043)

What would you recommend then for a 30user non-profit business with a 136gb scsi drive and tape backups they have been running (but not realizing that they weren't completely backing up everything)? Thats the crap I just inherited...I was thinking moving to 2 sata drives in raid 1(the scsi drive sounds like it's gonna die soon) and fixing the bloated folders so that tape backups will work again. Differential appended tape backups M-R and a normal backup on friday after the workday seems like the best idea.

Re:Carefully protected? (1)

Carrion Creeper (673888) | more than 5 years ago | (#25462061)

My semi-cheap fire/water safe with a disconnected backup drive inside beats a RAID array any day.

The fire safe is not that complicated, and I can put in and take out mismatched drives any time without rebuilding.

Also the fire safe is much harder to carry away, not prone to power surges, and much less likely to catch on fire.

Re:Carefully protected? (0)

Anonymous Coward | more than 5 years ago | (#25462063)

I had a 6TB RAID 5 array, 3 drives failed (not 2, 3) and I lost it all.

I now have a RAID 6 array with a hot spare. Scripts back up the *important* 100GB or so. The rest can always be downloaded again (so in a way I do have an offsite backup)

Tapes? Bluray is cheaper...

Re:Carefully protected? (2, Interesting)

networkBoy (774728) | more than 5 years ago | (#25461589)

True.
Also FWIW I only run RAID 1 and JBOD.
For things that must be on-line, or are destined for JBOD but not yet archived to backup media, they are located on one of the RAID volumes. For everything else it's off to JBOD, where things are better than RAID5

Why?

I have 6 TB of JBOD storage and 600(2x300 volumes) GB of RAID 1. If I striped the JBOD into 6TB (7 drives) and one drive failed all the near-line data would be virtually off-line (and certainly read-only) while the array re-built. With JBOD, should a disk fail, I pop in a replacement, grab the stack of DVDs from the local backup, and plug the data back in. Now all the other near-line is still available and honestly takes about the same amount of effort and time as re-building a stripe set w/ parity. Never mind that I've had a read error on rebuilds before and had to re-do the entire array from scratch anyway.

While my system would not work in an environment where the files on the JBOD change often, they are basically .archive anyway, so handling them by way of staging on RAID1 pending copy to DVD and storing on JBOD works fine.

Naturally this system also really gives an incentive to keep up on the backups, with no false sense of security of having files on a RAID5...
-nB

Re:Carefully protected? (1, Interesting)

Anonymous Coward | more than 5 years ago | (#25461745)

If I striped the JBOD into 6TB (7 drives) and one drive failed all the near-line data would be virtually off-line (and certainly read-only) while the array re-built.

What kind of crappy raid array is that? Better raid arrays will model & predict performance under degraded conditions like failure & rebuilding. They certainly don't stop or go read-only during a rebuild.

When tour groups of non-tech people come by the server room, I used to emphasize reliability by pulling a hard disk out of a running server, hand it to them, and then put it back in the server. The server doesn't skip a beat (and these were common off-the-shelf Dell rackmount servers costing $2,500 or so).

Aside from automated alarms paging some IT people, no one would notice.

Re:Carefully protected? (1)

Bandman (86149) | more than 5 years ago | (#25462013)

Nice shock value, but I'm not sure I'd trust to fate to rebuild the array every time. The risk/reward ratio isn't in balance to me.

RAID5 isn't a false sense of security (1)

cbreaker (561297) | more than 5 years ago | (#25461835)

RAID 5 isn't a false sense of security. It actually DOES protect you from a disk failure.

I made the decision about two years ago that all disks at home will be either mirrored or RAID5. Disks are so dirt cheap that there's no reason not to.

RAID doesn't prevent you from having to have some sort of backup solution, and if you can't trust yourself to do them unless you're being risky with your data, I'll happily avoid dealing with restoring data and all that bullshit from a single disk failure and you can sink your time into doing it all manually.

Re:Carefully protected? (0)

Anonymous Coward | more than 5 years ago | (#25462051)

This article is stupid. RAID was never intended as permanent storage of digital assets since its inception in 1987. Also, drive capacities don't matter when read errors occur, which is a natural affect of a media error (that in most cases can be recovered). Sure individual drive capacities do increase over time however, spinning disk (mechanical) will always be prone to failure. A more intelligible article might point out that the storage industry is moving more to SSDs (Solid State Disks) by next year. There's no moving parts whatsoever thus MTBF will be decreased big time and will keep RAID volumes up longer as a direct result of non-mechanical failure which by the way is the number 1 issue affecting spinning disks today which SSDs mitigate.

Illin with the panicillin? (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#25461465)

Is she illin with the panicillin?
Is she reelin in the panicillin?
Is it feelin with the panicillin?
Are you steelin in the panacillin?

Panka Panka

Is she liable no suitifiable no not on trial but so suitifiable
Is she viable no suitifiable pliable style is so suitifiable
so reliable no suitifiable shes not on file but so suitifiable
im on the dial its so suitifiable its like im liable but more suitifiable

Re:Illin with the panicillin? (0)

Anonymous Coward | more than 5 years ago | (#25461849)

can someone tell what's the meaning of this shit?

Re:Illin with the panicillin? (0)

Anonymous Coward | more than 5 years ago | (#25461899)

Probably someone attempting the 1000 monkeys on a thousand keyboards theory? They have made quite some progress don't you think?

Dont worry too much (1)

Gat0r30y (957941) | more than 5 years ago | (#25461471)

When HDD's move to bigger sectors - there should be better error recovery reducing the probability of unrecoverable read errors. Right? Ok, I'm moving to ZFS.

Re:Dont worry too much (1, Insightful)

tepples (727027) | more than 5 years ago | (#25461651)

When HDD's move to bigger sectors - there should be better error recovery reducing the probability of unrecoverable read errors. Right?

Not if what fails is the drive motor.

Re:Dont worry too much (5, Informative)

SatanicPuppy (611928) | more than 5 years ago | (#25461693)

The real issue is one that anyone who has ever had to recover a multi-drive array can tell you instantly: if one drive fails, and the other drive was bought at the same time, and has had a nearly identical usage pattern, the odds of the other drive failing are well above average.

I once had a single drive fail in a 24 disk array. The disks were arranged, RAID 5, in groups of 3, glued together by Veritas (from back before it got bought by crappy symantec). By the time the smoke cleared we had replaced 19 out of 24 drives. They had all been bought at the same time, and as they thrashed rebuilding their failed buddies, they started dying themselves. The remaining 5 drives we replaced anyway, just because.

That's a worst case, but multiple failures are far from uncommon, and very few people correctly cycle in new drives periodically to reduce the chance of a mass failure.

Re:Dont worry too much (5, Insightful)

Angus McNitt (542101) | more than 5 years ago | (#25461893)

... very few people correctly cycle in new drives periodically to reduce the chance of a mass failure.

That is also because very few people buy a Raid setup piecemeal. Most end up buying a solution, fully populated. The idea of swapping out some drives as you go, or growing your RAID over time doesn't always look good, either to the PHBs who usually run the budget, or to the vendor. We had a vendor trying to sell us a iSCSI SAN device tell us that varying the drive lots and dates increased the chances of failure. Needless to say we went elsewhere.

When we bought the RAID array for our Exchange box, this is going back a few years, everybody looked at my like an idiot because I asked for drives with different lot numbers. It was the best I could do as buying over time was not an option. HP was actually pretty cool about this request and out of 8 disks, no 3 have the same lot number or manufacture date.

Of course we are also running RAID on that machine for non-backup and do a nightly replication, so your mileage may vary.

Re:Dont worry too much (0)

Anonymous Coward | more than 5 years ago | (#25461933)

Indeed, but did you lose any data?

RAID 5 is invaluable to protecting from disk failures. You're absolutely right: There's a REAL risk of double-disk failures with RAID sets because they are usually purchased together, were manufactured in the same group, and follow the same usage patterns.

You can't really be secure in ANY RAID configuration. Even if you had two mirrored mirror sets, you could lose your data. The only way to be sure you don't is to have good backups.

Good backups is a problem, but for Joe Home User you can get 1TB external disks for so cheap it's scary, and many times these disks come with "push to backup" buttons you can use to instantly back up all your shit.

For me, I have too much to back up so I replicate all my important stuff (about 2TB) to a friend. He does the same. We use DFS Replication. It works like a frigging champ (it's my favorite Microsoft thing ever!) and it's the only way for the both of us to ensure that if the building burns, we're protected.

Multiple disk failures aren't uncommon, however I've been responsible for... an uncountable number of RAID sets (including many housed on Symms, Clariions, DAS, and other types of disk systems) and I've personally never experienced a double disk failure on a RAID 5. Maybe I'm just lucky..

Re:Dont worry too much (1)

LarsG (31008) | more than 5 years ago | (#25462019)

and the other drive was bought at the same time

Sequential serial numbers too, to be *really* safe, right? ;-p

Re:Dont worry too much (1)

Bandman (86149) | more than 5 years ago | (#25462025)

wow, I don't envy that period of time in your life, to be sure.

Re:Dont worry too much (1)

afidel (530433) | more than 5 years ago | (#25461923)

Actually, yes. Or at least that's one of the way my SAN vendor is dealing with increasing drive size. They are the first vendor to enable ANSI T10-DIF end to end checksumming which includes additional bits per block (AFAIR it's the same as mainframe drives have been using all along). They have also made the recoverable element the surface rather than the drive so recovery times are several times faster. Check them [xiotech.com] out.

Backup (2, Informative)

Anonymous Coward | more than 5 years ago | (#25461473)

RAID is not, and has never been, a substitute for backups.

At least..... (1)

stun (782073) | more than 5 years ago | (#25461493)

I am not running Windows.......oh wait nvm

RAID != Backup (3, Insightful)

vlad_petric (94134) | more than 5 years ago | (#25461521)

I mean, WTF? Many people regard RAID as something magical that will keep their data no matter what happens. Well ... it's not.

Furthermore, for many enterprise applications disk size is not the main concern, but rather I/O throughput and reliability. Few need 7 disks of 2 TB in RAID5.

Re:RAID != Backup (4, Insightful)

Anonymous Coward | more than 5 years ago | (#25461643)

Furthermore, for many enterprise applications disk size is not the main concern, but rather I/O throughput and reliability. Few need 7 disks of 2 TB in RAID5.

Some of us do need a large amount of reasonably priced storage with fast read speed & slower write speed. This pattern of data access is extremely common for all sorts of applications.

And this raid 5 "problem" is simply the fact that modern sata disks have a certain error rate. But as the amount of data becomes huge, it becomes very likely that errors will occur when rebuilding a failed disk. But errors can also occur during normal operation!

The problem is that sata disks have gotten a lot bigger without the error rate dropping.

So you have a few choices:

- use more reliable disks (like scsi/sas) which reduce the error rate even further
- use a raid geometry that is more tolerant of errors (like raid 6)
- use a file system that is more tolerant of errors
- replicate & backup your data

Re:RAID != Backup (3, Insightful)

MBCook (132727) | more than 5 years ago | (#25461739)

I've always understood it as RAID exists to keep you running either during the 'outage' (i.e. until a new disk is built) or at least long enough to shut things down safely and coherently (as opposed to computer just locking up or some such).

It's designed to give you redundancy until you fix the problem. It's designed to let you limp along. It's not designed to be a backup solution.

As others have mentioned: if you want a backup set of hard drives, you run RAID 10 or 15 or something where you have two(+) full copies of your data. And even that won't work in many situations (i.e. computer suddenly finds it's self in a flood).

All that said, the guy has a possible point. How long would it take to build a new 1TB drive into an array? That could be problematic.

There is a reason SANs and other such things have 2+ hot spares in them.

Re:RAID != Backup (1)

Rayeth (1335201) | more than 5 years ago | (#25461765)

I never understand why this idea persists. RAID is more useful to me to increase write speeds when moving large files across multiple drives, than it is for having fault tolerance.

Re:RAID != Backup (1)

cbreaker (561297) | more than 5 years ago | (#25461991)

Wait - WHO SAID THAT?

I don't think I've ever met anyone that thought RAID was a replacement for backups. Have you? Wait don't answer that, I don't need a made-up story.

And I beg to differ on the "many enterprises concern most with disk speed" - no. Even small companies now have large data needs, and the very first thing to consider on any storage solution is usable disk space - because if there's not enough space then it doesn't work does it?

Performance is a close second, and reliability is simply taken for granted. You're always going to use a RAID set. It just depends on how much performance you need and how much you can spend.

Storage capacity is always #1 on the list.

RAID is fine, stupid admins are not! (0)

Anonymous Coward | more than 5 years ago | (#25461541)

RAID 5 and "carefully protected"?

RAID 5, as well as RAID 6 is nothing more at an attempt to add some amount of redundancy without sacrificing too much space. Go RAID 1 instead with the same number of disks. Also do off-site mirroring of all your data.

And if you get get "unrecoverable read error" after a drive failure, it means the administrator should get fired, as he was too stupid to type "echo check > sync_action" followed later by "cat mismatch_cnt".

Re:RAID is fine, stupid admins are not! (3, Insightful)

DrVxD (184537) | more than 5 years ago | (#25461729)

RAID 5, as well as RAID 6 is nothing more at an attempt to add some amount of redundancy without sacrificing too much space. Go RAID 1 instead with the same number of disks.

As far as I'm concerned, RAID 5 really has no redeeming features (it's slow, not particularly safe, but lulls people into a false sense of security).

From a data integrity perspective, though, RAID6 is a better solution than RAID1.

Given arrays of equal sizes, with RAID6 your data can survive the loss of *any* two disks; with RAID1, if you lose two disks which happen to be a mirrored pair, then you're hosed.

But, as you point out, RAIDn doesn't really qualify as "carefully protected"

What's your beef with RAID 5? (3, Insightful)

cbreaker (561297) | more than 5 years ago | (#25462057)

Seriously - what's the problem with RAID 5? It's not a FALSE sense of security: It actually DOES prevent data loss or down time on a single disk failure. If you're a moron, you're creating 14 disk arrays. If you're smart, you keep it to 7 disks at the very most.

RAID 5 is great. It's fast, unless you have a shit controller without enough cache. It's going to prevent down time on a single disk failure (which is overwhelmingly the most common type of failure) and it doesn't cost you too much capacity.

Usually I'm more concerned with a fire or flood than a double-disk failure.

RAID 6 is good, but you get the same (actually worse) performance hit over RAID 5. More parity calculations. You can lose any two disks, which is nice, and if you can spare the space, go for it!

I don't see RAID 6 as being all that much more of a big deal over RAID 5 and actually it shouldn't really have it's own number since it's exactly the same technology and parity system as 5. It should be RAID 5.1 or something. Or maybe RAID5+1. The only reason it's become more available now is because controllers have gotten fast enough to deal with the additional parity.

What. (3, Insightful)

DanWS6 (1248650) | more than 5 years ago | (#25461545)

The problem with Raid 5 is that the more drives you have the higher probability you have that more than one drive dies. That's why you have multiple raid 5 arrays of 4 disks maximum instead of one array of 7 disks.

Re:What. (0)

Anonymous Coward | more than 5 years ago | (#25461691)

The problem with Raid 5 is that the more drives you have the higher probability you have that more than one drive dies. That's why you have multiple raid 5 arrays of 4 disks maximum instead of one array of 7 disks.

But what am I going to do with my 7000 2GB disks then?

Oh look, noobs. (1, Insightful)

Anonymous Coward | more than 5 years ago | (#25461553)

If you use RAID to 'protect' your data, you clearly don't value your data at all.

While the interesting bit of this article is the coming demise of RAID 5, what you should be bringing away with it is, if RAID is all that stands between you and data loss, you're a noob.

RAID is about avoiding PRODUCTION downtime. (2, Informative)

khasim (1285) | more than 5 years ago | (#25461809)

Spell it out for everyone.

RAID won't save your data if there is a fire.
Or if you delete a file.
Or if two drives fail.
Or a thousand other scenarios.

All RAID does is prevent the system from going down when a single drive fails (except RAID 0). Thus giving everyone in the office time to finish up their important work and log out for the day so you can swap the drive. Or, if you're brave, swap the drive during regular work hours.

For the home user (not working on huge graphic files) RAID 1 (mirroring) should be sufficient. As long as it is paired with another EXTERNAL hard drive that you copy your important information to. And leave with your brother or something. I'm talking family photos and such. Your tax information should be small enough to fit on a USB drive.

If your computer completely failed TODAY what would be the really irreplaceable files on it?

Back those up. Then store them with a friend or someone in your family.

There, problem solved.

Re:Oh look, noobs. (1)

cbreaker (561297) | more than 5 years ago | (#25462083)

At home, I use RAID to protect my non-replicated data between backups.

At work, I use RAID to protect my data between backups and to help prevent down time due to a disk failure.

RAID is an excellent tool to protect yourself. There's multiple levels of protection and RAID is just one of them.

Redundant Array of Irrelevant Data (1, Interesting)

mschuyler (197441) | more than 5 years ago | (#25461561)

That's what RAID stands for. It's a nice idea in theory, as long as the disks remain cheap, but I've never trusted them to work properly and had more than one break on me. "All you have to do is unplug the bad disk, plug in a good one in its place, and in a few minutes all will be hunky dory." Bzzt. Wrong. Thanks for playing.

Backup every day to tape, to another disk entirely on a diffrent machine, to R/W DVD, twice a day if you have to, or all of the above--anywhere else but the machine itself. RAID: the accident waiting to happen. Yeah, I'm paranoid. It comes from experience.

Just double-up on everythign (3, Informative)

realmolo (574068) | more than 5 years ago | (#25461573)

If you have one RAID5 box, just build another one that replicates it. Use that for your "hot backup". Then back that up to tape, if you must.

Storage is so cheap these days (especially if you don't need super-fast speeds and can use regular SATA drives), that you might as well just go crazy with mirroring/replicating all your drives all over the place for fault-tolerance and disaster-recovery.

Re:Just double-up on everythign (1)

Anonymous Coward | more than 5 years ago | (#25461869)

Consumer storage is cheap. Enterprise still costs around $10,000/TB in fast SAS storage that is fully featured.

It sucks.

You're missing the point. (2, Informative)

Polarina (1389203) | more than 5 years ago | (#25461585)

A RAID 5 setup is only a precaution in case of an hardware failure. It serves as no excuse for not having backed up your data.
And the topic is also flawed - RAID 5 doesn't have any self destruct mechanism.

7 2TB Disks in RAID 5???????? (1)

quantumplacet (1195335) | more than 5 years ago | (#25461595)

This story is just ridiculous. It clearly states that this doesn't affect Enterprise users, as their URE rate is lower and unless they're idiots they use smaller drives. What home users will have 7 disk RAID 5 arrays of 2TB disks? Is this really a large enough percentage of RAID5 users to call for the death of it?

Re:7 2TB Disks in RAID 5???????? (2, Informative)

cong06 (1000177) | more than 5 years ago | (#25461769)

The main point of the article is to point out a problem that is going to eventually occur. If you read the article he mentions that later on with large enough hard drives, everyone will require a RAID set up with their "Dell manufactured" Computer. (assuming Dell hands out >>2-4TB disks to their average user)

I fail to see the great insight (1)

wintermute000 (928348) | more than 5 years ago | (#25461597)

If you put 7 disks in a single RAID5 without backup then its called bad design and bad implementation.

This has always been true regardless of disk size/speed.

As above posters have pointed out once you get past 4 disks the non-ZFS way to go is multiple blocks of RAID-(whatever number is appropriate for your scenario).

Though ZFS is awesome and if your OS/hardware supports it 100% there is little reason to stick with RAID

!news (0)

Anonymous Coward | more than 5 years ago | (#25461605)

July 18th, 2007

Testable assertion (3, Interesting)

merreborn (853723) | more than 5 years ago | (#25461613)

But even today a 7 drive RAID 5 with 1 TB disks has a 50% chance of a rebuild failure. RAID 5 is reaching the end of its useful life.

This is trivially testable. Any slashdotters have experience rebuilding 7TB RAID 5 arrays?

You'd think, if this were really an issue, we'd be hearing stories from the front lines of this happening with increasing frequency. Instead we have a blog post based entirely on theory, without a single real-world example for corroboration.

What's more, who even uses RAID 5 anymore? I thought it was all RAID 10 and whatnot these days.

Re:Testable assertion (0)

Anonymous Coward | more than 5 years ago | (#25461649)

My RAID goes up to 11.

Re:Testable assertion (1)

mbone (558574) | more than 5 years ago | (#25461721)

Yeah, this sounds like FUD to me, although I have no data one way or the other. I know lots of people with lots of video, and I haven't heard any screams about this, so I am inclined to doubt.

Re:Testable assertion (4, Informative)

theendlessnow (516149) | more than 5 years ago | (#25461837)

I have large RAID 5's and RAID 6's... I generally don't have any RAID columns over 8TB. I HAVE had drive failures. Yes... I'm talking cheapo SATA drives. No... I have not see the problem this article presents. Do I backup critical data? Yes. The only time I lost a column was due to a firmware bug which caused a rebuild to fail. Took awhile to restore from backup, but that was about the extent of the damage. I would call this article FUD... deceptive FUD, but very much FUD.

Re:Testable assertion (1)

MrPerfekt (414248) | more than 5 years ago | (#25461847)

What's more, who even uses RAID 5 anymore? I thought it was all RAID 10 and whatnot these days.

Just because the number is higher doesn't mean it's 'better'. RAID levels are to be chosen based on what (performance, size, redundancy) is specifically required. One size most definitely does not fit all. RAID 10 is really just RAID 1+0 anyway, which is to say a stripe of mirrors.

For many 'cold storage' or online-archiving cases, RAID 5 is desired because it offers the greatest redundancy and storage combination with not bad performance. I have many RAID 5 arrays at home and work. They typically consist of drives under 500GB each though and consist of 6 drives or under. The (old) article brings up valid points though and it is something that should be considered.

ZFS does offer some more protections like byte-level parity checks but at the heart of it still consists of striping. Sun does recommend good practices on using ZFS on an X4500 (Thumper) like keeping raid groups (which are then concatenated into a pool) smaller than 9 drives and using double-parity like in RAID 6.

The big hurdle here is that until ZFS makes it into the stock Linux kernel tree, adoption will be limited. (Full) Mac OS X support will definitely help on that front.

Raid decay (0)

Anonymous Coward | more than 5 years ago | (#25461631)

Sure, everyone should use atleast Raid 6 in production, atleast it's an improvement over the classic Raid 5 with a hot-spare.

But the big problem with Raid isn't disk failure, it's disk decay, and a major reason for that being a problem is the lack of hashing on most modern filesystems.

They basically don't check that what you put somewhere is what you get back, which means that the Raid can decay slowly and your data will just corrupt, sure it's still raid, it's just that the distributed data is corrupt.

Yay, stories from July 2007! (1)

MrPerfekt (414248) | more than 5 years ago | (#25461647)

I'm going to troll ridiculously old articles and post them to Slashdot and hope the editors don't notice... oh cool, they didn't here either!

Sounds.. well. Stupid (4, Insightful)

EdIII (1114411) | more than 5 years ago | (#25461661)

I can see a lot of people getting into a tizzy over this. The RAID 5 this guy is talking about is controlled by one STUPID controller.

There are a lot of methods, and patented technology that prevent just the situation he is talking about. Here is just one example:

PerfectRAID(TM) is Promise's patented RAID data protection technology; a suite of data protection and redundancy features built into every Promise RAID product.

        *
            Predictive Data Migration (PDM): Replace un-healthy disk member in array and keep array on normal status during the data transition between healthy HD and replaced HD.
        *
            Bad Sector Mapping and Media Patrol: These features scan the system's drive media to ensure that even bad physical drives do not impact data availability
        *
            Array Error Recovery: Data recovery from bad sector or failed HD for redundant RAID
        *
            RAID 5/6 inconsistent data Prevent (Write Hole Table)
        *
            Data content Error Prevent (Read/Write Check Table)
        *
            Physical Drive Error Recovery
        *
            SMART support
        *
            Hard/Soft Reset to recover HD from bad status.
        *
            HD Powercontrol to recover HD from hung status.
        * NVRAM event logging

RAID is not perfect, not by any stretch, but if you use it properly it will serve it's purpose quite nicely. If your data is that critical, having it on a single raid is ill advised anyways. If you are talking about databases, then RAID 10 is more preferable and replicating the databases across multiple sites, even more so.

Smells Like FUD. (4, Insightful)

sexconker (1179573) | more than 5 years ago | (#25461697)

What is this article about?

They say that since there is more data, you're more likely to encounter problems during a rebuild.

The issue isn't with RAID, it's with the file system. Use larger blocks/sectors.

Losing all of your data requires you to have a shitty RAID controller. A decent one will reconstruct what it can.

The odds of you encountering a physical issue increases as capacity increases, and decreases as reliability increases. In theory, the 1 TB and up drives are pretty reliable. Anything worth protecting should be on server-grade hard drives anyway.

The likelihood of a physical problem popping up during your rebuild is no higher with new drives than it was with old drives. I haven't noticed my larger drives failing at higher rates than my older, smaller drives. I haven't heard of them failing at higher rates.

Remember, folks, RAID is a redundant array of inexpensive disks. The purpose of RAID is to be fault-tolerant, in the sense that a few failures don't put you out of production. You also get the nice bonus of being able to lump a bunch of drives together to get a larger total capacity.

RAID is not a backup solution.

RAID 5 and RAID 6, specifically, are still viable solutions for most setups. If you want more reliability, go with RAID 1+0, RAID 5+0, whatever.

Choosing the right RAID level has always depended on your needs, setup, budget, and priorities.

Smells like FUD.

Taking published stats too seriously? (2, Interesting)

Vellmont (569020) | more than 5 years ago | (#25461705)

The whole argument boils down the published URE rate being both accurate, and a foregone conclusion. Will disk makers _really_ make drives that have a sector failure for every 2 terabytes, or will they improve whatever technology is causing these URE's to be much more rare? (if the rate was real in the first place).

RAID Is not a Backup !!!!! (4, Insightful)

mbone (558574) | more than 5 years ago | (#25461707)

How many times does this have to be said.

RAID is not a backup. RAID is designed to protect against hardware failures. It can also increase your I/O speed, which is more important in some cases. Backups are different.

Depending on what you are doing, you may or may need a RAID, but you definitely need backups.

Re:RAID Is not a Backup !!!!! (0)

blair1q (305137) | more than 5 years ago | (#25461867)

The chances that your RAID will have a double failure causing your data to be lost are just about the same as the chances that your RAID will have a single failure and your tape backup also has a failure.

You can back up your RAID, but that'd be like backing up your backup tapes.

Feel free. Tape-hangers need jobs.

Re:RAID Is not a Backup !!!!! (1)

bendodge (998616) | more than 5 years ago | (#25462089)

Read the article. Once you get into the 2TB and higher range, RAID5 won't protect much against hardware failures. As a previous poster noted, expecting even savvy home users or SMBs to keep offsite tape backups of a 7-disk array is expensive and unrealistic.

Can I tell you where to insert your plug? (0, Flamebait)

hacker (14635) | more than 5 years ago | (#25461747)

"I'll keep the ZFS plug short."

Start with a short plug, because where I'll be asking you to put your ZFS "plug" will most-definitely hurt if your "plug" is any larger...

  1. Lacking in file system utilities (yes, fsck IS necessary even on healthy filesystems, especially on desktops and portables)
  2. License-incompatible with anything worth running it on, other than Solaris itself... which is NOT worth running (see #1 above)
  3. Proprietary and full of patented technologies (see #2 above)

Need I go on? There are plenty more reasons.

We'll have a viable replacement soon enough, which is already designed to have quite a few more features that ZFS does not have, and cannot delivery in its current incarnation.

Re:Can I tell you where to insert your plug? (0, Troll)

jumon (1390957) | more than 5 years ago | (#25461995)

Lacking in file system utilities (yes, fsck IS necessary even on healthy filesystems, especially on desktops and portables)

So, why is fsck needed? If you'd actually investigated ZFS, you'd know its built to never need such an archaic utility. Either it works or it doesn't, so if it does, then no external checks are needed.

so wrong (0)

Anonymous Coward | more than 5 years ago | (#25461749)

12 TB of your carefully protected â" you thought! â" data is gone. Oh, you didn't back it up to tape? Bummer!"

Ummm... Wow.

A RAID is no substitute for a backup. A RAID cannot recover a file that you accidentally deleted, for example. A RAID can't be used to rebuild your server if the building burns down. If you aren't using some kind of offsite backup, your data is not carefully protected.

RAIDs are handy for giving you a little more reliability. If one HDD fails you can usually recover without any downtime.

RAIDs also give you much better speed over a single drive.

RAIDs can give you increased capacity over a single drive.

But a RAID is not a substitute for a backup. Never was, never will be.

The problem is time, not reliability (2, Interesting)

petes_PoV (912422) | more than 5 years ago | (#25461777)

The larger the drives, the longer it takes to resilver (rebuild the RAID) the array. During this time performance takes a real hit - no matter what the vendors tell you, it's unavoidable: you simply must copy all that data.

In practice, this means that while your array is rebuilding, your performance SLAs go out of the window. If this is for an interactive server, such as a TP database or web service you end up with lots of complaints and a large backlog of work.

The result is that as disks get bigger, the recovery takes longer. This is what make RAID less desirable, not the possibility of a subsequent failure - that can always be worked around.

ZFS is not Obama! (1)

theendlessnow (516149) | more than 5 years ago | (#25461787)

It's implied in the Slashot description that ZFS solves the problem of drive failure. It does not. Just want to make that clear. In fact, I'd argue that there is actually more risk inside of ZFS with regards to the actual problem presented here.... for those that believe all is doom and gloom with regards to RAID.

Doesn't make sense to me (0)

Anonymous Coward | more than 5 years ago | (#25461795)

I don't understand how the spec'd URE of a single SATA drive can be translated across multiple drives whose combined capacity happens to match that URE. I would think that the success or failure of each drive is independent of the others and the fact that you have 6 2-terabyte drives doesn't mean that you have to have a URE. You'd think that you'd have to have an URE on 7 2-terabyte drives in any configuration fi that were true.

Raid 5 failing, Raid 6 not far behind... (0)

Anonymous Coward | more than 5 years ago | (#25461831)

...clearly, Raid 7 is needed.

RAID6 = Win (3, Insightful)

MukiMuki (692124) | more than 5 years ago | (#25461851)

Scrub once a week, or once every two weeks.

RAID6 isn't about losing any two disks, it's about having two parity stripes. It's about being able to survive sector errors without any worry.

It's about losing ONE drive and still have enough parity to replace it without any errors.

RAID6 on 5 drives is retarded, tho, because it leaves you absurdly close to RAID1 in kept space. RAID6 is for when you have 8-10 drives. At that point you barely notice the (N - 2) effect and you have a fast (provided your processor can handle it all) chunk of throughput along with an incredibly reliable system. Well, N-3 with a hotswap.

Personally, I think I'd go RAID-Z2 via ZFS if only because it's a little bit sturdier a filesystem to begin with.

In Soviet Russia... (1)

InSovietRussiaTroll (1282606) | more than 5 years ago | (#25461871)

They recycle old news forever!

For fuck sake, at least read the summary (1)

nedlohs (1335013) | more than 5 years ago | (#25461927)

Sure expected the "editor" to actually look at the article is excessive. But:

"Disk drive capacities double every 18-24 months. We have 1 TB drives now, and in 2009 we'll have 2 TB drives."

Is an obvious indication that this article is old since 18-24 away puts you in 2010 now...

PANIC! (0)

Anonymous Coward | more than 5 years ago | (#25461959)

12 TB of your carefully protected -- you thought! -- data is gone.

oh noes!!! it will be even worse when you have 13TB instead of 12TB!!!!

now read that BS article and downmod this "news"...

RAID-10 (2, Insightful)

tonytnnt (1335443) | more than 5 years ago | (#25461979)

RAID-10 ftw? Expensive I know, but at least you have a full layer of redundancy rather than just a parity drive.

Not true by my experience... (1)

mr_rarr (636892) | more than 5 years ago | (#25462029)

Well what the author is saying is not true by my experience..

When I configure a new server with RAID, before handing over the box, I will test every single RAID 1/5/10 sets by pulling one drive out at a time for a couple of minutes then I would put it back in until the rebuild is complete.

I worked with multiple Dell PowerVault MD1000 using 15 1TB SATA drives and never have I ran into an issue of not being able to complet the rebuild because of the issue mentioned in the article.

I can tell you that drives will fail but if you have the right monitoring in place and catch when a drive fails or about to fails and with the right RAID solution in place, you can consider your data pretty safe. RAID is not a backup not 100% but chances are that a RAID solution will surly save your butt when a disk dies and you just don't have the time to rebuild / restore a box.

1 in 10^14 bit is not what I observe (4, Informative)

gweihir (88907) | more than 5 years ago | (#25462041)

My observed error rate with about 4TB of storage is much, much lower. I did run a full surface scan every 15 days for two years and did not have a single read error in about two years. (The hardware has since been decomissioned and replace dby 5 RAID6 Arrays with 4TB each.)

So, I did read roughly 100 times 4TB. That is 400TB = 3.2 * 10^15 bits with 0 errors. That does not take into account normal read from the disks, which should be substantially more.

My solution (2, Insightful)

SuperQ (431) | more than 5 years ago | (#25462053)

I'm in the process of building a new 8x 1T array. I'm not using any fancy raid card. Just a LSI 1068E chipset with a SAS expander to handle LOTS (16 slots in the case, using 8 right now).

I'm not putting the entire thing into one big array. I'm breaking up each 1T drive into ~100GB slices that I will use to build several overlapping arrays. Each MD device will be no more than 4-5 slices. This way if an error occurs on one disk in one part of a disk I will have a higher probability of recovery.

I may also use RAID 6 to give me more chance of rebuilding.

Disk errors tend to not be whole disk errors, just small broken physical parts of a single disk.

SMART will give me more chance to detect and replace dying drives.

Error Detection and Correction (1)

Moof123 (1292134) | more than 5 years ago | (#25462069)

In communications design, such as cell phones or digital TV, channels with low reliability (fading, burst interference, etc) are tasked with getting much better overall bit error rates than you'd think you could given all the crap spewed into the RF spectrum. I'm kind of confused by why the same techniques of forward error correction, interleaving, and such aren't employed more aggressively for hard drives (maybe they are more than I am thinking, maybe that's how you get to 10^-12 in the first place?). 10^-12 bit error rate is phenominally good compared to what most digital communications devices deal with.

Typically you throw away 20-30% of your available channel with extra bits due to the encoding (imagine hardware encoding by the hard drive as it writes the bits), but you are guaranteed that if you can get most of the bits right (I'm talking 99%, not 99.9999999%) you can get the original data back, or at least know that you didn't. Interleaving spreads the bits around, so one dead sector (or 10 in a row) can easily be rocovered automatically.

UnRaid? (1)

speedingant (1121329) | more than 5 years ago | (#25462081)

I use Unraid for all my data storage. The parity and data isn't striped across all drives, only one drive is parity. If I loose two disks, I only loose the data on those two disks. I've got 4TB of home storage at the moment, but will eventually scale it. Best thing is that it runs on most hardware, and boots off a memory stick. It happens to run Reiser FS though, so when a HD dies, it really makes your life hard to get any data back. But thats why you have backups, right?

Poor summary (0)

Anonymous Coward | more than 5 years ago | (#25462091)

If one read error occurs in reconstruction of the array you lose the piece of data its tied to - not everything. Still get to keep 99.999% of it.

*eye roll*

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...