RAID Problems With Intel Core 2? 284
Nom du Keyboard writes "The Inquirer is reporting that the new Intel Core 2 processors Woodcrest and Conroe are suffering badly when running RAID 5 disk arrays, even when using non-Intel controllers. Can Intel afford to make a misstep now with even in the small subset of users running RAID 5 systems?" From the article: "The performance in benchmarks is there, but the performance in real world isn't. While synthetic benchmarks will do the thing and show RAID5-worthy results, CPU utilization will go through the roof no matter what CPU is used, and the hiccups will occur every now and then. It remains to be seen whether this can be fixed via BIOS or micro-code update."
don't worry (Score:5, Funny)
sum.zero
Re:don't worry (Score:5, Funny)
Where's the bug? My RAID 0+0.999999998 works just fine Intel Core 1.99904274017.
Why aren't you running a dedicated controller...? (Score:5, Insightful)
Seems odd to me that the inquirer is the only one reporting this. How about a real hardware review site?
Re:Why aren't you running a dedicated controller.. (Score:5, Interesting)
Still, this is a problem for Intel. Their products are supposed to do what they do extremely well under all conditions. I hope that they find a way to fix this admittedly niche problem.
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
The interesting question is what other peices of software that we run will get unexpectedly bad performance.
( I have > 2TB of hardware RAID 5 at home so I was wondering
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:3, Informative)
Anyway, it's not entirely a hw/sw combo. These types of raid controllers are entirely software based. They consist basically of an ata or sata controller and an interrupt handler. When the disk is being accessed in legacy bios mode (ie during an os install, etc) the cpu pulls the interrupt to write to the disk and the BIOS calls the software stored on the card. This software is executed by the BI
Re:Why aren't you running a dedicated controller.. (Score:3, Funny)
No, it's aFRAID array. Oops, forgot the space.
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
RAID stands for Redundant Array of Inexpensive/Independent Disks. Nowhere does it say "Controlled By A Dedicated CPU" ("RAIDCBADC"? Doesn't quite sing like "RAID"). Software RAID is as much RAID as a top of the line server RAID controller with RAM and a battery backup. It isn't as fast, sure, and it loads the system CPU, but it is still RAID. Calling it "FAKERAID" is just pretentious and misleading. The data integrity benefits are still present, as are some performance benefits in some circumstances (in fact, Linux RAID is demonstrably faster in some workloads than a top end Adaptec hardware RAID controller, though this is the exception rather than the rule)
That said, I hate pretty much all RAID controllers (whether software or hardware). Linux software RAID means that I can drop the disks into any PC and access the data. Every RAID controller from Promise, Adaptec, and Tektronic requires me to use their disk format, and if I lose the controller I lose the data until I can get another controller. Sure, in high availability environments, you keep a spare...but with Linux software RAID, every PC in the office is a spare controller. That's my kinda redundancy. I've even had two identical Adaptecs with different firmware lead to pretty massive data loss during a server migration. Thankfully there were good backups. I've never had similar problems moving Linux software RAID disks into a new Linux box.
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
What if you have a lot of photos, music or movies - these aren't unusual things these days. I don't want to go rummaging through DVDs to find the picture I want, I want to fire up f-spot and see it there straight away.
RAID5 provides sensible protection against data loss when using consumer hard disks - software RAID5 is readily available on linux and hard disks in the 2-300GB range are easily affordable. You can often pick them up for $50 after rebates. So I can get a TB of storage for a few hundred dollars, but to use hardware RAID5 would probably double the cost. Fine if you're an enterprise, but not fine if you're using it at home.
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
My other issue is with people forgetting the idea behind being sensible about what needs to be protected and how much it should cost. There is no reason why my personal collection of photos, music and video should cost me so much. Software RAID is way more than adequate for providing a cheap way to store my files. If data protection AND peak performance are what you need, then yes you need to go full hardware. WHERE'S THE MIDDLE GROUND PEOPLE?
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:4, Insightful)
There's no "middle ground", there's cost-benefit analysis.
I.e., is it worth my time to spend $50, $100, $200, $500, etc, and an hour a week to mirror a pr0n collection? Some people would say $50 and 5 minutes, and others would say $500 and 6 hours a week. And some would say, "Chunk it. If the disk dies, I'll just download it all again."
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:3, Informative)
The nice 3ware cards for 100 bucks are NOT hardware raid, they use the CPU to calculate the RAID, it might even say it is in the literature but working at company (tech support) who sells servers that use 3ware for 80% of it's business, I can definitely tell you this isn't the case.
You CAN get a hardware based 3ware card, but then you are looking at 400-500 bucks (+some for the battery backup unit).
Plus if you read the parent correctly, 4 300GB hard drives for 50 bucks t
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:3, Informative)
The parallel card is the $110 on newegg.
From Newegg StorSwitch switched architecture delivers the full performance benefit of Parallel ATA's pointto- point architecture up to 133MB/sec per port On-board processor provides true hardware-based RAID and intelligent drive management functions BIOS set up utility and 3ware Disk Management (3DM) web-based management software Bootable array support for greater system fault tolerance
http://3ware [3ware.com]
Re:Why aren't you running a dedicated controller.. (Score:2)
Although if you really want someone to do your legwork for you I'd point out that there are cards in the 100 dollars range with 4 channel hardware raid 5 from LSI, Highpoint, and Promise. I can't really speak to the quality of these cards as I use 3ware cards myself. But as with most things the more $$$ the more features, support, and reliability. Then again I don't really understand the need for super high storage speeds for TB+ systems in
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
You're generalizing a little too much. For example: I have >1TB storage on my mythtv box (I just like to have a good selection of stuff to watch when I finally get to watch tv, and I'm never at home when the shows I like are being broadcasted), and I'm using software RAID5 on that. That is, software raid5, on shared controllers: All together seven disks off the mainboard, from a mixture of pata and sata connectors. I wouldn't do this on something like a server, but it's plenty fast enough for mythtv. It also gives a lot of protection for the array of disks, and it's a much, much better option than the weekly backup you suggest (first of all, a backup would take ages, cost waay more in disks (which wouldn't even fit in the HTPC), and last but not least: without raid5, if one disk dies, I could lose up to 7 days of recordings...).
Re:Why aren't you running a dedicated controller.. (Score:2)
Oh the humanity..
True, it's only TV, but still, better with raid5 than trying to do weekly backups...
ps: TV does become a whole lot more interesting when you can choose from a large set of recorded shows, plus when you can play it back at 1.4x real-time speed with mythtv's fantastic 'time stretch', which speeds up TV shows without distorting the sound.
Re:Why aren't you running a dedicated controller.. (Score:2)
Am I halucinating to recall something happening like this a long time ago with Intel?
Re:Why aren't you running a dedicated controller.. (Score:2, Informative)
Using RAID5 in software (be it completely in software like Linux MD or Windows Dynamics Disks, or 99% in Software, like most Onboard RAID Controllers out there) isn't a good idea if you want to run an "enterprise" setup. It might be okay for your mom's basement, or for test systems.
But productive systems should be using real raid controllers, equipped with half a gig of cache memory, a battery backup in case of a power failure for the cache, and dedicated processor for the raid5
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:2)
Especially given the fact that most CPU usage is less than 18% when calcualting parity for RAID 5, so compare 18% of your CPU cost and see if it is worth it to lose that much over head (or buy the next higher model) OR pay 400+ for hardware RAID.
I'd say RAID 5 is the best solution for home backups, you have redundancy for disk failure and with monthly or even quarterly backups you should be fine. (who
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:2, Informative)
CPU utilization in RAID5 configurations is almost entirely offloaded to the RAID controller.
The article (including spelling errors) fails to mention a lick about t
Re:Why aren't you running a dedicated controller.. (Score:4, Insightful)
That's because you can do RAID 0, 1 or any combination of 0 and 1 without needing parity data. The performance killer on RAID 5 (and any other form of RAID that requires parity) is in the XOR operations used to compute and verify the parity information. In order for RAID 5 to perform at a satisfactory rate and not totally bog down your CPU, the XOR calculations should be handled on a dedicated hardware controller, not in software.
However, for non-parity RAID setups the amount of CPU overhead is almost trivial, so referring to "fake RAID" or "software RAID" with the integrated RAID controllers on most motherboards is a misnomer. That being said, at least one of these articles is talking about servers using third-party RAID controllers.
Re:Why aren't you running a dedicated controller.. (Score:5, Informative)
No, no, no, no. The processing overhead of parity calculations is miniscule on any remotely modern CPU (even a paltry 300Mhz Pentium 2 has a parity throughput of ~700M/sec).
The performance killer on parity-based RAID configuration is the additional disk reads required to calculate the parity, *not* the parity calculations themselves. Which is why modern software RAID is typically faster than hardware RAID until you get into larges numbers of disks and/or machines with limited bus bandwidth.
This "RAID 5 is slow because of parity calculations" meme must die (although, admittedly, it's a good indicator of whether or not someone really understands what's going on).
Re:Why aren't you running a dedicated controller.. (Score:2)
This test is interesting for two reasons:
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:3, Interesting)
Re:Why aren't you running a dedicated controller.. (Score:3, Interesting)
Re:Why aren't you running a dedicated controller.. (Score:2)
Until the 9500 (and MAYBE 9000) series ALL of 3ware's offerings were software RAID. this is why you couldn't initalize a RAID array until the OS booted (and 10 minutes after that), it even says this in the manual.
I worked in tech support for a company that sold these for 2 years and talked with the 3ware techs quite extensively on the subject.
As a company they don't officially admit it, but from a technical standpoin
Re:Why aren't you running a dedicated controller.. (Score:2)
-Rick
Re:Why aren't you running a dedicated controller.. (Score:2)
I, for one, intend to run RAID 5 on my next home system, and not just for Geek Pride. Three hard drives with the advantage of mirroring for reliability, striping for performance, and the loss of only 1/3d of my hard drive space for this redundancy, verses 1/2 for mirroring alone, and no redundancy with striping alone. And since I'm doing this for improved performance, I want that performance. Three modern moderate performance hard drives are hardly expens
Re:Why aren't you running a dedicated controller.. (Score:2)
If you'd read TFA you'd see that the problem has shown itself with a Woodcrest (the next Xeon) CPU using an IBM RAID controller.
Re:Why aren't you running a dedicated controller.. (Score:2)
-Rick
Re:Why aren't you running a dedicated controller.. (Score:2, Informative)
Because it's often slower to do so. We ran tests on a good Adaptec u320 raid controler about a year back and though cpu usage was good. We got much better performance out of Linux softraid5. I would suspect this was because the host cpu was faster than that on the controler.
Not to mention there is a huge cost savings in going with a softraid solution.
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:2)
No RAID systems use one channel per drive, maybe you are thinking connector, in which case that would be SATA, and possibly IDE to maximize bandwidth.
Most RAID cards have more than one channel per card, but multiple connectors per channel (7-15 on SCSI, up to 12 that I have seen on SATA, and 2 per channel for IDE).
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:2)
I have installed a software Raid5 at work for online backups of workstations. 250GB SATA disks cost nothing (~80?); it'd pain my anus to fork out a kilobucks or two to pilot them. Sorry if that's not enterprisy [thedailywtf.com] enough for you!
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
Because if your dedicated controller goes you have to find the same make & model of controller. On no notice. Possibly a few years after that make and model has been discontinued.
With software RAID-5, any controller that works with your host bus (PCI) and HDD bus (ATA, SATA, or SCSI) will do just fine.
Re:Why aren't you running a dedicated controller.. (Score:2)
If your not going for ultimate disk performance, and your cpu isnt overly burdened, the use of software raid is sensible.
Re:Why aren't you running a dedicated controller.. (Score:2)
The first of the two referenced articles talks about them using Woodcrest CPUs (the Conroe-based replacement for the Xeon server CPUs) on IBM systems that used IBM ServeRAID controllers. They obviously aren't talking about the Intel RAID controller integrated into the chipset. And while I agree that the CPU should have little
Re:Why aren't you running a dedicated controller.. (Score:2)
Because such controllers use a proprietary disk format. So when (not if) your controller breaks, you need to get a new one to be able to read your data. Whereas when using a free (libre) operating system and software raid, you'll always be able to get to your data.
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:5, Insightful)
The articles are both very light on technical details, and somewhat vague as to what's really going on. (Admittedly, maybe they don't know it.) In the first article [theinquirer.net], they allude to the problems being the result of the "softmodem"-like RAID systems that modern integrated motherboards use, which would remove some of the blame from the processor. But then they also suggest that the same problem occurs with dedicated RAID controllers [theinquirer.net] (IBM ServeRAIDs -- I think these are dedicated controllers), which don't cause too much CPU load at all
It doesn't seem like it would be that difficult to pin the blame down to the particular component: is it the integrated RAID subsystem utilizing the processor inefficiently? Or is it the processor itself, being slow? And if it was the processor, why wouldn't this slowness be exhibited in other situations?
Seems to me that what needs to happen, is for somebody to do a test with a Conroe processor in a motherboard that doesn't include any of the integrated, offload-work-to-the-processor type of integrated subsystems (RAID, sound, Ethernet), use a 'real' hardware RAID controller, and see what the results are. If there are still problems in that scenario, then there would seem to be something wrong with the processor, and this could be confirmed with simulative benchmarks.
As a criticism of Intel's complete "systems" (processor plus chipset) I suppose this is a valid criticism, but I'd like to see more of a breakdown as to where the performance hit is coming from.
Re:Why aren't you running a dedicated controller.. (Score:2, Insightful)
Re:Why aren't you running a dedicated controller.. (Score:2)
Re:Why aren't you running a dedicated controller.. (Score:3, Interesting)
Problem (Score:4, Insightful)
Re:Problem (Score:2)
Re:Problem (Score:3, Interesting)
Load byte 1.
Load byte 2.
XOR bytes 1 and 2.
Store result. There are a few things that could be wrong here. The XOR performance could be bad. This seems a bit unlikely but XOR is not an incredibly common operation so it wouldn't slow down too much else.
It could be that the pattern of data was bad for cache usage. This would be slightly odd, since it should be a series of 4K linear blocks.
It could be low I/O performance between the chip and the on-board controller. This seems the
XOR is very common (Score:4, Informative)
Re:XOR is very common (Score:2)
Re:Problem (Score:2)
I'd be more likely to bet there's SMP locking issues in the driver. The performance of XOR is negligible in the equation here.
--JoeRe:Problem (Score:3, Interesting)
Core 0 loads byte 1, Core 1 loads byte 2, Core 1 or Core 2 has a cache miss on the XOR...(Do the cores share a cache?) Or it could be a locking problem. XOR is very common, and it would surprise me if it was slower than on previous intel chips.
Re:Problem (Score:2)
It is, however, a lot less likely that some of the other alternatives.
It's only onboard RAID (Score:3, Insightful)
Re:It's only onboard RAID (Score:2)
Re:It's only onboard RAID (Score:2, Informative)
Professional IT doesn't work like that. You have a maintenance contract on your machine, usually from the machine manefacturer itself (like IBM, HP, DELL, whatever floats your boat). You buy this maintenance contract depending on the time you will need the machine (they're usually available from 3-5 years).
You renew the machine before the contract runs out. IBM, HP, DELL running out of Business seems very
Re:It's only onboard RAID (Score:2)
It happens with other hardware based non intel based boards as well. IBM raidserve as an example
Another poster mentioned that it could be DMA related or the way the new speculative branch prediction algorithms work. Perhaps a race condition exists that rarely happens with earlier chips but is executed with the newer ones often which would slow it down.
Re:It's only onboard RAID (Score:2)
*NOT* just on-board RAID (Score:2, Interesting)
The reason was that there were severe problems when Woodcrest was paired with a 1E RAID field when using IBM ServeRAID controllers. The problems didn't occur just in benchmarking, it was the every-day usage model that produced unexpected errors.
ServeRAID controllers aren't some cheapo CPU-based RAID, it looks like this might be a more serious problems.
Timing problem (Score:4, Insightful)
There was a similar problem that I had to wrestle with on a Linux when runnig 3Ware RAID controllers w/ RHEL3 on fast dual-processor servers. When battery backed write caching was turned on, the fast acceptance of IO requests (by the CPU's and then by the hardware RAID controller) lead to awesome sustained performance for short bursts, but under constant load would suddenly hit a wall and then IO would practically hang. (https://bugzilla.redhat.com/bugzilla/show_bug.cg
Re:Timing problem (Score:3, Insightful)
Is the sky falling? (Score:5, Funny)
No. No, it cannot. Sell your stock. Rip the CPU out of your boxen. One hundred ten billion dollars in market capitalization has disappeared in a flash with the publication of this groundbreaking article in the Inquirer.
Intel has signed its own death warrant. As goes RAID5, so goes the world.
The Inq (Score:2, Flamebait)
Anyone got independant verification of this startling discovery?
Re:Oh Ye of Little Faith (Score:2)
For the record, no I still haven't read TFA. Though I did break my unwritten rule of not reading slashdot stories whose title ends with a "?".
I dunno I've had bad luck with Raid5 (Score:2)
For saftey's sake, I just used 6 hard drives with 2 pairs striped and then had those drives mirror each other with 2 extra that would go to a drive when one failed. So in theory I could loose 3 drives instead of two and still keep my data.
And yes... This was on my personal setup for no good reason other than a big ego, but in reality Raid5 isn'
Re:I dunno I've had bad luck with Raid5 (Score:2)
2 chunks minus 2 chunks equals zero chunks, get it?
"...in reality Raid5 isn't that useful ore efficient unless you are using enteprise applications that requires 100% uptime and you have way more than 3 hard drives (just in case two of them fail on you at once for no particular reason) and then you should have that server mirrored by another one s
Likely a driver bug (Score:4, Insightful)
It's almost a certainty that this is a software problem of some sort. Driver bugs are the most common source of "hardware" instability, particularly on Windows. Drivers are often written by clueless intern-level engineers, and quickly forgotten once the drivers initially pass based Windows hardware quality tests.
Re:Likely a driver bug (Score:3, Insightful)
Why is Intel's hardware bad just because Windows performs poorly on it?
I'm a little surprised that, all the way through BOTH articles and this thread, it took this long for someone to ask if it was a HARDWARE or SOFTWARE issue.
As Carlos Mencia would say, "Dee Tuh Dee".
Re:Likely a driver bug (Score:2)
Re:Likely a driver bug - Excuse Me, But!!! (Score:3, Insightful)
Oh wait, they did. Not only did they say so a long time ago, they publish documentation and maintain a compiler to help you optimize for the way their processors work.
Does B-2 Step Fix This? (Score:2)
Intel hints that Conroe is going to release at B-2 Stepping as Intel Core 2 Duo processor. As for the previous version, a problem was found to make the system full loaded. It's only solved in the new stepping. We don't encourage anyone to buy the engineering sample from the web. The retail version is going to release in the end of this month, and it's much stable.
Software RAID not CPU-bound on modern hardware (Score:2)
With the drives on the PCI SATA controller, I'm bu
Re:These are the cheesy RAID cards, right? (Score:4, Informative)
Re:These are the cheesy RAID cards, right? (Score:4, Informative)
Re:These are the cheesy RAID cards, right? (Score:2)
He's probably smoking some of the numerous benchmarks on Linux systems demonstrating that hardware RAID from vendors like 3ware perform similarly to plain SATA controllers with a software RAID configuration. Most of the free ones online are a little old, but ten seconds with google gives me this cheap-ass solution [wustl.edu] for a start). It's true that you can choose to spend enough to buy a controller that does perform better, but they're incredibly costly. For all but front-line high-performance computing machines
Re:These are the cheesy RAID cards, right? (Score:2)
Performance gets increased significantly by decreasing the number of iop's to phsyical media or decreasing the ammount of time to ack to the host write complete. How do you do tha
Re:These are the cheesy RAID cards, right? (Score:3, Informative)
Re:These are the cheesy RAID cards, right? (Score:2)
Lets say you have a disk subsystem that can perform 10 GBPS. Now I will take all of that and run it over a PCI-X bus (same thing applies to PCIe except the numbers are higher) and get throttled to 6.8Gbps for a throughput of 4.3Gbps.
Now I have a hardware RAID solution. I get 10Gbps from the disks, process it on the card - ship 6.6 Gbps over the bus, no problem.
Now I haven't even mentioned the CPU utilization. Lets not go t
Re:These are the cheesy RAID cards, right? (Score:2)
Right. The extra few fractions of a percent of CPU and memory bandwidth consumed by software RAID-1(0) can cripple even the most powerful of servers. And of course by "cripple" I mean "go almost completely unnoticed while reducing dependencies on notoriously buggy hardware and poorly supported drivers and management tools you can't even get the source for". Clearly
Re:These are the cheesy RAID cards, right? (Score:2)
If it's a server, with heavy processor load and lots of disk reads and writes, a good, intelligent RAID controller is the way to go.
And having a second identical unit on the shelf is very smart. After all, how much is your data worth?
Re: (Score:3, Interesting)
Re:These are the cheesy RAID cards, right? (Score:3, Interesting)
Organizations should look into this and not the vendor for their server for any raid setup. It would be nice if they all did as a server is not a desktop and the data is needed NOW when it goes down.
Re:These are the cheesy RAID cards, right? (Score:2, Insightful)
Re:These are the cheesy RAID cards, right? (Score:2)
Re:These are the cheesy RAID cards, right? (Score:5, Insightful)
Oops. (Score:2)
Did you get a refurbished drive that was once part of a RAID-5 array? ;)
Re:Won't matter for most users (Score:2, Funny)
Re:Won't matter for most users (Score:3, Funny)
Re:Talk about Fear Mongering (Score:2, Informative)
Next QUESTION!
Re:Talk about Fear Mongering (Score:2)
Re:doesn't matter (Score:2)
Mirroring uses 1/2 the disk space for redundant storage. It works with even numbers of hard drive 2 and above.
RAID 5 puts all the redundant storage on a single drive, and works with numbers of hard drives 3 and above.
Example: with 4 equally sized hard drives, mirroring would only give you .5 the storage capacity of all the drives. RAID 5 would give you .75 the storage capacity of all the drives. A much better buy, especially considering that many computers
Re:doesn't matter (Score:2)
No, parity is distibuted across all disks in a raid 5 array. raid3/raid4 use a single drive for parity.
Re:Question (Score:2)