×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Cray Replaces IBM To Build $188M Supercomputer

samzenpus posted more than 2 years ago | from the we-have-the-technology-we-can-make-it-better dept.

IBM 99

wiredmikey writes "Supercomputer maker Cray today said that the University of Illinois' National Center for Supercomputing Applications (NCSA) awarded the company a contract to build a supercomputer for the National Science Foundation's Blue Waters project. The supercomputer will be powered by new 16-core AMD Opteron 6200 Series processors (formerly code-named 'Interlagos') a next-generation GPU from NVIDIA, called 'Kepler,' and a new integrated storage solution from Cray. IBM was originally selected to build the supercomputer in 2007, but terminated the contract in August 2011, saying the project was more complex and required significantly increased financial and technical support beyond its original expectations. Once fully deployed, the system is expected to have a sustained performance of more than one petaflops on demanding scientific applications."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

99 comments

Raise taxes NOW! (-1)

Anonymous Coward | more than 2 years ago | (#38048052)

We need to undo the Bush-era and Reagan-era tax cuts, and raise taxes across the board. I'm willing to pay slightly more, are you? It won't impact my day-to-day life at all, having a financially sound USA will do wonders for our children's lifestyle. It will keep us as the strongest and most prosperous country in the entire world. I will not vote for anyone that answers to Grover Norquist. Not now, not ever. This man was not elected by anyone and deserves no role in our government. The people who would cut government to the bone, while living in their 'encampments' or 'compounds' need to be overwhelmed by the bulk of us who understand what a society needs to prosper. Excessive government spending and taxation is bad. Nominal spending and taxation is necessary. Just because some is bad doesn't mean all is bad.

Re:Raise taxes NOW! (-1)

Anonymous Coward | more than 2 years ago | (#38048934)

My tax rate (what I pays vs AGI) is around 19-20%. What's your tax rate, since you don't pay enough? Better yet, if you're willing to pay more, why aren't you paying more?

Awesome (0, Troll)

dev380 (2507404) | more than 2 years ago | (#38048102)

They also noted [evenweb.com] that they will give free runtime for 1000 non profit organizations that help make world better place that are voted by internet users.

Aha. Bulldozer sucks my ass. (3, Interesting)

unity100 (970058) | more than 2 years ago | (#38048128)

Along with the cray they are upgrading (#3 in the world now, will be #1 when complete) and the one lockheed martin ordered (3 days ago) this is the third supercomputer that was ordered in the last 3 weeks to use opterons (bulldozer 16 cores).

the cpu sucks so much that, it is exclusively dominating the SUPERcomputer market.

Re:Aha. Bulldozer sucks my ass. (5, Informative)

dutchwhizzman (817898) | more than 2 years ago | (#38048178)

Designing supercomputers involves a lot of investment in inter-CPU messaging and memory sharing. Once a supercomputer-vendor has committed themselves to a platform, it's not easy to migrate to another. Given the volumes they sell, design costs will have to be spread on just a few actual installations. Maybe AMD was the best platform to use when these computers were originally designed, but they are outdated now. The fact that these new AMD CPUs will work in "ancient" sockets and use the same interconnects, will make development cost for a performance upgrade lower.

Obligatory car metaphore: Most car manufacturers put old technology in cars they bring out today as well, just because the cost of developing new technology and building production lines is commercially prohibitive.

Re:Aha. Bulldozer sucks my ass. (2)

LWATCDR (28044) | more than 2 years ago | (#38048248)

AMD has held an advantage in systems with more than 4 sockets for a while. You just don't see that many Intel x86 based systems with a dozen or more sockets. Itantium tends to be used on those systems.

Re:Aha. Bulldozer sucks my ass. (2)

Kjella (173770) | more than 2 years ago | (#38049218)

Not sure if they count as systems, but Intel has about 75% of the TOP500 list with AMD about 13%. And that's coming from a period where AMD has had really strong Opterons. But then I don't think each node has a dozen or more sockets...

Re:Aha. Bulldozer sucks my ass. (1)

LWATCDR (28044) | more than 2 years ago | (#38049450)

The Xeon has gotten much better but I will bet that a lot of those systems use smaller SMP nodes coupled with Infinityband or some custom interconnect as their structure where the AMD systems use larger NUMA Clusters linked with Infinityband or some custom interconnect. That is just my guess of course.

Re:Aha. Bulldozer sucks my ass. (1)

trum4n (982031) | more than 2 years ago | (#38048318)

Exactly. Why are cars still gas powered? Because building a real electric car takes real design work. They rather just strap a new engine and some uglier panels on the body of an existing car.

Re:Aha. Bulldozer sucks my ass. (1)

Billy the Mountain (225541) | more than 2 years ago | (#38048946)

Trolling? OK I'll bite. "So you don't think it has anything to do with the fact that the energy density of gasoline is 12,200 Wh/kg while lithium ion batteries have an energy density of 439 Wh/kg plus a limited charge cycle of ~1000?"

Re:Aha. Bulldozer sucks my ass. (1)

drinkypoo (153816) | more than 2 years ago | (#38057812)

Cars are still gas-powered because of massive collusion. They are now finally starting to bring out production EVs because China was going to do it sooner or later -- they're producing electric vehicles of all types as fast as they possibly can, most of them suck but it's only a matter of time as there's a lot of problem-solving to do, but EVs are conceptually simple. But the big three automakers can agree on one thing, and that is that EVs are bad for their bottom line. The dealers for the Big 3 depend on service to keep them afloat, and a percentage of owners will always go to the dealer. Since American cars have improved significantly in quality of late since the Japanese were totally eating our lunch previously, that revenue has dropped off to next to nothing. GM has succeeded in raising non-warranty service revenues only about ten percent. Dealers also sell parts at typically exorbitant prices, so they depend on vehicles failing to make their money. But EVs ought to fail less, and who takes their car to the dealer for suspension work? It has to go somewhere with an alignment rack, and small dealers don't have those anyway.

Re:Aha. Bulldozer sucks my ass. (1)

trum4n (982031) | more than 2 years ago | (#38058572)

The one big problem i have is that GM already did it. The EV1 is still the mark we homebuilt EV guys compare to. All they had to do for today's market is extend it slightly, add a back seat, and drop in some Lithium batteries.

Re:Aha. Bulldozer sucks my ass. (1)

drinkypoo (153816) | more than 2 years ago | (#38062722)

The one big problem i have is that GM already did it.

Sigh. The one big problem you have with WHAT?

All they had to do for today's market is extend it slightly, add a back seat, and drop in some Lithium batteries.

And make it meet the federal crash test standards that they helped write in order to keep cars like that off the market, in order to step on the people of California's attempt to improve our air quality through vehicle emissions reductions.

Re:Aha. Bulldozer sucks my ass. (0)

Anonymous Coward | more than 2 years ago | (#38048430)

Poster obviously hasn't done any benchmarking to back up his baseless claims. Performance is more than clock speed.

Re:Aha. Bulldozer sucks my ass. (1, Informative)

flaming-opus (8186) | more than 2 years ago | (#38048948)

It is true that it costs a lot to switch processors, but lets remember that HPC systems are also very price sensitive. Blue waters will have more than 50,000 processor sockets in it. Xeon processors may be better than opterons, but they also cost a LOT more. Multiply that by 50,000. In the benchmarks I've seen, 16 core opterons almost compete with 8 core xeons in raw performance, but blow the xeons away in price/performance.

Re:Aha. Bulldozer sucks my ass. (2, Insightful)

drinkypoo (153816) | more than 2 years ago | (#38049152)

Because of AMD's design glue logic is cheaper. You can see this reflected in the cost of all levels of motherboard for both AMD platforms vs. their intel competition. This is especially important in a supercomputer. AMD has been easier to build into massively parallel systems for longer. The intel processors are slightly snazzier dollar for dollar, but not that much more amazing. Therefore there are only two reasons you would use intel over AMD to build a supercomputer (cluster size, maximum power limitations) whereas AMD provides the advantages of a better-known platform for the purpose and an improvement in cost which can be significant over the total number of nodes. Since boards exist to let you have more AMD cores in a single system than intel cores, it remains a viable platform for supercomputing, and not just through momentum — though let's face it, how much of intel's success is due to the same factor?

Most car manufacturers put old technology in cars they're bringing out today because we keep buying it, and the same is true of computers.

Re:Aha. Bulldozer sucks my ass. (1)

JBMcB (73720) | more than 2 years ago | (#38049338)

Obligatory car metaphore: Most car manufacturers put old technology in cars they bring out today as well, just because the cost of developing new technology and building production lines is commercially prohibitive.

Not quite - car technology lags behind the marketplace because type acceptance on electronics takes years. A new engine can take six or seven. Especially on low-margin cars, like compacts. A single warranty recall can blow the profit margin on an entire production run. They want the latest tech in their products, but they aren't going to throw profit out the window if it breaks.

Re:Aha. Bulldozer sucks my ass. (1)

neokushan (932374) | more than 2 years ago | (#38048216)

It sucks as a desktop processor, though, which is what most of the people on slashdot care about. It's still up in the air if OS optimisations will solve that, though.

Still, hopefully a good foothold in the supercomputing market will give AMD the oomph it needs to compete with Intel in every field.

Re:Aha. Bulldozer sucks my ass. (2)

iggymanz (596061) | more than 2 years ago | (#38048698)

it sucks for certain kinds of desktop work. for the things I do, core count and size of RAM beats mips, flops, GHz. Modest six-core AMD is better for me than highly priced double or quad core Intel

Re:Aha. Bulldozer sucks my ass. (0)

DurendalMac (736637) | more than 2 years ago | (#38049154)

So...two ordered...DOMINATION! Are you actually astroturfing for AMD's marketing department and getting paid or are you really this much of a desperate fanboy?

Re:Aha. Bulldozer sucks my ass. (1)

unity100 (970058) | more than 2 years ago | (#38050782)

3 weeks, 3 supercomputer orders, the minimum value of each is 200 million. 2000 cpus minimum goes to just one of them. none intel, none anything other than amd.

yes, its pretty much domination with this picture.

Re:Aha. Bulldozer sucks my ass. (1)

DurendalMac (736637) | more than 2 years ago | (#38051120)

So new CPU, orders for the CPU come in, and that's domination? Come back when the SB Xeons hit and we'll see what happens.

Re:Aha. Bulldozer sucks my ass. (0)

Anonymous Coward | more than 2 years ago | (#38053590)

you throw a few hundred thousand cores of any processor into a cluster, and you're going to generate FLOPs galore, even with a shitty interconnect. It doesn't make for a decent desktop, or even server cpu.

Jurassic Park (2)

Dyinobal (1427207) | more than 2 years ago | (#38048158)

Ever since reading Jurassic Park, I've always wanted a Cray supercomputer. No other super computer company had a hand in bringing dinosaurs back to life. Once you've resurrected dinosaurs I don't think that can be topped. I wonder if U of I is planning on doing any dinosaur resurrections with their new super computer.

Re:Jurassic Park (2, Informative)

CaptainJeff (731782) | more than 2 years ago | (#38048206)

The supercomputers in Jurassic Park were Thinking Machines systems, not Crays.

Reference: http://en.wikipedia.org/wiki/Thinking_Machines_Corporation [wikipedia.org]

Re:Jurassic Park (4, Funny)

Dyinobal (1427207) | more than 2 years ago | (#38048228)

I'm referring to the Novel not the movie. But thank you for playing.

Re:Jurassic Park (1)

ackthpt (218170) | more than 2 years ago | (#38048844)

I'm referring to the Novel not the movie. But thank you for playing.

Indeed. The movie was garbage. Spielberg at his worst. Turned a great story into a "Hey, it's full of computer animated dinosaurs!" movie. Changed a lot of characters, too. What's his problem with a lawyer being a fairly decent character, not all of them are scum.

Re:Jurassic Park (1)

bruno.fatia (989391) | more than 2 years ago | (#38049652)

Are you a lawyer? you really should know better...

Re:Jurassic Park (1)

ackthpt (218170) | more than 2 years ago | (#38052684)

Are you a lawyer? you really should know better...

In the book the lawyer is actually a pretty stand-up guy. In the movie the lawyer and some other weasel character are combined and chomped in an act of gratuitous violence.

Spielberg did the same thing in War of the Worlds with the ambulance driver character - again, killed in an act of gratuitous violence.

Re:Jurassic Park (3, Interesting)

LWATCDR (28044) | more than 2 years ago | (#38048368)

There was a time when Supercomputer meant Cray. I remember seeing pictures of Crays when I was a kid and saying. "That is what a computer should look like".

Re:Jurassic Park (0)

Anonymous Coward | more than 2 years ago | (#38049686)

Absolutely! I remember reading about this artist that acquired a Cray Y-MP (think that was the model) and hacked the thing to pieces and set them in amber. I shed some tears.

Re:Jurassic Park (0)

Anonymous Coward | more than 2 years ago | (#38049770)

There was a time for me when using the internet mostly meant using NCSA Telnet. Here's hoping Cray + NCSA = some sort of new wonderfulness.

Re:Jurassic Park (1)

DigiShaman (671371) | more than 2 years ago | (#38056230)

Seymour Cray wasn't just a engineer, he was also a marketing showman. He designed the supercomputer to have form as well as function for the express purpose of capturing market share among those who held the corporate purse strings. Even the Fluorinert cooling feature was turned into a work of art with that whole waterfall reservoir thing.

Re:Jurassic Park (-1)

Anonymous Coward | more than 2 years ago | (#38048458)

We already created HAL at Illinois. You want dinosaurs, too?!

Fine. We'll get on that right after we finish Skynet.

Re:Jurassic Park (0)

Anonymous Coward | more than 2 years ago | (#38048792)

Who cares who makes it, so long as it comes out of Urbana [wikipedia.org] .

Re:Jurassic Park (1)

g00ey (1494205) | more than 2 years ago | (#38048964)

> I wonder if U of I is planning on doing any dinosaur resurrections with their new super computer.

Well, I suppose they could throw in a mainframe emulator while flipping the bird at IBM...

Re:Jurassic Park (0)

Anonymous Coward | more than 2 years ago | (#38049752)

I believe the supercomputer in Jurassic Park was a Thinking Machines CM-5.

Re:Jurassic Park (1)

mcgrew (92797) | more than 2 years ago | (#38049850)

Ever since reading Jurassic Park, I've always wanted a Cray supercomputer.

The computer on your desk is more powerful than a 1972 Cray, at the time the fastest supercomputer in the world. So you already have one.

I want to add GO ILLINOIS! U of I has had supercomputers and lots of computer research for decades. Not sure if U of I has a paleontology department...

Totally surprised. (0)

140Mandak262Jamuna (970587) | more than 2 years ago | (#38048164)

Cray is still alive? Wasn't it gobbled up by Silicone Graphics? and then SGI too went belly up? Well, it is a blast from the past.

Re:Totally surprised. (2)

Dynetrekk (1607735) | more than 2 years ago | (#38048184)

Cray has several of the Top 10 supercomputers on earth, especially in the US. They're pretty nice to work with, too.

Re:Totally surprised. (3, Informative)

wezelboy (521844) | more than 2 years ago | (#38048222)

The Cray name was bought from SGI by Tera. SGI was later bought by Rackable.

Re:Totally surprised. (2)

DrgnDancer (137700) | more than 2 years ago | (#38048462)

It's worth noting that the "new Cray", while they obviously don't make the old vector processor systems that they did originally, makes a really nifty hybrid cluster/SSI (single system image) supercomputer that is notably different than most of what's on the market. Man, seeing articles like this makes me want to get back into HPC stuff. I'm making a bit more doing this corporate crap, but I really miss getting to play with the cutting edge stuff.

Re:Totally surprised. (2)

David Greene (463) | more than 2 years ago | (#38050030)

They didn't just buy the name. They also bought all of the people who designed and built those earlier Cray machines. There are still people at Cray who had a hand in the original Cray 1. It's actually a rather nice mix of expertise, multithreading experience from the Tera side, scalable MPP and vector experience from the Cray Research side.

Re:Totally surprised. (1)

wezelboy (521844) | more than 2 years ago | (#38055972)

Good to know. I'm kinda curious to hear what they've got planned for their integrated storage solution. :-)

old times (0)

mynicknamewasused (962741) | more than 2 years ago | (#38048186)

i used to know by memory, every socket on the market, and every cpu family. but now im like "What is this I don't even " when i decide to update my desk (c2q 2.6 4gb) im gonna have a very hard time choosing.

i also used to know the relative performance of every graphics chip. up to the nvidia 9800 era...

itÂs just sad... im getting old.

Re:old times (0)

Anonymous Coward | more than 2 years ago | (#38048322)

or ... you finally got a life.

Congratulations on that :-)

Re:old times (1)

TheReaperD (937405) | more than 2 years ago | (#38048354)

It's far more likely that you just have better things to do with your time now. It's ok, you'll be able to read reviews from people who still have nothing better to do with their time (or get paid to do it) to help make that decision.

Re:old times (2)

mynicknamewasused (962741) | more than 2 years ago | (#38048408)

even whorse... that comment was intended for the "Intel Launches Sandy Bridge-E Series Processors" article.... OMG, i better go and find me some coffe.

IBM Can do that? (0)

Anonymous Coward | more than 2 years ago | (#38048202)

So IBM bid on something, realized they couldn't do it. And just stopped? What a waste of the customer's time...

Re:IBM Can do that? (0)

Anonymous Coward | more than 2 years ago | (#38048272)

That has happened where I work on multiple occasions

Re:IBM Can do that? (2)

UnknowingFool (672806) | more than 2 years ago | (#38048288)

No they bid on something but later the time and expense was more than they could make a decent profit. I'm sure they could do it, but they don't want to do it.

My first reaction upon hearing the news: (0)

Anonymous Coward | more than 2 years ago | (#38048320)

"That s*** Cray"

Wow (2)

lmcgeoch (1298209) | more than 2 years ago | (#38048336)

Last time I was at the air and space museum in Washington DC I saw a Cray Supercompter http://www.nasm.si.edu/collections/artifact.cfm?id=A19880565000 [si.edu]
    I was extremely excited and tried to show my kids who only saw a very weird big computer thing. A new supercomputer built by Cray sounds like a great idea :)

Re:Wow (4, Interesting)

DrgnDancer (137700) | more than 2 years ago | (#38048630)

Cray has had Supercomputers on the top ten list (and even in the number one spot) again for years now. Ever since they spun off from SGI they've had one of the more interesting architectures in HPC. I was interviewing at ORNL when they were installing Jaguar [ornl.gov] , and I got a pretty in depth description of the hows and the whys. It's no longer the most powerful computer in the world, but it's still a very impressive piece of machinery. Sigh. I really need to get back into HPC.

Re:Wow (1)

timeOday (582209) | more than 2 years ago | (#38049284)

Names like Cray and Silicon Graphics are associated with a time when most of us could only imagine what incredible technology existed behind closed doors, inaccessible to mere mortals. Now all the excitement is behind commodity items that sell for $500 or less. It's fantastic. Yet, where's the mystique? I miss it.

Re:Wow (1)

David Greene (463) | more than 2 years ago | (#38050146)

The mystique is in scaling. It's very hard to run codes on hundreds of thousands of cores and get decent performance. Communication is a huge problem which is why you still see custom interconnect on the high-end systems. Memory architectures on these machines are pretty exotic. It's not just about having a fast processor. It's more about making sure that you can feed that fast processor.

Netcraft confirms it (0)

Anonymous Coward | more than 2 years ago | (#38048422)

IBM is dying

Stick a fork in 'em, this whole "computer" thing was just a fad

Re:Netcraft confirms it (1)

cashman73 (855518) | more than 2 years ago | (#38048646)

Strange. If IBM is dying, then why is Warren Buffett investing $10.7 billion in them [cbsnews.com] ? Perhaps he knows something that we don't know?

Re:Netcraft confirms it (2)

ackthpt (218170) | more than 2 years ago | (#38048778)

Strange. If IBM is dying, then why is Warren Buffett investing $10.7 billion in them [cbsnews.com] ? Perhaps he knows something that we don't know?

I would normally say, "This isn't your father's IBM", but with respect to Mr. Buffett's age, I'm not sure it is his father's IBM, either.

In the 60's and 70's IBM was the company to work for.

In the 80's they began cutting.

In the early 90's they were slashing. We were trying to buy an RS6000 and from week to week I didn't know who I was talking to as the people were exiting so fast. When I ran into difficulty with a security flaw I found myself talking to someone from IBM in Australia who had them send me a stack of tapes and no directions.

Since then I expect IBM has done what a lot of IT companies have done, shop out bits of the work, bring in a lot of green (cheap) workers and try to muddle through the project. I don't see IBM as the tiger they once were. I don't think any IT company is, come to that.

Re:Netcraft confirms it (1)

TheRaven64 (641858) | more than 2 years ago | (#38049890)

I would normally say, "This isn't your father's IBM", but with respect to Mr. Buffett's age, I'm not sure it is his father's IBM, either.

I think his father's IBM was selling typewriters and tabulating machines to the Nazis...

Re:Netcraft confirms it (0)

Anonymous Coward | more than 2 years ago | (#38048700)

You know IBM is on the downward slide when Warren Buffett buys 5.5% of IBM's stock (today) and IBM barely makes $100B/yr in revenue. Slackers!

I've got one (0)

Anonymous Coward | more than 2 years ago | (#38048622)

I've got a 16 core proc for my desktop. It performs so well, I barely know what to do with it besides distributed.net, folding@home etc..

So, when finished it's 0.1 x's the perfrmance of K (0)

Anonymous Coward | more than 2 years ago | (#38048668)

the one petaflop performance mentioned here is significantly below that of the 'K Machine's 10 petaflop,
and i hear that plans are made to increase thaT to 25 pf's .. So, what's all this hoopla about??

ref:
http://www.semiconportal.com/en/

Re:So, when finished it's 0.1 x's the perfrmance o (1)

Sulphur (1548251) | more than 2 years ago | (#38049098)

the one petaflop performance mentioned here is significantly below that of the 'K Machine's 10 petaflop,
and i hear that plans are made to increase thaT to 25 pf's .. So, what's all this hoopla about??

ref:
http://www.semiconportal.com/en/ [semiconportal.com]

The Hooplites are coming. The Hooplites are coming.

Re:So, when finished it's 0.1 x's the perfrmance o (0)

Anonymous Coward | more than 2 years ago | (#38049136)

The one petaflop mentioned is sustained performance for a number of real-world candidate applications, not peak theoretical performance. The machine's peak theoretical performance will be considerably larger than 1PF.

Re:So, when finished it's 0.1 x's the perfrmance o (2)

David Greene (463) | more than 2 years ago | (#38050074)

It's one petaflops sustained performance, not peak. That means actual real scientific codes running at one petaflops, not just Linpack.

IBM backing out... (2)

ackthpt (218170) | more than 2 years ago | (#38048702)

I've been working with an agency who contracted a large project to IBM a few years ago. The results have been ... unimpressive. The training was largely a waste of time, I don't believe they even understood their audience.

Better to see Cray, I think as IBM is shopping out a bit too much of their work to people who aren't up to it .. unless IBM has seen the light.

Just (1)

DinDaddy (1168147) | more than 2 years ago | (#38048806)

imagine a Beowulf cluster of these . . .

Re:Just (1)

Bitmanhome (254112) | more than 2 years ago | (#38054048)

Okay, hang on.

hhhnnnnnnnnnggggg...
hhhhhhhhhNNNNNNnnnngg..
hrrrrrrrhrhrhrrrgg..

I think I got it. It looks alot like a small city. But programming the whole mess is exactly like programming one of them, since each is already a cluster.

Obligatory: +5, Seditious (0)

Anonymous Coward | more than 2 years ago | (#38048962)

But will it run iOS !!!!!!!

Yours In Moscow,
K. Trout

P.S. : John McCain For Prezeedent !!!

Completely different contract/machine/goals (4, Interesting)

gentryx (759438) | more than 2 years ago | (#38049724)

As covered earlier here [slashdot.org] , IBM backed out of the contract because they thought they wouldn't be able to meet the performance requirements for existing codes. They were concerned about clock speeds (POWER7 [wikipedia.org] runs at 4 GHz). POWER7 excels at single thread performance, but also in fat SMP nodes.

What NSCA ordered now is system that is pretty much the antipode to the original Blue Waters: the Bulldozer cores are sub-par at floating point performance, so they'll have to rely on the Kepler GPUs. Those GPUs are great, but to make them perform well, NSCA and U of I will have to rewrite ALL of their codes. Moving data from host RAM to the GPU RAM over slow PCIe links can be a major PITA, especially if your code isn't prepared for that.

Given the fact that codes in HPC tend to live much longer than the supercomputer they run on, I think it would have been cheaper for them to give IBM another load of cash and keep the POWER7 approach.

Re:Completely different contract/machine/goals (0)

Anonymous Coward | more than 2 years ago | (#38049910)

For fuck sake, people. First *maths* and now *codes*?! What's this world coming to?

Re:Completely different contract/machine/goals (1)

NoNonAlphaCharsHere (2201864) | more than 2 years ago | (#38050974)

I know, right? Don't you wish these fucking Brits spoke English like normal people?

Re:Completely different contract/machine/goals (1)

lastx33 (2097770) | more than 2 years ago | (#38052348)

And what, pray, is wrong with "maths"? It is a contraction of mathematics, plural. I agree with the use of "codes" however as while it is syntactically well formed, a better and more correct term to convey the meaning is "orders". For your information, you spelled "furcating" incorrectly, by the way.

Re:Completely different contract/machine/goals (1)

David Greene (463) | more than 2 years ago | (#38050106)

Well, they won't have to completely rewrite all of their codes thanks to OpenACC [cray.com] . They will probably still have to do a bit of restructuring (and that's not a small task) but the nitty-gritty low-level stuff like memory transfers should be handled and optimized by the compiler.

Re:Completely different contract/machine/goals (1)

gentryx (759438) | more than 2 years ago | (#38050978)

That's similar to what PGI [pgroup.com] is doing. And you know what? It's not that simple. You seldom achieve competitive e performance with this annotation type parallelization, simply because the codes were written with different architectures in mind.

This is also the reason why the original design did emphasize single thread performance so much. The alternative to having POWER7 cores running at 5 GHz would have been to buy a BlueGene/Q with much more, but slower cores.They didn't go into that avenue because they knew that their codes wouldn't scale to the number of cores well.

None of the supercomputer codes I know uses such a type of parallelization or accelerator offloading. And the reason for that is not that folks enjoy doing work that a tool could handle for them, but because the tools don't work as well as advertised.

Re:Completely different contract/machine/goals (1)

David Greene (463) | more than 2 years ago | (#38051448)

That's similar to what PGI is doing.

Yes. In fact they've been working together on it.

It's not that simple.

You are absolutely right. That's why I wrote, "not a small task."

You seldom achieve competitive e performance with this annotation type parallelization, simply because the codes were written with different architectures in mind.

The nice thing about this is the restructuring one does for GPUs generally also translates into better CPU performance on the same code. So one can enhance the code in a performance-portable way. That isn't possible to do without compiler directives. With directives, one can avoid littering the code with GPU-specific stuff.

They didn't go into that avenue because they knew that their codes wouldn't scale to the number of cores well.

This article [hpcwire.com] explains that five years ago when NCSA made the bid, accelerators were very exotic technology. The move toward GPUs was actually at the behest of scientists who now see a way forward to speed up their codes with accelerators. Technology shifts and we adapt.

None of the supercomputer codes I know uses such a type of parallelization or accelerator offloading.

None? I know of several. It's all still in its infancy of course, but I'm convinced it's possible to get good speedup from GPUs on real science codes. It's not applicable to everything, but then that's why they aren't CPUs.

Re:Completely different contract/machine/goals (1)

gentryx (759438) | more than 2 years ago | (#38051612)

This article [hpcwire.com] explains that five years ago when NCSA made the bid, accelerators were very exotic technology. The move toward GPUs was actually at the behest of scientists who now see a way forward to speed up their codes with accelerators. Technology shifts and we adapt.

If they are so willing to adapt, why weren't they willing to accommodate IBM's change requests? It's not like IBM was totally unwilling to build a $200 million machine.

None? I know of several. It's all still in its infancy of course, but I'm convinced it's possible to get good speedup from GPUs on real science codes. It's not applicable to everything, but then that's why they aren't CPUs.

I was referring to annotations for GPU offloading. Codes that run on GPUs are in fact so common nowadays that in fact you'll be asked on conferences why you didn't try CUDA if you present any performance measurement sans GPU benchmarks. :-)

Re:Completely different contract/machine/goals (1)

David Greene (463) | more than 2 years ago | (#38051858)

If they are so willing to adapt, why weren't they willing to accommodate IBM's change requests?

I don't have any knowledge of what those change requests were, so I don't know the answer. Everything I have read indicates that IBM wanted too much money.

It's not like IBM was totally unwilling to build a $200 million machine.

From what I have read, it seems that they were. They couldn't keep their costs low enough to justify the expense.

I was referring to annotations for GPU offloading. Codes that run on GPUs are in fact so common nowadays that in fact you'll be asked on conferences why you didn't try CUDA if you present any performance measurement sans GPU benchmarks.

Ah, I misunderstood. I don't think directives have been around all that long (PGI's earilier directives and CAPS's directives come to mind) and they certainly weren't standardized. OpenACC, like OpenMP, should allow scientists to write more portable accelerator-enabled code. In fact the OpenACC stuff came out of the OpenMP accelerator committee as explained here [cray.com] . I think it's highly likely some version of it will be incorporated into OpenMP.

Re:Completely different contract/machine/goals (1)

gentryx (759438) | more than 2 years ago | (#38052982)

I don't have any knowledge of what those change requests were, so I don't know the answer. Everything I have read indicates that IBM wanted too much money.

From what I have read, it seems that they were. They couldn't keep their costs low enough to justify the expense.

True, but only because of the strict requirements of NCSA. If they had been willing to change them, a BlueGene/Q would have been viable.

Ah, I misunderstood. I don't think directives have been around all that long (PGI's earilier directives and CAPS's directives come to mind) and they certainly weren't standardized. OpenACC, like OpenMP, should allow scientists to write more portable accelerator-enabled code. In fact the OpenACC stuff came out of the OpenMP accelerator committee as explained here [cray.com] . I think it's highly likely some version of it will be incorporated into OpenMP.

The reason why I'm so allergic to annotation based parallelization is the experiences folks had with OpenMP. The common fallacy about OpenMP is that it is sufficient to place a "#pragma omp parallel for" in front of your inner loops and *poof* your performance goes up. But in reality your performance may very well go down, unless your code is embarrassingly parallel. In reality especially simulation codes are tightly coupled and memory bound. The parallelization on GPUs is very different from the code on traditional multi-cores. On the latter you'll want to do pipelined cache blocking, while on the former you'll want to do tiling in the GPU DRAM. These are differences in the high level algorithm of a kernel, something which is beyond the compiler to change. Even with annotations.

Instead of a revamped OpenMP, I expect OpenCL to grab a larger share of the market when it comes to writing portable code. Even though OpenCL code by far isn't write once, run everywhere.

Re:Completely different contract/machine/goals (0)

Anonymous Coward | more than 2 years ago | (#38053618)

You are correct that SW has to be reworked to be optimal on GPUs. However, many of the top supercomputers are already headed that direction. 3 of the top 5 super computers in the world are using NVIDIA GPUs.

Re:Completely different contract/machine/goals (0)

Anonymous Coward | more than 2 years ago | (#38054922)

IBM backed out because their quality sucks.

Re:Completely different contract/machine/goals (1)

flaming-opus (8186) | more than 2 years ago | (#38066870)

The bulk of the new blue waters will be pure opteron nodes, with only 35 of the 270ish cabinets using GPUs. They obviously are assuming that most users will be running with only x86 cores. They ordered a few dozen cabs of GPUs, probably with an eye to where the industry will be heading over the lifetime of the machine, not where the users are today.

It's true that interlagos cores are a poor competitor to power7 core to core. However, they fair much better if you use the entire module. Think of interlagos as an 8-core processor with 2 threads per core, and all of a sudden it looks a lot better. Power7 is probably still better, but at ten times the cost.

Furthermore, just because a single node of power7 is an awesome node, does not mean that a many-thousand node supercomputer, composed of those nodes, is also awesome. If the IBM blue waters machine were just about the processors, they would have clustered together a bunch of bladecenter704s. They would not have bothered to bid the p775 system. If you want really fat SMP nodes, then they would have bid a bunch of p795s. Obviously they tried to make a really high-bandwidth shared-memory interconnect for the p775, and they failed. Either it didn't work reliably, wasn't fast enough, or cost too much. IBM didn't step away from the deal because the clock speed was 10% low, or because their costs rose enough to make their margin slim. You don't screw a customer in such a high-profile way unless you're going to lose a LOT of money on the deal.

I notice, looking at the top500 list, that IBM managed to sell the 10 cabinets of p775 that they built for NCSA. A few weather forecasting sites in canada and the UK are running 5 systems at 2 cabinets each. Tellingly, a couple of those customers have a pair of 2 cab systems, rather than a single 4 cab system. That tells me that the interconnect isn't scaling to larger numbers of nodes. A problem for a really big system like blue waters.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...