Breaking Supercomputers' Exaflops Barrier 96
Nerval's Lobster writes "Breaking the exaflops barrier remains a development goal for many who research high-performance computing. Some developers predicted that China's new Tianhe-2 supercomputer would be the first to break through. Indeed, Tianhe-2 did pretty well when it was finally revealed — knocking the U.S.-based Titan off the top of the Top500 list of the world's fastest supercomputers. Yet despite sustained performance of 33 petaflops to 35 petaflops and peaks ranging as high as 55 petaflops, even the world's fastest supercomputer couldn't make it past (or even close to) the big barrier. Now, the HPC market is back to chattering over who'll first build an exascale computer, and how long it might take to bring such a platform online. Bottom line: It will take a really long time, combined with major breakthroughs in chip design, power utilization and programming, according to Nvidia chief scientist Bill Dally, who gave the keynote speech at the 2013 International Supercomputing Conference last week in Leipzig, Germany. In a speech he called 'Future Challenges of Large-scale Computing' (and in a blog post covering similar ground), Dally described some of the incredible performance hurdles that need to be overcome in pursuit of the exaflops barrier."
Has this been turned into another pissing contest? (Score:1, Redundant)
All the talk about who has the fastest / most awesome computer in the world used to make sense --- there were a lot of problems which need huge computational power to help solve
They went from mere gigaflop to petaflop and now they are aiming all the way to break the exaflop barrier
Now, let me ask this --- is there really a case which justifice all the juice ?
From giga to peta, it's already a difference of 1,000 times
From peta to exa, another 1,000
Which means, when they finally break the exa-barrier, they al
Mea Culpa (Score:2)
Oops, sorry,
Should have used "tera" in place for "giga" ...
Re:Mea Culpa (Score:5, Interesting)
Should have used "tera" in place for "giga"
I'm getting tired of all the prefixes, couldn't we just use scientific notation? 1e18 flops means a lot more to me than exaflop.
Re: (Score:3)
1 exponentiated to the 18th power is still 1.
Re:Has this been turned into another pissing conte (Score:5, Interesting)
Well I don't know anything at all about nuclear simulations and fluid dynamics modeling...
But for pure benefit to mankind I'd say folding@home is a pretty worthy project. It's been running for years and has helped make actual discoveries and raised understanding of protein folding's effects.
According to Wikipedia it was running at 14 Petaflops when last updated. Would taking that up to an exaflop be a huge benefit? You bet!
How about being able to simulate an entire life cycle of a human body at atomic scale? That would gain us tremendous understanding of well... EVERYTHING.
Most definitely there are worthy projects that have a real need for exaflop computing and it's not a waste of time.
You remind me of my friend who years ago said that his 802.11b wireless network was as fast as he'd ever need. Guess he didn't plan on people watching multiple HDTV streams throughout the house.
Information != benefit (Score:1)
But for pure benefit to mankind I'd say folding@home is a pretty worthy project. It's been running for years and has helped make actual discoveries and raised understanding of protein folding's effects.
According to Wikipedia it was running at 14 Petaflops when last updated. Would taking that up to an exaflop be a huge benefit? You bet!
While not wishing to critisise folding@home specifically, we should be careful not to assume that there is an automatic progression from data to knowledge to understanding and hence to benefit. And with rising costs (both financial and environmental) we should not blindly assume that building huge supercomputers or running millions of inefficient home computers 24/7 is an inherently good idea.
Re: (Score:3)
Huge supercomputers have the advantage that they are efficient, when compared to projects such as those running "@home", and their interconnects allows them to solve problems that need strong communications between the computing elements. Such problems cannot be solved in an efficient way by this "@home" model, where a machine receives a work unit, computes it and returns the result for final aggregation.
Those interconnects can sum to as much as half the price of building a supercomputer.
When you mention th
That'd be quite a piss! (Score:2)
So if we JUST put roughly 30 of the Tianhe-2s or 500,000 nodes with 100,000,000 computing cores in one big system, we'd have our exascale computer!
Anyone want to venture a guess how long it'd take Intel to make 1,000,000 Xeons and 1,500,000 Phis?
I can't wait to see the day, but me thinks we have a long way to go!
I can't believe some folks thought the Tianhe-2 was going to be the one to break the exaflop barrier! OOPS, only made it 3% of the way there...
Cheers!
Re: (Score:1)
So, we need a beowulf cluster of Tianhe-2's?
Re: That'd be quite a piss! (Score:1)
Hahah, quite nice. You sir should do standup.
Re: (Score:3)
Actually, no, that's the problem/challenge... linking 30 Tianhe-2s would make a supercomputer that is only slightly faster than a single Tianhe-2, because the cores would mainly be sitting idle due to communication latency. Granted this is not true for computations that are completely parallel (e.g. cracking passwords) but that is NOT what "exaflop" means; it
Re: (Score:1)
Exactly. Also, if you just link lots of these together naively, the computation would be crashing all the time, never having time to finish, because the currently standard ways of communicating (such as MPI) make it difficult to handle the loss of a single process, and when you're starting to talk about many millions of nodes, the chances that not a single one of them will crash in the space of the minutes to hours a computation takes is pretty minuscule.
Re: (Score:1)
artificial intelligence will never match natural stupidity
Re: (Score:2)
Re: (Score:2)
The human brain can perform around 200-2,000 Petaflops (0.2 - 2 Exaflops) - when we compare it to computers.
The problem is that we can only access the conscious part, which is probably in the range of 100-200 Flops (Note, no mega, giga, or tera).
The subconscious part is where the real processing power lies. If we could simulate that in a computer, it would be tremendous in maybe understanding how it works.
For example: the human brain has the ability to "foresee" the future within a timeframe of around 0.2 s
Re: (Score:1)
You know, it's comment like this is why I rarely bother coming back to slashdot anymore.
You know, it's comments like this is why i think you're a total dweeb, and everyone knows you really can't get enough of /. while you're sitting there cooped up in your "command center" in your mom's basement stuffing your pizza face with McDonald's fries.
And no doubt behind Slashdot you have a bunch of tabs with Google image searches for "boobies".
1000x buys 6x 4D grid size (Score:2)
I heard a NOAA talk in Boulder about the erroneous Hurricane Sandy prediction. The "European" weather model correctly predicted the rare west turn of the northeastern hurricane while the US models did not. The Europeans used a 20 km wou
Re: (Score:2)
Re: (Score:2)
humans will even worship a rock or lump of baked clay, for something to be a "god" only requires worshipers.
Re: (Score:2)
you'll be glad too know I use money and consumerism only to worship myself.
Re: (Score:2)
nonsense, man is the measure.
Re: (Score:1)
Those are called "sheeple" by those who truly understand the distinction
no, they're called apple customers
Re: (Score:1)
So, you are claiming that Obama is really a puppet animated by a DoE supercomputer, running software written by the NSA?
Re: (Score:1)
So, you are claiming that Obama is really a puppet animated by a DoE supercomputer, running software written by the NSA?
fuck no... obama isn't that smart... he's more like a commodore 64 with a virus
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Re: (Score:1)
That reminds me, what exactly IS that top secret program they use the Department of Energy's super computer for?
virtual porn... the government felt it would be a little unethical to use pixar's infrastructure
Re: (Score:2)
The Jaguar/Titan system mentioned in your link is used for unclassified scientific computing. The NSA is building a computer facility at ORNL, but that's a different system (and was never claimed to be for stockpile stewardship). They don't put classified jobs onto unclassified systems.
Barrier? (Score:2)
Re: (Score:1)
RTFA
Re:Barrier? (Score:4, Insightful)
Re: (Score:1)
Yes, it's the quantitative carrot... when I was learning parallel computing, teraflops were the fantasy milestone we'd reach some day and terabytes was the crazy storage you imagined existed in some NSA datacenter, rather than in your cousin's USB drive. People feel like brilliant strategists every time they point out the next 500-1000x milestone and declare that as the thing that matters to differentiate themselves from all the myopic folk working on today's problem.
Re: (Score:2)
It is a barrier, but that being said it just means no one has done it yet. It doesn't mean it's impossible. A barrier is something to strive to overcome and in spite of all the striving, it feels like a fully blown case of Zeno's paradox, for a while. Only now that we're so much closer to the day that an exaflops will be reached, it seems that we must all chatter about it lest no one will have enough motivation to actually make it happen.
Re: (Score:3, Interesting)
I'm pretty sure the parent is questioning why the word "barrier" is used instead of something like "milestone", which I would have chosen. A barrier implies there is something special stopping you there that you need to work around or resolve, but milestone is just a convenient number to stop at, as in this case. I see no difference between passing exaflop and say 0.9 exaflop, since both require "a really long time, combined with major breakthroughs in chip design, power utilization and programming", so it
Re: (Score:3)
yeah, strange harmonics and shit, and word around the cooler is that it would require an infinite amount of energy as well... that or set the atmosphere on fire or some shit.
Re: (Score:2)
It's just speech. It's a milestone. It's not difficult to exceed one exaflops (the name stands for operations per second, it's not a plural) once you got to, say, 0.99 exaflops. Scientists like to talk in orders of magnitude. Right now we are in the tens of petaflops, but didn't get yet to hundreds. Tiahne-2 gets to 55 pflops, but its sustained speed is a bit bigger than half of that.
Problem is much more about how to get there. It's not just machinery. Is how to actually write and debug programs at that sca
Re: (Score:2)
There is nothing special about reach the exaflop level, unlike say the sound barrier where there are real physical forces that make it difficult to pass.
Scaling is a challenge of course, but the difference between say 0.9 exaflops and 1.1 exaflops is basically just money.
NVIDIA's bread and butter long term (Score:3)
My take away from reading this and the blog post is that, while NVIDIA may consider graphics to be their bread & butter, it looks like they're looking at this space (HPC) very seriously in the long term--perhaps they even think they can dominate it. This is a big difference from the other players: IBM isn't bothering to throw POWER at it, and AMD/ATI is only present on older machines; ATI in particular seems more interested in going after the mobile space rather than HPC. I don't know what to make of Intel other than they know they're the choice for the non-GPU side and are at the top of their game.
One problem I see is that NVIDIA is still a fabless house and has performance limitations tied to whatever fab they partner with; perhaps this is why they downplay process gains in the blog post.
Of course, if the conspiracy theorists are to be believed, NSA and friends already have this 10-years-into-the-future technology...
Re: (Score:3)
Of course, if the conspiracy theorists are to be believed, NSA and friends already have this 10-years-into-the-future technology...
I heard 20 years - they're still learning stuff from the Roswell crash.
Re: (Score:2)
Of course, if the conspiracy theorists are to be believed, NSA and friends already have this 10-years-into-the-future technology...
With a nearly unlimited budget, no need to sell a product or make a profit, some of the best and brightest talent in the world (they especially like math majors), and the ability to spy on and thus learn from nearly anyone ... well, they'd be pretty damned incompetent if they somehow aren't ahead of the mainstream. Make no mistake, "national security" is a very high-stakes game, these are people who play to win, and "winning" means superiority.
That is a conspiracy theory? Usually those involve aliens o
Re: (Score:1)
Xeon Phi vs GPU (Score:2)
The advantage of Xeon Phi cards is that parallelization on those cards works similar like classical parallelization on supercomputers.
Not really, no. Classic supercomputers were vector machines whereas Xeon Phi is wide SIMD.
You just use MPI
MPI is equally applicable to GPU or Xeon Phi, it operates at a level above the raw computation. In both cases you have controlling CPUs with accelerators attached (GPU in once case, Xeon Phi in the other). MPI is used to manage the data flow between these units but has little to do with the architecture of those units themselves.
For GPUs, on the other hand, you have to adapt a lot of code.
You have to adapt code either way:
For GPU you express the problem as a scalar kernel that i
Re: (Score:3)
Actually Intel is pretty much the king of the hill at the moment for HPC. They don't have a "GPU" solution, but they do have a massively parallel CPU + PCIe compute card available called the "Xeon Phi". Extremely confusing, yet this is what the current fastest supercomputer uses
http://www.datacenterdynamics.com/focus/archive/2013/06/xeon-phi-powered-supercomputer-tops-top500
Xeon phi is easier to deal with than Nvidia's solution for GPU, essentially because it is currently much easier to program.
http://gopar
2 gigawatts... (Score:1)
Hmm, Mr. Fusion is due in a couple of years...
chaining cellphone CPUs (Score:2)
Imagine (Score:1)
Why (Score:2)
Does anyone have an idea of what these extremely expensive systems are even for? And don't say password cracking/NSA, because both of those tasks are "embarrassingly parallel", so that you can use a cloud of separate computers rather than a tightly interlinked network like a supercomputer.
Are there real world problems right now where another 100x more CPU power would make real, practical differences? (versus making the algorithm more efficient, etc)
Re: (Score:2)
CFD simulation. Lattice Boltzmann [wikipedia.org] simulations of fluid dynamics is one such application. Folks at the various DOE national laboratories have a pretty keen interest in this kind of simulation.
Re: (Score:2)
Exascale computers would be helpful for climate modeling. Right now climate models don't have the same resolution as weather models, because they need to be run for much longer periods of time. This means that they don't have the resolution to simulate clouds directly, and resort to average statistical approximations of cloud behavior. This is a big bottleneck in improving the accuracy of climate models. They're just now moving from 100 km to 10 km resolution for short simulations. With exascale they c
yea but (Score:2)
we all know Chinese numbers represent a value exactly 14% less than what the rest of the world agrees on.
Moore's law. (Score:2)
Moore's law predicts that the "factor-of-33" will be bridged in about 10 years. There is only a factor of 20 to the "peak performance", so about a year before that, peak performance might topple the exabyte "barrier".
(Some people plug in different constants in Moore's law. I use factor-of-1000 for every 20 years. That's 30 every 10, 2 every 2, and about 5 every five. This has never failed me: it always works out).
It has slown down (Score:2)
A more pessimistic estimate would say Moore's law only gets you a doubling every 3 years nowadays, so a factor of 32 would take 15 years to work out. See the troubles there were for e.g. TSMC moving to 28nm, and now 20nm.
An exaflops supercomputer would still be possible, with a 10x boost from Moore's law over 10 years and building a 3x bigger supercomputer.
Re: (Score:2)
I think the DOE was predicting last year that their first exascale system will come online in 7 to 9 years.
This isn't going to happen for awhile. (Score:2)
We're at 5.4% of exaflop scale. Somehow I don't think this is a 2013 / 2014 goal ;)
"Some developers" make ridiculous predictions (Score:2)
Some developers predicted that China's new Tianhe-2 supercomputer would be the first to break through.
Wait... *what* uninformed developer(s) predicted that? The previous record (six months ago) was set by Titan, at 17.59 Petaflop/s. So to pass the exaflop barrier this time around would require over a fifty-fold improvement -- something never before seen in the history of the Top500 list. Did someone *really* make this prediction, or is author Kevin Fogarty just making shit up?
Oracle already did (Score:2)