Low Energy Supercomputing 159
Faith Singer at TACC writes "The term 'supercomputing' usually evokes images of large, expensive computer systems that calculate unfathomable algorithms and run on enough energy to support a small city. Now, imagine a supercomputer, but run on the electrical equivalent of three standard-size coffee-makers. This year's international supercomputing conference, SC10, will feature the Student Cluster Competition that challenges students to build, maintain, and run the most-cutting edge, commercially available high-performance computing (HPC) architectures on just 26 amps."
Amps = current, not energy.... (Score:5, Informative)
Amps = current, not energy....
Re:Amps = current, not energy.... (Score:5, Informative)
P = (26 A)(100E6 V) = 2.6GW, more than twice the amount of power required to travel from 1985 to 1955 or vice versa.
And energy is measured in joules, not amperes...
Re: (Score:2)
Informative? Modders on drugs? I would understand Funny.
I was captain of the team that won last year... (Score:5, Informative)
Links to more info from the conference: SC10 CC Page [supercomputing.org], rules [supercomputing.org], and app list [supercomputing.org].
The competition is harder than it sounds, you have to build a cluster from the ground up, fit it into the power requirement (which means stripping out redundancies among other things), strip down a distro (we used Debian as a starting point), get the apps optimized, and then run through the data sets. Your team needs to *understand* the apps, the OS, and the hardware in order to win. There are several people from various teams from past years who have moved on to doing their PhDs in comp sci based on work from this competition (At Carnegie Mellon, MIT, and UMich off the top of my head).
It's important too, in a few ways. For one I know I learned more about clusters the first day I started working on the team for this competition back in 2007 than I ever knew before. That knowledge has led to research fellowships, jobs, and knowledge of what I want to (biochemical modeling). It's an experiance that very few undergrads get, and I think that's a shame.
For the industry it's an important highlight of what can be done with a lot of dedication and a focus on wringing the most from your hardware and software. and in doing that we showcase a lot of work that people dont think about. For example our cluster last year ran off a single disk, plus a large ramdisk as scratch exported over QDR infiniband to the compute nodes. No, it's not new, but it was novel to a lot of people who dropped by our booth.
For another, the ASU team was the first time *I* and many others ever saw a windows cluster in the wild.
Competitions like this are important, they showcase technology and introduce it to undergrads early, with positive benefit!
Re: (Score:3, Funny)
Resistance is futile.
Times voltage times session time (Score:5, Informative)
Current * voltage = power. In the United States, alternating current from the wall is nominally 115 volts, and 115 V * 26 A = 2990 W. So I think the actual figure was supposed to be 3 kW of power. Run this for one eight-hour day* for 24 kJ of energy per session.
* This can be business hours (if interactive) or the most efficient cooling hours (if batch).
Re: (Score:2)
Any hour is an efficient cooling hour if you're using geothermal for cooling your cluster ;)
Re: (Score:2)
Energy from the wall can be one of several voltages. 120V or 240V for single phase or 440V for three phase power, all of which are available in your typical commercial building.
Re: (Score:3, Interesting)
And nominally, all the power of a 486! (Actually, it supported 12 users doing data entry.) Not what I called a super computer, even then.
I think my phone outperforms it in proportion to its 16G or memory: 12,000 times more powerful - Now THAT is what I call a super computer!
Re: (Score:2)
Not all buildings get all voltages. Typically a building gets a single "service voltage", which could be (in the USA) 120/240 volts for single phase, 120/208 volts for three phase, or 120/240 volt three phase using a depricated delta wired transformer bank, or 277/480 volt three phase. Canada has a lot of 347/600 volt three phase around (and also a few places in the USA). And in a few places you can get 138.5/240 or 240/416 in three phase. In many places, street lights run on 480 volt single phase or th
Re: (Score:2)
Mod AC up. Without a power factor, there's no way to determine the actual wattage consumed knowing just the RMS voltage and current.
26 amps at 110V 1 phase is a lot different than say 26 amps at 7200V 3 phase. The summary at least is pretty non-specific, although the article that I didn't fucking read probably was a tad more specific.
Re: (Score:3, Informative)
How many computers do you know that plug into 7200V 3-phase outlets? It's pretty standard when plugging computers into power-limited circuits to measure the allocated power by amperage. That's how power allocations are quoted at colocation facilities, for example: you rent a rack with a 20-amp circuit, or with a 30-amp circuit, or whatever. Unless stated otherwise, in North America it's implied that the circuit is at 110-V.
Re: (Score:2)
ones trying to use a lot of power with few amps, and that have a massive transformer in them....
Re: (Score:3, Insightful)
*Cough* Um, unless you're an embedded programmer. Then, you're expected to fix hardware guy's mistakes by making changes to the firmware.
Re: (Score:3, Insightful)
You have variable voltage outlets?
Re: (Score:2)
If you're very careful, you can with a 240V outlet, at least the way we do things in the US.
Re: (Score:2)
It's an amazing feature my power company provides. That alternating sag/surge/sag/surge really shows management why they can't slash my power conditioner/UPS budget.
Re: (Score:2)
Get a transformer that steps the voltage to 155 volts. Then you'd be logarithmically right in the middle of the 100-240 volt range most PSUs can handle. That would cover a rather wide range of sags and swells (surge is a transient ... swell is the term for a longer term rise in voltage).
Re: (Score:3, Funny)
Mine's always alternating...
Re: (Score:2)
Oops forgot the <sarcasm> tag again.
Re: (Score:2)
It makes me wonder if they can get away with running on a higher voltage for more power..
Re: (Score:2)
Re: (Score:2)
Hey, don't tell them that an ampre isn't a measure of energy, because then I wouldn't be able to enter my 46.54 million volt computer (x 26 amps = 1.21 Gigawatts total power dissipation).
Re: (Score:2)
Loophole.
Give me those amps at 130 KV, please.
I'll supply the step-down transformer.
Sure Thing! (Score:5, Insightful)
Re: (Score:2)
Can I use as many volts as I'd like?
Inquiring minds want to know! Lets see, the most we could probably get into the building is 32KV, at 26 amps on a 3 phase line we can use 2.4 MW before crossing their limit. That should be enough for a little bit of supercomputing.
Re: (Score:2)
Am I allowed to use a transformer to boost the voltage before it gets counted as 26 amps?
Mmm... caffeine. (Score:5, Funny)
Re: (Score:2)
Coincidentally, the number of coffeemakers on this floor is...three.
Re: (Score:2)
Meh. At a job a couple of years ago we had free Espresso. Now that'll get you cranking.
Amps? [pedant] (Score:3, Insightful)
Re: (Score:2)
You can also save on energy by just turning the computers off when you don't need them. That's why we got remote control PDUs on ours.
Re: (Score:2)
I hope they're smarter than the article writer (Score:4, Insightful)
"The competition challenges students to build, maintain, and run a cutting edge, commercially available HPC architecture on just 26 amps of energy."
Only problem is that the Ampere is a unit of CURRENT, not energy. It's like saying someone weighs 686 Newtons.
While I understand that if the voltage is kept the same, then the amps are proportional to the energy involved per unit time because W = V x A. However 26 amps at 120 volts for 1 second is not the same energy as 26 amps at 5 million volts for 20 years.
Re: (Score:2, Redundant)
Only problem is that the Ampere is a unit of CURRENT, not energy. It's like saying someone weighs 686 Newtons.
Wait... what? Newtons are a unit of force, weight is force due to gravity. Maybe you meant that it's like saying something weighs X kg or something masses X Newtons?
Re: (Score:2)
Re:I hope they're smarter than the article writer (Score:4, Informative)
It's like saying someone weighs 686 Newtons.
Nope, because that would be correct (other than capitalization). Newtons are a unit of force, and weight is the force of gravity on a thing.
It's not even like saying someone weights 100 kg, which is conventional but wrong: weight is not mass, weight is mass*acceleration.
It's more like saying someone weights 150 m/s. That is, it makes no sense whatsoever without supplying some additional information. If I had an instrument that accelerated objects under a constant force for a fixed time I could get a value in m/s that would allow me to compute the weight if I had the force and the time, in the same way it would be possible to get the energy from the amperage if you had the voltage and the time.
Re: (Score:2)
It's more like saying someone's mass is 686 N. You can guess the correct answer if you make a reasonable assumption, but the statement's wrong. (As a minor difference, the acceleration due to gravity is much less variable than the voltage supplied to electrical devices.)
Re: (Score:2)
If you're still unsatisfied, just ask Wikipedia: http://en.wikipedia.org/wiki/Weight [wikipedia.org]
Re: (Score:2)
It's exactly like saying someone weighs 100 kg. It's technically nonsensical but commonly done. And it does make sense if you have one extra bit of information (100 kg in a gravitational field of 9.81 m/s or 26 amps of 110 V AC).
Re:I hope they're smarter than the article writer (Score:4, Insightful)
Yup. Just to add some international touch to it: here in central Europe we have 240-250V outlets, which is radically different from the U.S., so putting amperage even with implied voltage is at least confusing and entirely unscientific and shows a lack of understanding about even the most basic principles of unit notations.
Re: (Score:2)
Even just in the US, its common to have both 110V and 220V (or is it 120V and 240V? I've seen both numbers used and I remember once reading why, but I don't remember and its mostly irrele
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Don't even get me started on the 50Hz thing.
don't take it personally, this whole coffee maker analogy really pisses me off.
Stick to cars people,CARS! Slashdot=CARS sheesh!
Re: (Score:2)
It's like saying someone weighs 686 Newtons.
I just felt a disturbance, as if a million swooshes went by at once...
Re: (Score:2)
It's like saying someone weighs 686 Newtons.
Well, a newton is about 0.53 grams (30 Newtons per 16 oz package) so 686 Newtons would be (686 * 0.53)/16 ~ 23 pounds? It's kinda light, but doable for a small child...
What's that? You didn't mean Fig Newtons?
(sigh)
Re:I hope they're smarter than the article writer (Score:4, Insightful)
The only problem with your analogy is that in physics, Newtons are actually a unit for weight
You're right and I saw that after I hit the submit button. It would work better had I said mass :)
As for the power source - my home computer's power supply provides 24 volts to the motherboard. My laptop uses 19 volts. Potential at the plug socket is not necessarily the potential that is used by the machine. Especially since electronics usually convert AC to DC and use DC in their circuits. Therefore I would argue that your "assumption" is incorrect - there's no way to know what voltage they plan on using. It can't be that hard to imagine 240V or even 550V for an "industrial strength" supercomputer. Why assume 110V?
Re: (Score:2, Informative)
Why assume 110V?
Because as stated before, 110 times 26 ~ 3 kW. And a coffee maker is about 1 kW, so three coffee makers is 3 kW, in agreement with the summary.
Student Energy Units (Score:3, Funny)
This is a student project so the correct unit of energy is a "Library of Congress Stacked with Red Bull Instead of Books."
Now, you may convert that into Joules, if you care to.
Re: (Score:2)
How much is that in electron volts?
Mountain Dew (Score:2)
Well.... (Score:4, Insightful)
This doesn't sound too difficult. The number one power-consumer is cooling. Distributing the same code over a larger surface area would allow you to reduce just how sophisticated and power-hungry your cooling needs to be. Any SIMD code will distribute just fine over such an architecture. If you're really clever, you'd design the cluster as a series of pentagons and hexagons, allowing you to build a geodesic. This would not only maximize the surface area but would also minimize the distance network traffic has to travel, networking being the biggest cause of latency in supercomputing. The really really clever geeks would then set up additional "regional" networks to allow for much higher performance when handling code that needed to talk much more locally, then distribute the code according to those regions. (Essentially, you then have a cluster of clusters.)
Re: (Score:2)
I for one, welcome a Beowolf cluster of Pentagons. Yes, Siree!
Re: (Score:2)
You never know, the contention for resources might keep them all out of trouble.
The limit is 26 amps @ 120VAC (Score:5, Informative)
The computational hardware (processors, switch, storage, etc.) must fit into a single rack. All components associated with the system, and access to it, must be powered through the two 120-volt, 20-amp circuits, (each with a soft limit of 13 amps) for a total of 26 amps, provided by the conference. Power to each system will be provided via metered power distribution units The equipment rack must be able to physically hold these metering power strips.
This makes it even harder since theyir hardware has to be power balanced between the two power strips. They'll have to come up with some dynamic load balancing between cluster nodes based on power consumption. I guess dual power supplies might help (do dual power supplies draw perfectly balanced power between both power inputs?), but at a loss of power efficiency.
Re: (Score:2)
It would have made more sense to power it at 240 volts. It would have been in balance at any number of loads, and actually reduced the current somewhat as most PSUs run more efficiently at 240 volts than at 120 volts.
Re: (Score:2)
Repeat post (Score:2)
*Post complaining about amps not being a unit of energy*
*Feeling smug*
*Actually reading thread, then feeling not so smart because 10 people have already mentioned it*
OK, now that the Amps/Watts thing has been sorted, (Score:2)
actually, it wouldn't trip your breaker(s) (Score:2)
You'll notice they specified two 120v outlets of 13A each. You could easily run that on two standard home outlets, as long as they were on different circuits.
Re: (Score:2)
Or just put them on ONE circuit with 240 volts.
Let's put CPU's in electric heaters. (Score:3, Insightful)
I've always found electric heaters (including geysers, etc. but mostly environmental heating) a huge waste of low entropy. You can achieve the same goal by powering enough chips -- would work especially well for floor heating. Now, if you're not recycling old computers, it might cost some, but if our only constraint is energy, we can thus create a supercomputer that spends 0 energy "for itself", just by installing this system to a few buildings.
You could even communicate through the power line, thus eliminating the need for a separate network installation. "Buy our @home geyser, that pays for itself!", that sort of thing...
PPFFFT Coffee powered (Score:2)
Re: (Score:2)
Power consumption is a critical performance metric for large-scale computing systems.
The whole ponit here is to prove you can do more with a limited resource where that resource is something that computing systems managers are desirous of reducing because it gives them the right to claim all sorts of greenish certification bonads from the marketosphere, and improve their bottom line for the investosphere.
Re: (Score:2)
Sorry, that's ponit[TM].
If watts = volts times amps... (Score:2)
then this means on a 120 volt system, we're talking 3,210 watts to run the thing. That's about 3 kilowatts.
Besides the [P] != A problem (Score:2)
Do they write anywhere how much of the superconducting architecture they have? Number of cores? Flops? Mips? Anything?
Heck, i can build any architecture with a few watts if i am allowed to underclock and only use 4 cores to demonstrate it.
But well.... It was in Texas... Down where hillbilly creationists roam in the educational boards. Probably they where thought in school to strip any experiment of units and sensible numbers and replace these but general claims, so that all theories are equally scientific.
More voltage, please (Score:2)
Most switching mode power supplies work more efficiently at the higher AC voltage of 240 volts. So, what draws 26 amps at 120 volts should typically draw less than 13 amps at 240 volts. You can thus run more than twice as much computer capacity on the same wiring gauge, which means additional savings in energy loss through the wiring feeding to the computers.
Most homes in the USA have 240 volts. If you are going to put a lot of computers in a room, wire up a 240 volt circuit for it. Just be sure to use doub
Stupid contest (Score:2)
Ok, so i'm sure a lot of students learn a lot, but after reading how this contest is run it seems like total BS.
First, with a power budget that small, building a cluster isn't even worth it. A large SMP machine will dominate on any bandwidth limited benchmark. A machine with GPU's is going to dominate any compute bound application. An 8 socket machine (aka DL980G7) packed out with low power 6-8 core processors is going to provide 96-128 threads. Load it up with SSD's, and Tesla's and your probably going to
Re: (Score:3, Insightful)
ARM would get you better performance per watt. Atoms only matter because they're x86.
Re: (Score:2, Insightful)
See, this is why some people are sticklers for grammar, spelling, and capitalization rules. Of course they're using atoms. But are they using Atoms? Your error is needlessly confusing and detracts from your point.
Re: (Score:2, Informative)
See, this is why some people are sticklers for grammar, spelling, and capitalization rules. Of course they're using atoms. But are they using Atoms? Your error is needlessly confusing and detracts from your point.
If I had capitalized Atom do you think anyone would have thought I meant the car?
Or the Christian martyr?
Or the unit of medieval time?
Or the 1983 educational game for the TRS-80?
Or the comic book?
Or the alias of German musician Uwe Schmidt?
Or the album by Carbon/Silicon?
Or the Acorn Atom, an early 1980s home computer?
Or the card game?
Or the sports teams of Annandale High School?
Or the XML-based Web syndication format?
It could mean ANYTHING! Oh how will anyone deduce the meaning of my convoluted sentence??!
Re:Article Doesnt Say (Score:4, Interesting)
Having participated in the first of the Student Cluster Challenges at SC07 when I was still in undergrad, I can attest that there's far more to this than what the summary lets on. Not only are you limited to 26 Amps, which is the significant limiting factor, but you're located on the show floor and running your system for 36 hours straight in front of the conference attendees. Moreover, all hardware must be in production and unmodified and fit within a single rack. The Taiwanese team lucked out in this regard as they were using the (then new) 45nm Intel Xeons that were announced the day before the competition started. The only thing you can modify is the code for the programs you have to run (except for the HPC benchmarks).
Some of you might be thinking "pfff...I can stay awake for 36 hours, no problem". That's true, but you're not allowed to be in your booth for more than 12 hours straight and after you leave you must take an 8 hour break. Furthermore, the machines are firewalled from all incoming connections and do not share the same internet connection that the rest of the conference uses.
At SC07, there was a significant power failure on the second day of the competition which brought most teams to their knees. The applications we were running (GAMESS [wikipedia.org], POP [lanl.gov], POV-RAY [povray.org]) are not designed to pick up from a power failure. While the Taiwanese had by far the most powerful system, they couldn't recover from the power failure that had corrupted their SAN in time to win.
To your point, I'm not sure you could get 158 Atoms in a set of off-the-shelf servers that would fit in a single rack to equal a cluster running the latest E series Xeons that perform at top clock but have a lower TDP.
Re: (Score:3, Interesting)
Re: (Score:2)
The point of the "unmodified, in production" simply means that you have to use off-the-shelf equipment. As the organizers put it, if someone wanted to walk up and buy your cluster, or an exact copy, they should be able to.
You're right that it's hard to fill a full rack, even with 4U servers that use 120mm fans, but I believe Stony Brook came the closest because they were using ULV Xeons. They had 16 servers I think and each was 4U
As far as fun, it was incredible and very stressful
Re: (Score:3, Insightful)
Moreover, all hardware must be in production and unmodified
That's an odd requirement. IMO a team that could design and build their own hardware that's more efficient than off the shelf hardware should be encouraged to do so.
Re: (Score:2)
Re: (Score:2)
Moreover, all hardware must be in production and unmodified
That's an odd requirement. IMO a team that could design and build their own hardware that's more efficient than off the shelf hardware should be encouraged to do so.
True, but the point of the competition isn't to show off the vendor, or even the hardware; it's to show off how easy it is for undergraduate students, using commodity hardware, to construct and run a cluster that has more computing power than the fastest supercomputers from a decade ago at a fraction of the TCO.
Re: (Score:2)
True, but the point of the competition isn't to show off the vendor, or even the hardware; it's to show off how easy it is for undergraduate students, using commodity hardware, to construct and run a cluster that has more computing power than the fastest supercomputers from a decade ago at a fraction of the TCO.
If that's the goal, then why isn't cost of hardware included as part of the submission criteria? And really, what is "commercial hardware"? If I can cram 36 Pico-ITX store bought motherboards into a 1U storebought case using standard power supply hardware connected with standard off the shelf wire, does that count as commercial off the shelf? Or does "commercial" mean I have to buy a complete pre-built server from an established vendor?
Re: (Score:2)
Your 36 Pico-ITX's wouldn't qualify. Teams are paired with vendors and some schools are paired together, for instance my university was paired with Technische Universität Dresden in 2008. You're allowed to customize, within reason, what comes in your server, but usually it's a system put together by the vendor and not purchased by the university because that's a considerable investment for a system that isn't anywhere near as powerful as current Top500 systems.
For example, in 2007, we had 9 Apple xSe
Re: (Score:2)
That makes this contest less interesting. It seems like the team with the best vendor contacts wins, regardless of how "smart" they are.
If team A's vendor is willing to pay $4,000 each for the top of the line 8 core Xeon's with 10Gig ethernet interconnects, but team B's vendor only supports 4 core Xeon's and only supplies GigE network, then it seems like team B loses no matter what they do.
Or are they judged on something besides raw performance?
Re: (Score:2)
Re: (Score:2)
Regardless of whether or not Xeon's offer better performance per watt, there has to be a point where more money buys you better performing hardware per watt, so it still seems that the contest boils down to which team has the vendor with the most $$ or best performing hardware.
I can't find past results online, do they give prices for the systems in the contest results?
Re: (Score:2)
Last year for stony brook we had Dell, AMD, a
Re: (Score:2)
I don't have links to results, but I was there for the first one and helped implement the second one. Prices are compiled by the teams but usually not in the press releases
My guess is systems usually run in the $100K to $150K range. The fastest system in 2007 was from Tiawan made by Asus with prototype 45nm Xeons (released the day before the competition).
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
That power outage was *awful* for us, we were going with the strategy we used last again of a large ram based communal scratch.. the outage *really* didnt help there
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
2007: University of Alberta, CA [theregister.co.uk]
2008: Indiana University/TU Dresden (Germany) combined team [iu.edu]
2009: Stony Brook University (State University of New York) [stonybrook.edu]
Re: (Score:2)
Depending on what you're doing, a Xeon LV would be better; and, again depending on the workload, the 12-core Opterons might win out, too.
Of course, those have higher non-CPU system power requirements, so an Atom's whole-system power requirements might win out.
But, as someone else mentions, ARM has excellent SoC power usage for the performance, so a massive array of ARM A9s could very easily smoke anything else.
Re:Give them Watts, not Amps. (Score:4, Funny)
Re: (Score:2)
Does he like beans?
Re: (Score:2)
> I expect better from Slashdot editors...
New here, aren't you?
Re: (Score:2)
If we are into nitpicking, Volt (singular!) is a unit of electric potential (energy/charge); Joule (Volt*Coulomb = Newton*Meter = ...) is a unit of (potential and all other forms of) energy.
Re: (Score:2)
this is a false comparison.
Unfortunately since their is no single accepted definition of what a supercomputer is, it is easy to lump the two together.
Re: (Score:2)