Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

A Closer Look At Immersion Cooling For the Data Center

timothy posted more than 3 years ago | from the where's-the-vinegar dept.

Data Storage 213

1sockchuck writes "Want to save money on data center cooling? Tip your racks on their side, fill them with mineral oil, and submerge your servers. Austin startup Green Revoluton Cooling first profiled here) has a video demo of its immersion cooling solution, which it says can handle racks using up to 100kW of power. A photo gallery on the company web site shows some early installations."

cancel ×

213 comments

Sorry! There are no comments related to the filter you selected.

This is (-1)

Anonymous Coward | more than 3 years ago | (#35805598)

Advertising Crap. Also that photogallery is annoying.

Mineral oil = nightmare (0)

haxor45 (2040278) | more than 3 years ago | (#35805614)

Didn't we discuss that before? One guy tried to use it on his rig [freeblogspot.org] (Link to his blog). It did work cool initially (no pun intended), but later it ruined his system as oil stated to leak. I bet they will run into same problems. (Imagine oil leaking out of Ethernet port...)

Re:Mineral oil = nightmare (1)

slim (1652) | more than 3 years ago | (#35805638)

Your link doesn't work, but I imagine it was a hobbyist. It looks as if this lot have built an industrial-strength product. It's clearly not practical in the home.

The entire computer is submerged in a bath of oil, and the oil is circulated through a cooling tower. I doubt there are any holes for IO below the oil level of the bath, so leakage isn't a concern.

I don't fancy the messy job of making hardware changes though.

Re:Mineral oil = nightmare (0)

Anonymous Coward | more than 3 years ago | (#35805696)

Leakage is definitely a concern, because mineral oil creeps along capillaries. Additionally, it has the nasty tendency to dissolve some plastics over time.
Of course, with careful parts selection you can work around some of these problems, and it is not as bad for servers as for workstations (no need to deal with mice/keyboard/whatever), but still...

But worth a try. I just would not bet (much) money on it.

Re:Mineral oil = nightmare (5, Informative)

PopeRatzo (965947) | more than 3 years ago | (#35806176)

Leakage is definitely a concern, because mineral oil creeps along capillaries.

I once drank half a bottle of mineral oil, and let me tell you, leakage was definitely a concern.

Re:Mineral oil = nightmare (3, Interesting)

GameboyRMH (1153867) | more than 3 years ago | (#35806550)

Additionally, it has the nasty tendency to dissolve some plastics over time.

From what I understand, this has been the main problem with immersion cooling. Mineral oil softens PCBs.

Re:Mineral oil = nightmare (2)

smpoole7 (1467717) | more than 3 years ago | (#35805938)

> I don't fancy the messy job of making hardware changes though.

THIS. If it's made by man, it will eventually fail and will require service or replacement.

The cost in labor (and cleanup!!!) (and replacement oil!) (and trips to the emergency room for employees who slipped and fell in the oil on the floor!) (not to mention the lawsuits) make this a supremely dumb idea. Now add in the cost of the hermetically-sealed rack(s) and it would be difficult to imagine a dumber idea.

Google is pretty innovative about stuff like this. They use their own in-house version of Linux on commodity hardware, thousands upon thousands of PCs in each data center. But they still use air cooling and air conditioning because, at the end of the day, it's the best bang for the buck. You would save more money by using more efficient equipment, I think -- as you replace each unit, just try to find one that's a little "greener." Or, consolidate servers and use virtualization. Be creative. But DON'T seal the danged thing in a rack filled with oil!

Now, if someone would design a liquid-cooled rack mount computer with coolant connections, you might could make an argument for that. Run hoses and put the radiators and fans on the side of the building in the shade, maybe. But I still don't see how it would be cost-effective, and the square footage required for the hoses and radiators (or whatever else you plan to use) will hardly improve the expense.

Not that I'm opposed to liquid cooling, per se. Continental makes a liquid-cooled 30KW FM tube transmitter and it's nice and quiet. My big transmitters still use big tubes and require huge blowers. They're so loud you can't use the phone in the building, and if I'm doing PM and have both the main and auxiliary transmitters on at the same time, it's so loud you can hardly think. As the prices on high-powered solid-state transmitters continue to drop, we're replacing the older tube units. Now THAT'S cost-effective.

Just my opinion, but if the boss ever told me to fill our racks with mineral oil, I'd take him out to dinner and have a LONG talk with him. (Not that he'd ever propose something so foolish. We have enough trouble with leakage on our hermetically-sealed, oil-filled dummy loads.)

Re:Mineral oil = nightmare (1)

DigiShaman (671371) | more than 3 years ago | (#35806534)

With a traditional rack (air cooled), your servers will build up heat in the event of an HVAC failure. But at least you have ample time to start a controlled server shutdown sequence. But with liquid cooling, I would imagine the heat buildup would happen very very quickly in the event of a pump failure. You want to talk about a mini-thermal meltdown, 100kW of heat will do that.

Re:Mineral oil = nightmare (2)

AstroMatt (1594081) | more than 3 years ago | (#35806602)

Oil will have a far higher heat capacity than air, so the heating rate after pump failure would be far slower than after an air handler failure.

Re:Mineral oil = nightmare (1)

slim (1652) | more than 3 years ago | (#35806624)

I would imagine -- and someone else has surely done the sums -- that the oil would have enough heat capacity, and be kept at a low enough standard temperature, that it could absorb enough heat without the pump running that a controlled shutdown could be done.

But it's not beyond the wit of man to have a standby pump.

Re:Mineral oil = nightmare (0)

Anonymous Coward | more than 3 years ago | (#35805640)

Error 404 - Not Found
Sorry, the page that you are looking for does not exist.

Re:Mineral oil = nightmare (2)

haxor45 (2040278) | more than 3 years ago | (#35805656)

Sorry, the link is supposed to be goatse, but they took down the blog I used for this :-(
Next time folks, today isn't my day
Anyway, the post is actually correct, I did read such an article sometime ago.
Don't remember the link unfortunelly

Re:Mineral oil = nightmare (0)

Anonymous Coward | more than 3 years ago | (#35805690)

No soup for you.

Re:Mineral oil = nightmare (1)

migla (1099771) | more than 3 years ago | (#35805718)

That's ok. I can live without it.

Re:Mineral oil = nightmare (1)

smpoole7 (1467717) | more than 3 years ago | (#35806170)

I honestly wonder if it's an April Fool's joke.

Re:Mineral oil = nightmare (2)

Inzite (472846) | more than 3 years ago | (#35805842)

There's a good play-by-play report of a hobbyist's adventures in mineral oil cooling here [bit-tech.net] . The first page is just an introduction, but contains links to all the juicy bits on successive pages.

Sorry, no goatse.

Re:Mineral oil = nightmare (1)

smpoole7 (1467717) | more than 3 years ago | (#35806192)

And one other thing, for any of you hobbyists out there who plan to try this.

Heat generally rises, but remember that many computers are designed so that the cooling fans force the air *horizontally* across the components. Case in point: my Dell Poweredges. They have a bank of fans near the front that force air "sideways" across the CPU, RAM and other heat-producing parts.

Unless you put in some sort of coolant pump to circulate that oil (or water, or whatever you plan to use), or *replace* the heatsinks with sealed, carefully-designed systems that pull out the heat and throw it elsewhere, you will probably destroy your computer in the long run.

In a word: if you try something like this, be careful. Use an infrared thermometer to confirm that you don't have hot spots.

Marketing that niche? (1)

pieisgood (841871) | more than 3 years ago | (#35805632)

The thing here is they are commercializing a cooling technique usually reserved for the hobbyist. I don't know about the energy saving claims, but their setup looks fairly organized. Interesting turn for a still niche cooling solution.

Re:Marketing that niche? (0)

Anonymous Coward | more than 3 years ago | (#35806136)

Many supercomputers have been doing this for decades. It's hardly reserved for the hobbyist.

Liquid Cooling in Military Applications (0)

Anonymous Coward | more than 3 years ago | (#35805646)

The thought of liquid cooling is nothing new. I know the us military uses water cooling aboard submarines for their server racks. In that app, however, they're pump water into those racks then use a heat exchangers to keep the equipment cool in an sealed rack. I'm a little shocked that using mineral oil really improves that power density THAT much for a massive data center. It just feels like cooling that much mineral oil can't be as efficient as cooling air.

Re:Liquid Cooling in Military Applications (2)

AlecC (512609) | more than 3 years ago | (#35805834)

I would expect oil to be far more efficient than air. It has a hugely greater thermal capacity (hundreds of times), so it can extract much more heat from the chips and similarly hand it over better to the cooling vanes. You use thermal paste to connect the chip to heat sinks better than air - this is a larger scale version of the same thing, where the whole system is immersed in a sort of thermal paste.

Re:Liquid Cooling in Military Applications (2)

Linuxmonger (921470) | more than 3 years ago | (#35806276)

You're right, it takes a lot of energy to heat up that much oil, that's the point. It doesn't take much energy to heat up a cubic foot of air, it takes a lot more to heat up a cubic foot of oil, the effort here is to remove as much of the energy that would go into melting chips as is possible.
Liquid to air heat exchangers can be made as big as you need pretty cheaply, it's easier to pump 800 cubic feet of air through a big radiator cooling one cubic foot of oil than it is to pump 800 cubic feet of air through an OEM heat-sink.

What about the hard drives? (2)

Compaqt (1758360) | more than 3 years ago | (#35805652)

Well, on the other hand, if they're supposed to be air-tight, I guess they're baby oil-tight, too.

But there's got to be something or another that doesn't react well with mineral oil, right?

I guess this means they save on fans, and the power to run fans. That's additional power and heat savings right there.

OK, I've got it: what about the CD/DVD drives? Or is it all network IPL in data centers? I'm racking my brains trying to think of something this would mess up.

Re:What about the hard drives? (1)

slim (1652) | more than 3 years ago | (#35805702)

TFA says the hard drives have to be sprayed with a coating, presumably to make the housing oil-tight as well as airtight.

Blowing air around is a tremendously inefficient way of cooling, and this replaces that with pumping the oil through a heat exchanger / cooling tower.

Things with unsealed moving parts obviously are vulnerable, but not everything needs to be submerged. If you really want a DVD drive, have it outside the oil.

There may be some things that don't react well with mineral oil: avoid having these in your system!

Re:What about the hard drives? (2)

petermgreen (876956) | more than 3 years ago | (#35805730)

TFA says the hard drives have to be sprayed with a coating, presumably to make the housing oil-tight as well as airtight.

This should IMMEDIATELY ring alarm bells. Hard drives are NOT airtight. They have a filtered air hole. They would never get away with such flimsy construction on an airtight product.

Plus does this system REALLY offer that much advantage over conventional "waterblocks" which keep the cooling fluid seperate from the electronics. I very much doubt it. The major heat generators in a PC are designed to pass out their heat by contact conduction anyway.

Re:What about the hard drives? (3)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805762)

I suspect that the main advantage over waterblocks is that your server vendor of choice doesn't have to cooperate, and you don't have to run so many small hoses, and can get away with the big huge tank. Now, if some sort of critical mass were achieved in terms of industry acceptance of waterblocks as a factory option(along with some sort of standardized fluid connector), I suspect that that would swiftly become a superior option(though, quite possibly not displacing fans entirely. A fair number of ancillary systems, too small to justify their own waterblocks, get very unhappy if they aren't getting any airflow, even if the CPUs are taken care of...)

Re:What about the hard drives? (1)

admiralranga (2007120) | more than 3 years ago | (#35805868)

A waterblock "plate" that sits flat over the mobo touching the heat generating componets, with groved water channels so the water goes in a loop (cpu first etc), with a water in and out that is connected to the rack, would do the job quite nicely i'd imagine.

Re:What about the hard drives? (1)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805942)

In systems not designed for it, that could get kind of ugly(DIMMs, riser cards, expansion cards parallel to the motherboard, etc) and having to re-CAD the part because your vendor went to rev A01 from rev A00 or they are now shipping memory modules with slightly different sizes would be a huge pain.

I am told, though, that for very harsh environment embedded systems, that is pretty much what they do. Conformal heatsink plate firmly attached to the board so that the whole thing is a solid block.

Re:What about the hard drives? (1)

drinkypoo (153816) | more than 3 years ago | (#35806630)

I am told, though, that for very harsh environment embedded systems, that is pretty much what they do. Conformal heatsink plate firmly attached to the board so that the whole thing is a solid block.

They used to do this on notebooks. I'm ripping apart a HP mobile P4 notebook for embedded use and I found out why it is so damned heavy, it has a single gigantic cast/milled heat sink which crosses CPU and GPU (old Radeon.) My later, bigger HP used heat pipes which actually made it lighter.

Re:What about the hard drives? (1)

GameboyRMH (1153867) | more than 3 years ago | (#35806622)

The other thing that rang my alarm bells is the idiocy of submerging hard drives at all. You DON'T DO IT. It doesn't help, it only causes problems. They should either do what immersion gaming cases do and use long SATA cables to run the hard drives to a dry compartment on the outside of the case, or network-boot these servers.

Re:What about the hard drives? (1)

slim (1652) | more than 3 years ago | (#35806842)

... or use SSD.

But if you're going to have HDD storage anywhere, that's going to be generating heat too. Possibly most of it, if you're doing non-CPU-intensive file serving. So the bean counters would probably *like* their fancy new cooling technology to work with HDDs.

Re:What about the hard drives? (0)

Anonymous Coward | more than 3 years ago | (#35806836)

Most components get their heat out through the leads, except those designed for a heat sink (e.g. CPU, graphics processors, some FPGAs, power devices)
Immersing in oil is a fine, but messy way to get higher density, at the cost of much lower maintainability.

Re:What about the hard drives? (4, Informative)

gclef (96311) | more than 3 years ago | (#35805706)

Actually, hard drives are *not* supposed to be air tight. They intentionally allow airflow into the HD, but through a filter to keep dust out. If you want a drive that is airtight, it'll cost more.

http://www.acsdata.com/how-a-hard-drive-works.htm [acsdata.com]

Re:What about the hard drives? (1)

camperdave (969942) | more than 3 years ago | (#35806438)

This, of course, doesn't apply to solid state hard drives.

ssd are to small for a data center may for booting (1)

Joe The Dragon (967727) | more than 3 years ago | (#35806810)

ssd are to small for a data center maybe for booting with the data on a SAN but that will give off heat.

Re:What about the hard drives? (2)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805742)

I've never tested to see if anything bad happens if you change this yourself; but hard drives are definitely not airtight. Pretty much all of them have one or more visible holes(usually on the top cover, often next to a "Do Not Cover" sticker). There is a fairly serious looking filter on the inside, covering the hole(some sort of fine carbonish powder sealed in what appears to be a teflon pouch); but that is to impede dust, rather than gases.

Nothing stopping you from PXE booting(or iSCSI or fiber channel HBAs on the high end); but I'd be leery of running a hard drive under fluid, except as an experiment.

The only other issue I can imagine might crop up would be discovering the hard way that some polymer used in one of the system's components doesn't handle oil exposure well in the long term. I suspect that most are fine; but if the plasticizer used to soften the insulator coating on some important bundle of wires leaches out over 18 months in a warm oil bath, and the embrittled insulator cracks and shorts the next time you mess with it, the joke would be on you...

Re:What about the hard drives? (3, Interesting)

macs4all (973270) | more than 3 years ago | (#35806138)

The only other issue I can imagine might crop up would be discovering the hard way that some polymer used in one of the system's components doesn't handle oil exposure well in the long term. I suspect that most are fine; but if the plasticizer used to soften the insulator coating on some important bundle of wires leaches out over 18 months in a warm oil bath, and the embrittled insulator cracks and shorts the next time you mess with it, the joke would be on you...

Hard drives aren't the only thing designed with "vent holes".

Every single electrolytic capacitor has a tiny vent hole (to keep them from acting like a mini fragmentation grenade if they develop an internal short circuit, etc.) Over time, with thermal cycling, the oil might get pumped in and out of the vent holes, thus degrading the electrolyte (guessing), and one fine day...

And as you say, think of the insulation on the cables...

Re:What about the hard drives? (2)

Smidge204 (605297) | more than 3 years ago | (#35806392)

Capacitors do not "breathe" like you describe, otherwise humidity would get in and ruin them. Instead they have "vents" in the sense that part of the casing is weakened to rupture safely if the electrolyte starts to break down and build pressure, rather than have the whole can explode.

The irony here is that the electrolyte in the "beer can" style caps is a mineral oil not completely unlike what they use as a coolant. The other option is to pay extra for non electrolytic capacitors on your equipment.

As for the oil degrading plastics and such, I'd like to think they thought of that and either use special wiring where it contacts oil or that the "special formulation" they speak of is designed to prevent that issue. Given that they warranty everything they're probably aware of something so obvious.
=Smidge=

Re:What about the hard drives? (1)

drinkypoo (153816) | more than 3 years ago | (#35806588)

The irony here is that the electrolyte in the "beer can" style caps is a mineral oil not completely unlike what they use as a coolant. The other option is to pay extra for non electrolytic capacitors on your equipment.

Is there a list of manufacturers using solid caps? My Gigabyte board proclaims loudly on the boot screen JAPANESE solid capacitor... whoops. Looks like it's time to find a new supplier. Having sucked up a lungful of blown cap smoke in the past, I'm glad to see solids...

Re:What about the hard drives? (1)

GameboyRMH (1153867) | more than 3 years ago | (#35806710)

My understanding is that if you cover a hard drive's vent hole, unusual atmospheric pressures could cause the disk head to either crash into the platter or float too high over it.

Re:What about the hard drives? (1)

Anonymous Coward | more than 3 years ago | (#35806198)

Perhaps it is time to invest in Solid State for booting and a SAN for storage?

Re:What about the hard drives? (0)

Anonymous Coward | more than 3 years ago | (#35806656)

Sure makes Solid State Drives more appealing.

Problems? (0)

Anonymous Coward | more than 3 years ago | (#35805654)

This has been proposed and tried many times, but I seem to recall that it doesn't work well in the long run, because the oil must be nonconductive, but to be nonconductive it has to be pure. Impurities make the coolant conductive. You also can't have hard disks in there, maintenance is a nightmare and I'd imagine that cables in and out of the tank are a bit problematic too.

Maintenance problems (1)

mangu (126918) | more than 3 years ago | (#35805890)

maintenance is a nightmare

You bet it is! Imagine the mess when you need to replace anything. Not to mention finding the fault. The first thing you do is unplug and plug again the cables just to see if it's just a bad contact problem. Now try to do that when everything is in an oil bath.

Boiling water (1)

dimethylxanthine (946092) | more than 3 years ago | (#35805670)

This could make a lot of hot tea for that staff running the DC... Or CRAC. Or spin some turbines perhaps?..

It certainly looks cool... (1)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805686)

I have to wonder at their claim that it works well with standard OEM gear. Even most cheap consumer shit monitors the speed of at least the CPU fan and tends to freak out if a fan that is supposed to be there is either absent or performing substantially below expected speed(and, given the relatively high stall current of these fans, burning a trace isn't totally out of the question, if the fan or fan controller isn't smart enough to give up after a short time...) Given that high-density servers and blades will be forced to shut down, or cook, within a short time after fan failure, I'm strongly suspecting that a lot of system management cards and firmwares will flip out at you nonstop if the fans are either removed or acting as mineral oil pumps at a few hundred RPM...

I'd also be interested to see the details of how they handle the rack rails. One major advantage of horizontal mounting is that, assuming the bearings aren't completely shot, a single person can easily and safely pull out even some 8U monstrosity. Vertically, you are working against gravity, and your spindly geek arms wouldn't need to be all that spindly to encounter some hilariously expensive dropping-the-coolant-lubed-hardware-with-an-expensive-crunch accidents. Unless the system is specced exclusively for dinky little blades and half-depth 1Us(in which case those massively built coolant ponds with aisles on all sides aren't actually all that dense...), they would have to have something in place to make working with the bigger stuff doable.

Re:It certainly looks cool... (1)

slim (1652) | more than 3 years ago | (#35805734)

It seems pretty trivial to replace the fans with mock fans, that either always reports an OK fan speed, or does something with the measured oil temperature.

True enough about the horizontal mounting and the weight. I don't fancy dealing with a heavy unit dripping with baby oil -- but surely, since they have an installation, they've addressed these practicalities?

Re:It certainly looks cool... (2)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805820)

I imagine that some sort of pneumatic/hydraulic piston system(like the one used on many vertical-open car doors and the like) would be well within the realm of the possible, and offer a zero-apparent-weight slide-up ; but I can't see whether or not there is anything of the sort from the pictures provided.

The one picture showing "easy serviceability" shows the operator having completely unracked a relatively small server. That doesn't scream 'easy' to me. Pull server up, hold with one hand, unclip a bunch of fiddly cables with the other(don't lose them in the oil pit, or you'll have to give the temp a snorkel...) lay it flat, pop the top? Aw hell no. The whole point of slide rails is that you can get to the guts without having to disconnect any cables(unless connected to the particular bit you are swapping, of course) or unrack anything.

All blades would be OK, assuming you never need to get at the back of the blade chassis; but I pity the fool who has to disconnect and reconnect(and be sure to get them back in the right places... those 8 are on different VLANS!) a nasty mess of ethernet cables just to swap a PSU or something.

Re:It certainly looks cool... (1)

Cap'nPedro (987782) | more than 3 years ago | (#35805954)

I have to wonder at their claim that it works well with standard OEM gear. Even most cheap consumer shit monitors the speed of at least the CPU fan and tends to freak out if a fan that is supposed to be there is either absent or performing substantially below expected speed

Enter the BIOS (hit DEL during Power On Self Test), go into the Power or PC Health (depending on what BIOS you have). Alter the value of CPU FAN to "Not Monitored" or "Ignored". Hit F10 (or whatever yor key is) to save settings and reboot.

SpeedFan (etc) will still give you a speed readout if a fan is connected, but your BIOS won't complain if one isn't.

This procedure should be similar for UEFI based systems.

Tried this (0)

Anonymous Coward | more than 3 years ago | (#35805692)

I did this years ago for a high school tech prep course. I used a non-conductive refrigerant, and an old pc. It's a nightmare in the fact that it's almost impossible to seal, and if you want to add any peripherals it's next to impossible. Once everything is coated there is no going back. It is a great cooling solution, but has major drawbacks.

Re:Tried this (1)

Albanach (527650) | more than 3 years ago | (#35806824)

The video shows the tech unplugging and replugging RAM, so I'm assuming there was no need to seal peripherals that don't have moving parts.

Paging Dr. Freeze (5, Informative)

upside (574799) | more than 3 years ago | (#35805694)

1999 [slashdot.org] Have I been reading Slashdot that long?

Rack density (2)

Askmum (1038780) | more than 3 years ago | (#35805720)

The setup looks nice as it is, but having 42" racks laying on their backs never gives the same rack density than the same racks standing upright.

Re:Rack density (0)

Anonymous Coward | more than 3 years ago | (#35805772)

Course it can. Just stack some racks on top of the horizontal racks.

Re:Rack density (2)

fuzzyfuzzyfungus (1223518) | more than 3 years ago | (#35805836)

All you need is very low ceilings, allowing multiple tiers of racks in what would ordinarily be a 1-story room. You can then hire malnourished Dickensian urchins to scuttle, bent over in a permanent half-crouch, among the dripping coolant tanks, swapping drives and cards... Just remember to flog them frequently, to remind them of their place, or you'll find one cooking his fish-and-chips in the hot-coolant line and clogging the heat exchanger with crumbs.

Re:Rack density (0)

Anonymous Coward | more than 3 years ago | (#35806086)

Or mount half of them upside down to the ceiling

Re:Rack density (1)

rjstanford (69735) | more than 3 years ago | (#35806314)

How do you keep their loose pants on in that case? I wouldn't think that the normal spec grubby suspenders would do the trick any more.

Re:Rack density (1)

Bill Barth (49178) | more than 3 years ago | (#35806206)

Given that you can lay two of these racks back to back and then run them end to end, and that you can remove most of the regular AC equipment from your room, the amount of stuff you can get in your datacenter is the same.

Smells like placebo marketing (1)

Anonymous Coward | more than 3 years ago | (#35805744)

Mineral oil immersion cooling is documented to cause deterioration of components. Perhaps the principals/marketing people are counting on the servers to be obsoleted before they suffer component failure?

Four years of college.... (5, Funny)

Ahimoth (2029382) | more than 3 years ago | (#35805760)

and I still get to go home smelling like fries!

Don't hard disks need air inside them? (0)

Anonymous Coward | more than 3 years ago | (#35805782)

I can understand cooling servers in mineral oil for everything except the hard disk. However, hard disks have a breather hole and actually need air on the inside, because the hard drive heads ride above the disk itself on pressurized air -- among other things this is why hard disks don't work well in space. So what happens when a disk is immersed in oil? The oil MUST get inside of the hard disk due to the breather hole and expansion/contraction of the air inside of the disk as the disk's temperature changes. So I can understand "easy hardware changes" -- but what do you do with a disk that's full of oil to get the oil out? OPEN IT? ?

Re:Don't hard disks need air inside them? (1)

gblackwo (1087063) | more than 3 years ago | (#35806144)

I'd like to know the answer to this also. There are harddrives designed for high altitude/low pressure that are apparently completely sealed, but they are of course going to be more expensive.

You are absolutely right and this is a somewhat normal cause of failure for harddrives. If you use your laptop on a plane and the cabin pressure drops sufficiently, or perhaps you are in Colorado- the read/write head can crash into the platter due to the lack of dense air to ride upon.

Re:Don't hard disks need air inside them? (1)

Larryish (1215510) | more than 3 years ago | (#35806520)

The company that is pushing the oil-as-a-coolant solution may need to remake servers in a form that is conducive (conductive? :) to oil cooling.

Think "oil-cooled server appliance".

False (2, Insightful)

pubwvj (1045960) | more than 3 years ago | (#35805804)

This is all based on a false premise.
There is still heat H to be removed from the server.
The oil speeds up the conductivity between electronics and medium
BUT the heat still must be removed and dumped.
This is not truly more efficient and certainly not "greener" than using air as the medium.

This will make servicing the equipment more difficult and possibly shorten the life of the equipment as well as requiring more complex coolant systems thus making this system less efficient and "green".

Re:False (3, Informative)

RingDev (879105) | more than 3 years ago | (#35806044)

Having worked in a fair number of server rooms, I'd say that the freequency of needing to service equipment has been dropping significantly over the lat 15 years. These days, it's almost a non-issue. I don't think I've pulled a single server for anything but replacement in the last 4 years.

Transfering heat to fluids is significantly more efficient, both on the recieving site (in the server) and the giving side (in the cooling tower). It requires less energy to transfer heat from components to the water (ie: no fans or heat sinks). And it requires less energy to transfer heat from the water in the cooling tower (ie: much smaller chiller/AC unit). So it is more efficient. Acording to the article, their solution consums 50% less energy than the traditional air conditioning and fans.

-Rick

Re:False (0)

Anonymous Coward | more than 3 years ago | (#35806064)

OK, let me as you this: would you rather your entire body be 36.9C uniformly, or your left eyeball be 150C but the rest of your body be 36.875C? Excess heat still needs to be removed, but that may even be taken care of by thermodynamics by simply opening the windows (on a cool day). Even if that's not enough, dispersing the heat over a large volume (read: area) still helps a lot, otherwise you need the ambient temp to be so low that the tiny processor that is generating the heat can lose it to the air around it, which is an extremely bad conductor, even when it is moving (i.e. you have fans).

And you should think a little before calling others stupid, lest you turn out to be the stupid one, with the added bonus of being an asshole.

Re:False (1)

gblackwo (1087063) | more than 3 years ago | (#35806168)

The greenest solution would be to make the heat useful. Air cooled server rooms often self heat portions of their buildings, and reduce heating costs in the winter.

An oil cooled system could transfer heat to the hot water in your building via a heat exchanger and lower your gas/electric costs. And this would be a much more efficient process than using your computers to keep your building warm...

Re:False (1)

slim (1652) | more than 3 years ago | (#35806306)

I wonder what the cost savings would be overall. TFA says it pays for itself in 1-3 years, but that's marketing and it's vague.

If it really saves a load of energy, I can imagine datacentre ops whining about the hassle and the greasy fingers and so forth, saying it's not been worthwhile -- while the suits who commissioned it look at the bottom line of their electricity bill, and deem it well worth a few inconvenienced staff.

Re:False (1)

CarboRobo (1932000) | more than 3 years ago | (#35806412)

Er, no it isn't. You still have the same amount of heat, but the oil is far, far more efficient at moving it away than air is.

Re:False (1)

slim (1652) | more than 3 years ago | (#35806494)

(I feel a bit patronising spelling this out)

It's simple. Pick a component that emits heat. The CPU is a good example. Of course people are working on more efficient CPUs, but they will always generate heat. ... and if that heat stays near the CPU, the temperature will increase until the CPU stops working -- or melts -- or a safety cutoff kicks in.

So, you have to move that heat away from the CPU, and that in itself takes energy -- powering a fan; powering an air-con unit to keep the room cool enough for the fan to be any use, etc.

What we have here is a method to move the heat away from the CPU, that uses less energy than the conventional method of blowing air around. ... and if it doesn't work, it's going to have some pretty dissatisfied customers, because the results will show up directly on their energy bill.

Aha (1)

jovius (974690) | more than 3 years ago | (#35805904)

So this is what happens when you drool too much.

new line of lead lined accessories for summer (0)

Anonymous Coward | more than 3 years ago | (#35805908)

from surviveuncomfortably.consume; these garments (the hats are hysterical), buildings, villages etc.., although heavy, & hot as hell, seem to stay put, & deflect stuff. & even to add weight & the ultimate odd atmosphere protection, there's the pyramidal plasmatic material underneath, for after the crud stops flying/the metal melts.

it's all in the genuine native elders rising bird of prey leadership initiatives, aka teepeeleaks etchings, where it also says that weapons are absolutely unrequired in a living/thriving civilization, after the chosen ones terrifying holycost ends, which is scheduled..

so the whole appenddicks thing is also scammage (0)

Anonymous Coward | more than 3 years ago | (#35806186)

it was tried. the results are seen here & there, from time to time. imagine the audacity of the neo-gods' 'scientists'? hymens? chariots? honestly... are they still here? that's why even the inherently awesome buttonless monkeys & birds etc..., still do not feel safe? yikes

babys welcome Parent Hood Planners (0)

Anonymous Coward | more than 3 years ago | (#35806330)

the php extension of the unstoppable planet/population rescue kode base is expected to add a more local reconstructible dimension to the 'inevitable' ongoings.

baal hearings are planned for the royal pains in the butt, but no positive outcome is expected.

condensation problems... (2)

Jerom (96338) | more than 3 years ago | (#35805934)

When I experimented with mineral oil based cooling, the main issue I had were water droplets condensing on the surface of the cold mineral oil and then promptly sinking... towards the motherboard sitting in the bottom of the old aquarium I was using as a case. Of course there were solutions to this problem but it was a quick and dirty (you can take that dirty word quite literally) test back when I was a student, so we gave up on the idea pretty quickly.

I wonder how they have managed to solved the condensation problem.

Re:condensation problems... (2)

Phs2501 (559902) | more than 3 years ago | (#35806072)

I wonder how they have managed to solved the condensation problem.

They run their oil at 40C. If the dew point in your server room is that high you have other problems...

Re:condensation problems... (1)

Jerom (96338) | more than 3 years ago | (#35806244)

of course!

Our oil was much colder: we were using the cooling element from an old refrigerator (we planned on overclocking that old 386 :-)) and managed to get it down to a couple of degrees celsius :-)

Re:condensation problems... (1)

RingDev (879105) | more than 3 years ago | (#35806082)

If you had condensation (water) that was dense enough to sink in your oil, you were using the wrong type of oil.

Mineral oil is way more dense than water. And condensation that occurs should sit at the top of the pool and never create an issue (and it should be pretty easy to skim off).

-Rick

Re:condensation problems... (1)

Jerom (96338) | more than 3 years ago | (#35806226)

Are you sure about the density? According to http://en.wikipedia.org/wiki/Mineral_oil [wikipedia.org] the density of mineral oil is 0.8g/cm3 while that of water is 1gg/cm3.

J.

Re:condensation problems... (0)

Anonymous Coward | more than 3 years ago | (#35806600)

A working in the petrochemical process industry (transformer oils and such), I have to say that if you are using oil that is heavier than water, you are using the wrong oil.
Bitumen [bp.com] , the heavy, nearly solid stuff, has a density about the same as water. The "regular" oils are about 800 g/liter IIRC.

There has to be a better way (1)

erroneus (253617) | more than 3 years ago | (#35805936)

Okay, oil is more efficient than air. But the problems with this are plainly obvious when it comes to anything that falls in the area of maintenance and upgrades.

I wonder what gasses can be used instead of oil? Something that wouldn't likely leave a residue? Substituting a gas for a liquid might reduce some efficiency, but you are still containing the unit completely and entirely. A lot of efficiency can be added merely through the act of containment. There must be some sort of gas that can be pumped into a container that will serve almost as well as oil.

And is this really so much better than liquid cooling systems placed directly on the electronics? It can't be cheaper.

Re:There has to be a better way (1)

MichaelSmith (789609) | more than 3 years ago | (#35806022)

I wonder what gasses can be used instead of oil?

Refrigerated air might be best. Cool it before it passes through the system, so it can remove more heat. Of course you either pay for the energy to pre-cool the air, or you locate your data centre somewhere very cold. If thats not good enough you could look at supercharging. Compress the air around the system. That increases density and heat capacity.

Re:There has to be a better way (0)

Anonymous Coward | more than 3 years ago | (#35806062)

http://www.bit-tech.net/modding/case-mod/2010/09/15/the-mineral-oil-pc-by-andrew-mollman/1

The biggest problem is the mineral oil damages PVC. There still has to be a radiator of some sort though. The biggest advantage it has over a traditional liquid cooling setup is the connectors of hoses to worry about. And presumably, if your radiator is working well enough, you shouldn't have to worry about condensation.

As for ease of maintenance, that's really all in how they design the case.

And no... it is most definitely NOT cheaper.

Re:There has to be a better way (1)

sh3rp4 (73755) | more than 3 years ago | (#35806180)

I use a combination of Nitrogen and Oxygen with misc other gasses added for flavor. Works pretty well with two modified AC units in my server room. Down side is fire (because of the O2) and my IT staff keep dropping dead from breathing the mixture.

I have to work on that.

Fukucenter (1)

Nobo (606465) | more than 3 years ago | (#35806012)

Wait till the tsunami wipes out the generators.

subject should not be required (1)

manitee (2974) | more than 3 years ago | (#35806094)

do cooling fans inside the servers need to be disabled? seems like churning that fluid would burn them out.

also, i dont know about lifting a loaded 8U blade housing vertically...

Re:subject should not be required (2)

slim (1652) | more than 3 years ago | (#35806178)

Yes, fans are removed. RTFA.

Re:subject should not be required (1)

The Grim Reefer2 (1195989) | more than 3 years ago | (#35806238)

do cooling fans inside the servers need to be disabled? seems like churning that fluid would burn them out.

Since the mineral oil will dissipate the heat from the components much better, they will most likely be fine. But will still use electricity unnecessarily, so I would guess they should be removed, or disabled.

Not really (1)

The Grim Reefer2 (1195989) | more than 3 years ago | (#35806150)

I don't see how this can be good for use w/ a server. It does nothing to increase cooling within the room itself. The server is going to emit the same amount of heat regardless of whether it's air cooled, or mineral oil cooled. The mineral oil will transfer the heat from the components faster, but it will not transfer out of the mineral oil into the air as fast. On a hobbyists computer, it will get shut down, or the load will decrease to almost nothing daily and allow the built up heat to dissipate. On a server that runs constantly, this heat will build continually. If the heat can dissipate out of the mineral oil fast enough to stabilize, then using mineral oil is overkill. If it doesn't, then the server will have to be shut down on a regular basis.

I would guess that a marginal amount of electricity could be saved due to reduced need for cooling fans. But I think that is trivial in comparison to the potential for leaks and the PITA of swapping hardware, not to mention the additional weight on the floor. Then all of trivial things, like the fumes, than any sticker on the equipment eventually coming off and floating around, slippery floors and components, etc.

Re:Not really (1)

The Grim Reefer2 (1195989) | more than 3 years ago | (#35806212)

Well, I just went back and watched the video in TFA. They are pumping the oil out to a radiator, or are cooling it somehow. With the added cost of needing to pump mineral oil and cool it, I'm not sure where the savings in electricity is coming from. And all the other problem are still present.

Re:Not really (1)

sandytaru (1158959) | more than 3 years ago | (#35806322)

Giant data warehouses, in addition to all the fans for their individual servers, also have enormous air conditioners trying to keep the ambient temperature from hitting 40C. All the hot air that is blow OUT of servers has to go somewhere, and if that somewhere is an enclosed warehouse, things would get hot pretty quickly if the room was not originally designed as a data center. I recall hearing something about "chicken coop" style data centers which have windows to allow for air flow and breezes outside, but that means a custom built building which isn't always an option in a world that likes to retrofit when possible. This setup would eliminate the need for anything more than a standard office air conditioner to keep the humans from sweating.

Improvement? (1)

yomammamia (1916736) | more than 3 years ago | (#35806216)

Would work much better with solid state hard drives.

Single point of FAIL (1)

DominicSayers (781748) | more than 3 years ago | (#35806240)

Four racks sharing one pump? What could possibly go wrong?

definitely not for everyone (1)

SpinningCone (1278698) | more than 3 years ago | (#35806372)

it's hard to tell but they might have a special adapter for plugs. i have read that cables will wick the oil in the shielding braid (usb cat5 etc) which can cause a mess.

also there are a lot of server rooms out there that aren't that organized. i'd imagine in a working installation everything would end up oily, i'd be also a bit wary of installing hard drives in these things. i thought they had the pressure hole for a reason and if your coating failed you could have a massive drive failure.

that said i'm sure it does a good cooling job and it would be easy to integrate it into the building hot water and heating system.

overall though I don't think this will make it big

oil in between the card/memory/etc. contacts? (1)

seifried (12921) | more than 3 years ago | (#35806386)

So the oil is not electrically conductive (a good thing right?). What happens when it seeps in between connectors, i.e. into the ram slots or PCIe slots? You start getting really odd random problems, or? How do they address this problem? Also how do you clean the system if you need to service it (i.e. replace bad ram/cards/etc.). If you don't the oils going to get into the slots for memory/cards/etc. when you start swapping components out.

Re:oil in between the card/memory/etc. contacts? (1)

slim (1652) | more than 3 years ago | (#35806774)

My guess about the contacts is that once things are plugged together, they're touching and oil won't break that contact.

I seriously suspect that the expectation is that a system won't ever be repaired -- if it breaks, it's binned. This is likely to be justifiable based on some sums to do with MTBF, depreciation etc. Of course that kind of thing only really works if your operation scales to hundreds or thousands of machines -- or, I suppose, if an insurance company takes on the spread of that risk.

void(warranty) (1)

SnickleFritz (17110) | more than 3 years ago | (#35806390)

Dell won't service my sever. They say it's too sticky and covered with some sort of slime. :(

Not a new idea... (2)

zevans (101778) | more than 3 years ago | (#35806398)

Cray-2 used Fluorinert. In 1985. Related jokes and memes abounded until... dunno. Certainly they were still part of HPC culture when I started my career in 1994.

Mineral Oil is not exactly green (2)

rabun_bike (905430) | more than 3 years ago | (#35806558)

A mineral oil or liquid petroleum is a liquid by-product of the distillation of petroleum to produce gasoline and other petroleum based products from crude oil. And it isn't exactly non-toxic nor non-flammable (see link below). Not to mention all the heavy metals still found in many servers. Inevitably some of those metals will be picked up by the circulating oil so disposal might become an issue as well. Don't get me wrong, I like new ideas that save energy but touting it is "totally green" is skipping a few steps.

http://www.jtbaker.com/msds/englishhtml/m7700.htm [jtbaker.com]

Canada is the answer! (0)

Anonymous Coward | more than 3 years ago | (#35806694)

Why not simply move your servers to a place like Iqaluit or Yellowknife? It is pretty cold up there.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>