Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Generator Delays May Slow Data Center Projects

ScuttleMonkey posted more than 7 years ago | from the so-that's-why-mmos-can't-expand-to-meet-needs dept.

257

miller60 writes "The data center building boom is causing backlogs for new generator orders, with some companies reporting delivery delays of up to a year for new 2,000kw units, which are the current standard for mission-critical facilities. Generator availability is 'the No. 1 thing that will drive your construction schedules,' according to Equinix, which is building centers in three major markets. 'This will be a big issue for the next wave of data center builds,' says another industry executive. Used generators and smaller units tend to be more available than the 2 megawatt units, but companies targeting the enterprise sector may be wary of relaying on used units or smaller generators than those powering competing facilities."

cancel ×

257 comments

Sorry! There are no comments related to the filter you selected.

2 MEGAwatts?!?! (4, Funny)

AKAImBatman (238306) | more than 7 years ago | (#16580584)

"Generator lead time for a nice 2 megawatt diesel engine is now up to a year for one generator," Josh Snowhorn of Terremark said in a panel at the NANOG conference earlier this year.

HOLY MOLY, that's a lot of power! If you had 250 watt power supplies, all running at maximum, you'd be able to power 8,000 power supplies simultaneously! You could run a small town on a generator that large! Or to put it another way, you could use it to power a Diesel Locomotive capable of pulling dozens of fully loaded cars.

Wow, just wow. That's just an incredible amount of power to be putting in a datacenter. Is it even possible for these centers to run off the grid?

"So we can build all the raised floor we want, and then sit around and wait six months for a generator."

The part that I don't undestand is this: Why do they need just one generator? If you're having difficulties obtaining a 2Mw unit, wouldn't it make sense to get two smaller units? You'd waste a bit of extra space, but you'd have redundancy that a single genearator couldn't offer.

Re:2 MEGAwatts?!?! (3, Insightful)

CastrTroy (595695) | more than 7 years ago | (#16580648)

Why don't they just go find an old diesel locomotive and convert that into a generator? Or get a really big diesel truck. A diesel generator isn't much more than a diesel engine. I see opportunities here for new manufacturers.

Re:2 MEGAwatts?!?! (2, Insightful)

MadEE (784327) | more than 7 years ago | (#16580714)

A deisel train engine is a generator the traction motors on the train are just that motors that are driven by the generator. It's actually a really cool design for a vehicle and allows for breaking using resistive loads across the traction motors (aka dynamic breaking)

Re:2 MEGAwatts?!?! (0)

Anonymous Coward | more than 7 years ago | (#16580746)

Why don't they just go find an old diesel locomotive and convert that into a generator?

That's actually not too bad of a an idea. A Diesel Locomotive is already a large generator, using electrical output to drive the motors over the axles of the train. The only problem I think you'll have is the same supply problems already occurring. Locomotives tend to be long-lasting pieces of equipment, and are not retired all that often. On the bright side, your power source would be mobile. (Assuming you could built a connecting track to the datacenter.)

Re:2 MEGAwatts?!?! (5, Informative)

Volante3192 (953645) | more than 7 years ago | (#16580782)

Well, converting an existing engine gives you the same, if not more, concerns as getting a used generator. As TFA mentioned, there's no problem finding used 2MW generators, but it's the stigma of using used equipment period.

Add to that the changes and parts necessary to change engines geared to creating propulsion to engines geared to creating electricity.

I doubt any company who considers their data center a key component of thier infrastructure to risk their backup solution on an untested refurbished generator.

Re:2 MEGAwatts?!?! (2, Informative)

DigiShaman (671371) | more than 7 years ago | (#16581218)

I doubt any company who considers their data center a key component of thier infrastructure to risk their backup solution on an untested refurbished generator

New or used, it doesn't matter. Your SUPPOSED to testing the generator at least once a week. Besides, it's not good to have aged fuel in the tank. You end up with all sorts of fuel-line and injector clogging due to the fact fuel will take on properties of varnish.

Point is, it does't matter if it's used. If it does the job and can be verified to do so once a week, you're ok.

Re:2 MEGAwatts?!?! (1)

Dun Malg (230075) | more than 7 years ago | (#16581410)

Add to that the changes and parts necessary to change engines geared to creating propulsion to engines geared to creating electricity.
Indulging in some Railroad Pedantry here, but locomotives use a diesel driven generator to drive electric motors at the drive wheels. You could, with a little jury-rigging, conceivably plop a diesel locomotive down next to your building and wire it up to your electrical main switch gear and just start it up...

Re:2 MEGAwatts?!?! (4, Funny)

johneee (626549) | more than 7 years ago | (#16581924)

Same with Hovercraft. Of course, the supply of hovercraft is quite a bit more limited than locomotives, but that's beside the point.

Plus, you'd have to deal with all the eels.

Re:2 MEGAwatts?!?! (1)

Intron (870560) | more than 7 years ago | (#16580996)

What you want is an old diesel-electric loco like the Alco S-2 [railfan.net] which is basically a 1000 HP diesel generator and some electric motors that you won't need. Imagine having one of these sitting outside your data center battery room. Unlimited nerd points.

Re:2 MEGAwatts?!?! (2, Insightful)

drrck (959788) | more than 7 years ago | (#16581022)

You are correct that a GenSet is not that much more than a generator paired with a diesel engine, however the entire engine industry as a whole is somewhat supplier constrained.
If a new company were to come and attempt to meet the demand present they would be fighting for the same parts and resources that other more established companies are.

Re:2 MEGAwatts?!?! (1)

networkBoy (774728) | more than 7 years ago | (#16582004)

So what you're saying is someone needs to start-up a company making fly-wheels and induction coils.
-nB

Re:2 MEGAwatts?!?! (1)

jcr (53032) | more than 7 years ago | (#16581610)

Why don't they just go find an old diesel locomotive and convert that into a generator?

That's been done, and frequently at that. If you already have a rail siding near your building, it's a pretty straightforward job.

-jcr

Re:2 MEGAwatts?!?! (1)

HornWumpus (783565) | more than 7 years ago | (#16581788)

Lots of small towns ran off old locomotives untill they were wired up to the grid.

Once that happened the locomotives could'nt compete with bigger generators as full time power. There might still be a few in 'bum fuck' Alaska and Canada.

Re:2 MEGAwatts?!?! (1)

Short Circuit (52384) | more than 7 years ago | (#16580688)

Combining the power output of to AC voltage sources isn't cheap.

Re:2 MEGAwatts?!?! (1)

GigsVT (208848) | more than 7 years ago | (#16581426)

You just have to keep the generators in phase. It's not that hard.

Re:2 MEGAwatts?!?! (2, Insightful)

SevenHands (984677) | more than 7 years ago | (#16580742)

"Wow, just wow. That's just an incredible amount of power to be putting in a datacenter. Is it even possible for these centers to run off the grid?" The generators have to supply enough on demand power to satisfy peak requirements. Examples could be getting the elevator(s) stuck between floors to where they're supposed to go. Air conditioning is another large power drain and in a datacentre, there are huge quantities of heat contributing components to deal with. Even backup lighting utilizes a surprising amount of power.

Re:2 MEGAwatts?!?! (1)

AKAImBatman (238306) | more than 7 years ago | (#16580832)

The generators have to supply enough on demand power to satisfy peak requirements. Examples could be getting the elevator(s) stuck between floors to where they're supposed to go. Air conditioning is another large power drain and in a datacentre, there are huge quantities of heat contributing components to deal with. Even backup lighting utilizes a surprising amount of power.

So what you're saying is, the generator has to be able to power a small skyscraper in an emergency?

Re:2 MEGAwatts?!?! (1)

ThosLives (686517) | more than 7 years ago | (#16581012)

I've often wondered why they don't liquid cool these, or use ducted air cooling, then recover a portion of the waste energy. Any time you have a temperature difference you can get some returns. Considering these datacenters generate so much thermal energy, you'd think they would attempt to recover some rather than spend *more* energy to move it outside.

This is why certain engine testing facilities use their dynos to power portions of their facility - instead of burning fuel and paying for electricity, they reduce their electricity costs. Of course, this does require some up-front capital, but if people would think about the long run rather than the immediate, it shouldn't be that difficult a proposition.

Think - if you've got a 2 MW genset, that probably means you're running 1 MW continuous; let's say only 25% of that is computer (250 kW) and you can recover only 10% of that somehow. You'd be recovering 25 kW, which is actually a significant amount of power - probably enough to run a significant portion of lighting (that's over 400 60-watt lighting sources - or if you've got the new 15-watt fluorescents, over 1600 lights).

Holy vishnu..or something...!!1! (1)

enven (992813) | more than 7 years ago | (#16580906)

Approximately how much is 2MW per hour for energy cost? Is it around 110.00 Per hour? Or 10 times as much?... I wonder if the data center will actually be worth the cost overall...

Re:Holy vishnu..or something...!!1! (2, Interesting)

AKAImBatman (238306) | more than 7 years ago | (#16581188)

Step 1: Take a price from this map [michaelbluejay.com] .
Step 2: Multiple that price times 2,000.

So for California, 2,000kWh would cost $240 per hour to run. That's $5,760/day, $40,320/week, and a whopping $2,096,640/year!

Of course, for diesel your prices may be higher. As of right now, diesel is approximately $2.669 per gallon [doe.gov] in California. To compute the costs, you'd need to know how efficient the generator is. This page [uaf.edu] claims "approaching 40%", so we can use that for a guesstimate. At about 146,520,000 joules per gallon of diesel, we can compute a need of 122.85 gallons per hour [google.com] . At the going rate, it would cost ~$327.89/hour to run a 2MW generator.

Re:Holy vishnu..or something...!!1! (1)

Aczlan (636310) | more than 7 years ago | (#16581808)

that is for ROAD Diesel, what you would be running in a generator would be UNtaxed Diesel (here in upstate NY that is about $0.80 /gal of difference... (making UNtaxed Diesel about $2 per gallon)) making it $245.7 per hour to run said generator

Re:Holy vishnu..or something...!!1! (1)

AKAImBatman (238306) | more than 7 years ago | (#16581862)

UNtaxed Diesel about $2 per gallon

Thanks! I figured there was probably a price difference, but I wasn't able find it at the time.

Re:2 MEGAwatts?!?! (1)

Vihai (668734) | more than 7 years ago | (#16580970)

Why do they need just one generator? If you're having difficulties obtaining a 2Mw unit, wouldn't it make sense to get two smaller units?
No, you would need at least 3, 1 MW units, otherwise you would double your probability of failure.

Re:2 MEGAwatts?!?! (5, Informative)

jeffmeden (135043) | more than 7 years ago | (#16580990)

On your comment about data center size: Consolidation is the buzzword of the decade. 'Big boy' data centers start at 2MW and get BIGGER. It's not uncommon for a main site facility to be closer to 4MW. These are usually fed from the grid at the 14kV level, often from dual substations. Yes it's a lot of CPUs and supporting equipment, but thats the way we like em!

Your question on two generators vs. one: Redundancy does NOT come from two half sized units doing a single job, it comes from two FULL sized units each doing half the job. Having two half sized generator units means losing one will cause failure. With a 10,000HR MTBF per unit (rough number used for demonstration) you get a system MTBF of 5,000HR, NOT what I would call redundant! You would need at least 3/2 redundancy (3 generators doing the work of 2) to have a reasonably reliable replacement for a single large unit.

Re:2 MEGAwatts?!?! (0)

Anonymous Coward | more than 7 years ago | (#16581096)

So you are saying that having two smaller generators instead of one large one doubles the risk of power failure?

Re:2 MEGAwatts?!?! (2, Informative)

Dun Malg (230075) | more than 7 years ago | (#16581584)

So you are saying that having two smaller generators instead of one large one doubles the risk of power failure?
Doubles the risk of a show-stopper failure in the backup system, yes. Twin engine aircraft are required to be able to fly with one engine out for this very reason. A crude way to visualize it is that the MTBF rates are like unto the chance of rolling a Critical Failure in [your favorite RPG]. One generator rolls the dice every (x) amount of time, and TWO generators is essentially rolling the dice TWICE every (x) amount of time. Odds of rolling a critical failure statistically double.

Re:2 MEGAwatts?!?! (1)

iwsnet (946715) | more than 7 years ago | (#16581850)

That's unbelievable all the power these centers need. Just think of the millions of users they are serving with data and Internet usage. It just sucks up electricity like crazy.

Re:2 MEGAwatts?!?! (1)

Barny (103770) | more than 7 years ago | (#16581948)

As to this, i would have thought it a good idea to have a pair of shiny new generators (each capable of full load) as well as a little more cash for a used one of same size:

A. you know its in working condition RIGHT NOW (would be nice to try and start up your pair of new generators to find them DOA)

B. if there is a fault, a problem-over-time fault, with the new generators, you don't want them all dieing in the same week, haveing an old POS there could just save the company if/when excrement hits the air excitation device :)

Imho treat em like a mirrored HDD setup, hell, you could go the whole hog, get 4 smaller sized generators and raid5 em together ^_^

Re:2 MEGAwatts?!?! (3, Funny)

megaditto (982598) | more than 7 years ago | (#16581000)

That's just an incredible amount of power to be putting in a datacenter.
Not if your customers requires sharks with freaking laser beams attached to their heads.

Why do they need just one generator? [...] wouldn't it make sense to get two smaller units?
I agree. And just imagine the publicity value of having two smaller 1337 kW genenartors powering up your data center in tandem.

Re:2 MEGAwatts?!?! (2, Interesting)

Zak3056 (69287) | more than 7 years ago | (#16581032)

The part that I don't undestand is this: Why do they need just one generator? If you're having difficulties obtaining a 2Mw unit, wouldn't it make sense to get two smaller units?

Keep reading the article--further down it says that large data centers (like MS and Google are building) need 20 or 30 2MW generators! My question: if you power requirements are that high, surely it must make sense to build your own powerplants? Multi-year construction time, I guess?

I know there are several 60-75MW units under construction in the US now (a mix of coal and gas, IIRC.) If it makes sense for a utility to build them, surely it would make sense for a data center owner to do so if they were going to use all of that capacity.

After the next bubble crisis (2, Funny)

DrYak (748999) | more than 7 years ago | (#16581140)

need 20 or 30 2MW generators! [...]I know there are several 60-75MW units under construction in the US now

Comming soon on eBay : 100MW generators
starting bid : 10$
condition : mint
reason : our startup went belly up.

When the next speculation bubble burst, there're sure going to be a lot of diesel mastodonts left every were...

Re:After the next bubble crisis (2, Funny)

Joe The Dragon (967727) | more than 7 years ago | (#16581362)

shipping starting at $1000+

Re:2 MEGAwatts?!?! (1)

Iphtashu Fitz (263795) | more than 7 years ago | (#16581066)

The part that I don't undestand is this: Why do they need just one generator? If you're having difficulties obtaining a 2Mw unit, wouldn't it make sense to get two smaller units? You'd waste a bit of extra space, but you'd have redundancy that a single genearator couldn't offer.

I used to work at a company who had a lot of equipment at an MCI enterprise-grade datacenter. I forget the exact terminology but they apparently have different classes of datacenters, and this was at the top of the line. It was physically located in an area where it could tap into redundant power sources, redundant network backbones, etc. It has a total of 7 generators. Here's the Google Maps view [google.com] of the datacenter. The empty field to the north is meant to house their second datacenter that they plan to build there when the first one is at capacity.

Re:2 MEGAwatts?!?! (1)

AuricTheCodePoet (934925) | more than 7 years ago | (#16581192)

Problem with multiple generators would be the wiring. Most companies putting in a generator this large also put in automatic transfer switches that will fire up the generator and cut over within seconds of an outage.

Re:2 MEGAwatts?!?! (1)

Kirgin (983046) | more than 7 years ago | (#16581202)

We rent cage space in the Equinix facility in San Jose. in our cage we have close to 40 servers. There was a yahoo cage that had close to 3000 servers. Even they did not use a lot of realestate. I wouldn't be suprised if the facility housed upwards of 20 000 individual devices (anything with power).

The facility have fuel tanks for their generators that wouldn't look out of place at an oil refinery.

With 60 20 ton air conditioners...110 ISPs with available network drops....UPS's that would put a military facility to shame...completely redundant power distribution and AC ducks you could drive a transport truck through.

Re:2 MEGAwatts?!?! (0)

Anonymous Coward | more than 7 years ago | (#16581464)

With 60 20 ton air conditioners...110 ISPs with available network drops....UPS's that would put a military facility to shame...completely redundant power distribution and AC ducks you could drive a transport truck through.


Uh I don't think so, scooter. You've obviously never been to an ELINT/COMINT facility. And yes, I have worked in large civilian data centers.

Re:2 MEGAwatts?!?! (0)

Anonymous Coward | more than 7 years ago | (#16581284)

HOLY MOLY, that's a lot of power! If you had 250 watt power supplies, all running at maximum, you'd be able to power 8,000 power supplies simultaneously! You could run a small town on a generator that large! Or to put it another way, you could use it to power a Diesel Locomotive capable of pulling dozens of fully loaded cars.

That's NOT a lot of power.

By your math. If I have 133 cages with 3 racks each that each hold 72 1-U servers that account for 7200 of the power supplies you mention. Add to that the air-conditioning - and you're way over budget.

That's not a very big data center.

Re:2 MEGAwatts?!?! (1)

Dun Malg (230075) | more than 7 years ago | (#16581322)

The part that I don't undestand is this: Why do they need just one generator? If you're having difficulties obtaining a 2Mw unit, wouldn't it make sense to get two smaller units? You'd waste a bit of extra space, but you'd have redundancy that a single genearator couldn't offer.
Remember the old aeronautical engineering maxim of "twin engine planes have twice the rate of engine trouble as single engine planes". Point being, that sort of "redundancy" is only a net gain if you can actually "fly" with one of your engines dead. If these datacenters require more than 1 megawatt, two 1 megawatt generators is actually a liability. The companion maxim to the above is "It's better to put all your eggs in one basket, so long as you've made sure you've got a REALLY STRONG basket."

Re:2 MEGAwatts?!?! (1)

hpa (7948) | more than 7 years ago | (#16581606)

Remember the old aeronautical engineering maxim of "twin engine planes have twice the rate of engine trouble as single engine planes". Point being, that sort of "redundancy" is only a net gain if you can actually "fly" with one of your engines dead. If these datacenters require more than 1 megawatt, two 1 megawatt generators is actually a liability. The companion maxim to the above is "It's better to put all your eggs in one basket, so long as you've made sure you've got a REALLY STRONG basket."

The right answer, really is to get THREE 1 MW generators instead of one 2 MW genny, so that you only need two out of three to operate.

Re:2 MEGAwatts?!?! (1)

petermgreen (876956) | more than 7 years ago | (#16581384)

Wow, just wow. That's just an incredible amount of power to be putting in a datacenter. Is it even possible for these centers to run off the grid?
2MW at 33KV is only 60 amps, a 33KV 60A line is hardly going to be a challange to construct.

so linking them to the grid really shouldn't be a problem.

you'd be insane to try and run a datacenter off generators all the time. Electricity from deisel generators costs several times what grid electric costs even if you pay consumer prices for grid electricity.

say 4 kilowatt used in each rack (a 42U rack with say a switch a patch panel and 40 machines at 100W each), another kilowatt per rack for aircon (probablly more in reality). thats only 500 racks to get up to two megawatts.

Re:2 MEGAwatts?!?! (1)

AKAImBatman (238306) | more than 7 years ago | (#16581826)

I wasn't really worried about the power lines. That's easy enough to handle, as demonstrated by the large transmission lines strung near my home.

I was more worried about the grid being able to supply the capacity. When you start demanding power in the multi-megawatt range, you enter the territory of small power plants. A LOT of power plants would have difficulties supplying their normal load to residential and commercial customers, plus your industrial needs for power. So either you locate near some rather large plants with sufficient capacity, accept long distance transmission from the national grid, or generate your own power internally. Each choice has some serious consequences for space availability, reliability, and cost.

In any case, data centers like these mean we're going to need a lot more power plants in the near future. 2MW may not be much for a gigawatt reactor, but multiplied by dozens (the article makes it sound more like hundreds) and you start noticing a serious drain on available power resources.

Re:2 MEGAwatts?!?! (2, Informative)

Critical Facilities (850111) | more than 7 years ago | (#16581440)

Yes 2 Megawatts. I work running the engineering department for a major financial provider and our campus has two 2 Megawatt generators dedicated to our datacenter (as well as 2 more for the rest of the campus). We actually have most of the infrastructure in place for a third 2 Megawatt generator as we develop the need for power.

You'd be surprised at how much the average load in KW/sq ft has increased in the last several years for a datacenter. Moore's law is FIRMLY in place and it's getting harder and harder to provide adequate, clean, redundant power in critical environments. Server manufacturers are really upping the ante on power consumption which in turn ups the ante on cooling the space(s).

As for your question "wouldn't it make sense to get two smaller units?" , the short answer is no. Mainly, you'd have to have more electrical components (switchgear, breakers, transfer switches, etc) in order to have more than one, plus you've now got another generator to maintain and you're going to need the physical space for it (and all the gear associated with it which is quite a lot). Add to that all the safety equipment you need (FM-200 fire supression, CO2 fire suppresion, Building Automation monitoring, fuel level/leakage detection) and you get really expensive really quickly.

Re:2 MEGAwatts?!?! (1)

nocaster (784709) | more than 7 years ago | (#16581456)

Our data center has a 10 Kw generator. Not only does it run our servers, but also the chillers and air handlers to cool the place.

Re:2 MEGAwatts?!?! (1)

Mattintosh (758112) | more than 7 years ago | (#16581970)

All of your points are pretty much spot-on.

The only major data center I've been in had 6 diesel 12.5kW generators. It also had power feeds from two different grids, water from two different mains, and a water tower that could keep enough chilled water to keep the place running for 2 hours if the chillers all failed. We're talking a MAJOR data center with plenty of redundancy. It did not need 2MW generators. It only needed a total of 75kW.

There's also a hydroelectric dam located near where I grew up. It could produce 20-30MW, and the tour guide mentioned that it could power a town of about 20,000 people. Clearly, 2MW is way too much power for a datacenter.

Re:2 MEGAwatts?!?! (2, Interesting)

NerveGas (168686) | more than 7 years ago | (#16581998)

Most places will let each cabinet get two 20-amp circuits, for roughly 5 kilowatts. That means you can only provide power forup to 400 cabinets. But don't forget, they also have to power all of their own equipement, and that you don't want to run a generator at 100% capacity. That means that you could be talking about as few as 200 cabinets from that sort of generator.

That's still a good number of cabinets in a datacenter, but it's not the unbelievable size that seems to jump out at you when you think of 2 megawatts.

As for two smaller units, if one fails, can the other handle the full load of the data center? If not, you're screwed.

May slow? (0)

Anonymous Coward | more than 7 years ago | (#16580590)

Without a generator, the project stops.

Easy, just buy... (4, Funny)

DigitalRaptor (815681) | more than 7 years ago | (#16580602)

Easy, just buy 400 5kw generators instead of 1 2000kw generator.

You may have to hire more people to start them, though. And change the oil.

Re:Easy, just buy... (5, Informative)

Detritus (11846) | more than 7 years ago | (#16581366)

Then you have to keep them phased together. That can be complicated and error-prone. When you're operating on diesel power and lose a generator, the load gets dumped on the remaining generators, which can cause large frequency errors and brownouts. To put a spare generator online, you have to match frequency and phase with a system that is under severe stress. It can get ugly.

Re:Easy, just buy... (1)

DigitalRaptor (815681) | more than 7 years ago | (#16581462)

You may have missed the sarcasm in my [entire] post, but you are correct. Using a farm of smaller generators has many complications, not the least of which is inefficiency and managability.

Multiple Smaller Generators? (1)

SevenHands (984677) | more than 7 years ago | (#16580640)

This would introduce partial redundancy into the system. However, synchronizing with 50/60 hz between gensets might pose an issue.

Re:Multiple Smaller Generators? (1)

Amouth (879122) | more than 7 years ago | (#16580924)

suprisingly not.. it is only the first two that battle it out.. the lower power source will sync up with the greater power source when you combine them (AC)

this is what happens when they bring new powerplants on the grid.. for a short time there is some phase issues but within minutes the new plant will be in sync with the rest of the grid.. i assume this would apply to using these smaller sources

There is a stopgap measure for this (4, Funny)

eln (21727) | more than 7 years ago | (#16580646)

If you've built out your data center but the generators have not yet arrived, a quick and inexpensive solution is to buy several thousand hamsters and hamster wheels. These will keep your data center humming along in the event of an outage. If PETA starts protesting, sysadmins with larger wheels can be substituted. Most of them will find this activity a welcome break from the inane busywork in their everday lives.

Re:There is a stopgap measure for this (1)

frosty_tsm (933163) | more than 7 years ago | (#16581194)

I know that sysadmins put up with a lot of crap normally, but this would bring it to a whole new level.

Re:There is a stopgap measure for this (1)

Apocalypse111 (597674) | more than 7 years ago | (#16581370)

I've actually found that I get a better wattage/food pellet ratio with gerbils. Had to put them in as a temporary measure a couple years ago, but they've worked so well since we installed them that we never bothered to upgrade.

I heard of a guy in L.A. doing the same thing with ferrets, but I for one don't think I could stand the smell. Plus, the little tube-rats tend to escape and crawl under the flooring to nest, and that just raises all kinds of fire hazard issues.

Seems redundant smaller used units would be more.. (0)

Anonymous Coward | more than 7 years ago | (#16580672)

...reliable than a single large unit, no matter how new.

Hi, I'm your host and... (0)

Anonymous Coward | more than 7 years ago | (#16580682)

... welcome to this night's episode of "SI Abuse"! Tonight, we will wildly mix prefixes, spaces, significant digits and, to top it all off, we will secretly replace a capital with a miniscule letter. The first team to find the misspelled unit gets 10 points!

Re:Hi, I'm your host and... (0)

Anonymous Coward | more than 7 years ago | (#16581330)

Not to mention inappropriate words...

...but companies targeting the enterprise sector may be wary of relaying on used...

The correct word is relying .

ed-i-tor
-n
1: a person responsible for the editorial aspects of publication; the person who determines the final content of a text (especially of a newspaper or magazine)

http://dictionary.reference.com/browse/editor [reference.com]

I can understand completely (5, Informative)

hurting now (967633) | more than 7 years ago | (#16580696)

You need to remember, its not just the computers & servers. Its the lights, the heating/AC, all other power needs. We have a generator at my work and it has to power EVERYTHING. 100 desktops 34 thin clients, 19 laptops, 34 servers, the A/C for the data center, the elevator, the emergency lighting, fire detection system, phones... you get the idea. We have a 30 KVA battery backup for the data center in case the generator doesn't kick on the right way.

Re:I can understand completely (3, Interesting)

scuba_steve_1 (849912) | more than 7 years ago | (#16581694)

I studied this topic for work this year and learned that the current data center rule of thumb is that for every watt that you spend powering a server, you must spend another watt on air conditioning...and it gets worse.

Now, with servers getting smaller (e.g., blades) and CPUs evolving to multi-core, heat and power usage density is increasing dramatically within the same floor space...and cooling it effectively ranges between difficult and nearly impossible...without ripping everything out and starting over...which few can afford to do. Effective cooling now requires knowledge of interior meteorology (to butcher a term). Data centers now end up with weather patterns...with cold zones, hot zones, wind, plague, pestilence...and it gets worse every day.

According to a study conducted last year by AFCOM (http://www.afcom.com ), a leading association for data center professionals, data center managers reported a 10% or more increase in power requirements during the previous year, with most expecting that number to continue to rise at a similar rate for the foreseeable future. As a result, 41% of those surveyed stated that the will be forced to upgrade their power and cooling systems within the next 36 months.

What to do? Well...first off, exploit the hardware that you have. Google VMware...and start using those spare cycles for more virtual platforms. Secondly, stop cramming everything into small spaces...learn to expand gracefully. Otherwise, that rule of thumb is going to fail...and we'll end up spending two-thirds of our power dollar on A/C...and still end up frying hardware. Lastly, how about demanding power usage metrics from vendors when specing out servers? Power supplies installed in many servers frequently run at 65% efficiency. The result is an enormous amount of excess power used that simply generates heat, which must then be removed. Unfortunately, the industry has not yet established benchmarks for performance-per-watt expended. Start demanding that info. You'll get blank stares and cryptic answers, but let them know that it matters, That electricity doesn't come out of the wall folks...it comes from burning coal, fossil fuels, hydro, and nuclear power. We either need to get over our fear of nuclear power and our distaste for dams...or start demanding better performance.

$0.02

DC power? (1)

vertinox (846076) | more than 7 years ago | (#16580704)

Wouldn't having datacenters switch over to DC power and then just using a massive battery backup system help with this? Or batteries more exspensive than having generators.

I'd still keep a few generators in backup, but woulnd't you need less of them with DC?

Re:DC power? (4, Insightful)

Silver Sloth (770927) | more than 7 years ago | (#16580790)

You can keep your generator(s) running indefinitely. Certainly longer than any predictable power outage, but if you're running on batteries you're against the clock. What are you going to do when they start to run down - nip over to the 7-11 and buy all the AAs they've got?

Re:DC power? (1)

kabocox (199019) | more than 7 years ago | (#16581530)

You can keep your generator(s) running indefinitely. Certainly longer than any predictable power outage, but if you're running on batteries you're against the clock. What are you going to do when they start to run down - nip over to the 7-11 and buy all the AAs they've got?

Um, don't forget fuel for those generators! During an interview at one company a question about the companies on-site generators poped up. The company folks related a humorus tale of having the generator running, but having a very difficult time finding a fuel truck to keep it fueled. The problem was that there was an ice storm through out the state and there was zero travel on the interstates so there wasn't any fuel that they could buy and use right then. The lesson was stock up and have fuel trucks waiting on site with enough fuel for about a week of fuel. Unlikely that you'll ever use a week of fuel during one event and that most power outages would be under 2 hours, but for a data center that services the entire nation or globe it is unacceptable that just because you have 5 ft of snow on the ground and no power at the data center that the entire company be down. That's also why most of them have atleast 2 data centers running things as well. Just in case one goes down because of hurricane, torando, flood, earthquake, or mudslides you'd have a backup.

Re:DC power? (1)

Silver Sloth (770927) | more than 7 years ago | (#16581728)

Good point. Have you seen 'The Dish' [imdb.com]

Re:DC power? (1)

silas_moeckel (234313) | more than 7 years ago | (#16581800)

Some of the DC's I have worked with have moved to natural gas generators and some big bottles of gas as well. I have seen one that went as far as a diesel and a NG genset in a Nx2 setup. The nice thing about BG and LP is you can bury the tanks easily.

Re:DC power? (0)

Anonymous Coward | more than 7 years ago | (#16580926)

Car batteries give about 100 Ah until they're sucked dry (at which point they must be replaced -- lead acid doesn't like that). 100 Ah means that it could power a 2 megawatt facility for .21 seconds per battery (obviously the battery has internal resistance that wouldn't allow that, but let's just pretend). To power the facility for, oh, 6 hours, you would need 102,857 car batteries.

Yes, there are batteries bigger/better than a car battery, but there's no way you are going to power that facility with ANY batteries if it would take 100,000 car batteries!

Re:DC power? (1)

wwest4 (183559) | more than 7 years ago | (#16581064)

Yes, you'd need fewer, but the scale they are talking about is still way too cost prohibitive for batteries... it sounds like in this case, there are no mains... just generated power. Even so, you couldn't just use a battery to cover anything but super-brief mains outages, because the extended runtimes would probably require way too much in the way of battery cost, weight, floor space, etc.

Re:DC power? (1)

Iphtashu Fitz (263795) | more than 7 years ago | (#16581446)

Wouldn't having datacenters switch over to DC power and then just using a massive battery backup system help with this? Or batteries more exspensive than having generators.

In large datacenters a battery backup system (UPS) will only last a few minutes, maybe a few hours. Generators can/will run indefinitely as long as clean fuel is available.

Any datacenter that has a generator will (should) have UPS's in any event. It takes a few seconds/minutes for generators to start up and provide full power when a power failure occurs. In a properly designed datacenter the UPS's provide short-term power to the equipment during a power failure while the generators start up automatically. Once the generators are up and running then the batteries cut out again. When power is restored the process basically reverses itself - the UPS's take the load while the generators shut down and power from the street again takes over. (Actually the generators shouldn't shut down at this point - they should recharge the UPS batteries. If steet power is lost again the UPS's and generators need to be ready to pick up the load again immediately)

Re:DC power? (1)

Iphtashu Fitz (263795) | more than 7 years ago | (#16581536)

Wouldn't having datacenters switch over to DC power and then just using a massive battery backup system help with this?

Another comment. The vast majority of equipment in datacenters is designed to run on AC. Most companies simply buy servers from the likes of Dell, IBM, etc. which by default come with AC power supplies. Getting all the customers of a commercial datacenter to switch everything to DC would be cost prohibitive. Then there's the issue of other critical equipment like HVAC for cooling. they typically run on three-phase AC (480 volts). I have no idea if DC HVAC systems even exist.

When I worked for a big internet company who had equipment in an enterprise grade colo facility we looked into using DC since our power consumption was huge. The datacneter was already providing DC service to a telco switch housed within the facility. We would have had to have paid for the colo facility to upgrade their DC power output as well as the cost of running DC circuits from one end of the building to the other. All of that was prohibitively expensive. The bottom line was that the datacenter considered DC to be a special case for the telco and wasn't something they intended to offer to their other customers.

Re:DC power? (2, Informative)

RxScram (948658) | more than 7 years ago | (#16581600)

In a former life, I worked on large (800KVA) uninteruptible power supplies as a field service engineer. Normally, for highly mission critical data centers, as described in the article, the data center is powered by 2 different, independent power grids (if possible.) For example, there would be one feed from power company A, and another feed from power company B. These would be connected by using an automatic transfer switch (ATS) which would immediately switch to the other power feed if the main failed. The output of this ATS would then be connected to another transfer switch which was connected to the emergency generator, which starts automatically in the event of a power failure.

On the downstream side of this second ATS (the one that switched between the redundant power company feeds and the generator feed), there would be the redundant UPS network, which would in turn feed a different type of transfer switch before heading to the actual critical load in the data center. Each of these multiple redundant UPS's were provided backup power with large strings of batteries. A typical 800KVA UPS would have 3-4 strings of 40 batteries each. Each of these batteries would cost around $100-250, and weigh about 100 lbs. The data centers I worked on, which were nowhere near as large as some of the MS or Google data centers described in the article, had between 4-16 independent, redundant UPS's.

How long would this huge amount of batteries keep the UPS up at full load in the event of a power outage?

About 15 minutes, if you were lucky and had batteries at the top end of their charge cycle.

The whole purpose to having the UPS redundancy and massive battery backup is to give the emergency generators enough time to start and come online.

When I was working at MGE, a typical figure we were told was that a minute of downtime for some of these data centers can cost the datacenter well over a million dollars in lost revenue, and immediate termination of the CIO (or whatever equivalent) of the company. I'm not sure how accurate this was, but it seems reasonable considering how much money goes through some of these places in a day.

Re:DC power? (1)

Dun Malg (230075) | more than 7 years ago | (#16581716)

Wouldn't having datacenters switch over to DC power and then just using a massive battery backup system help with this? Or batteries more exspensive than having generators. I'd still keep a few generators in backup, but woulnd't you need less of them with DC?
Batteries don't have nearly the power density of diesel fuel. The only advantage to battery backup is that it's instantaneous. Most datacenters DO have a battery system, but its purpose is merely to span the gap between the loss of mains power and backup generator startup.

Suggestion (4, Funny)

truthsearch (249536) | more than 7 years ago | (#16580724)

Take out a $10,000 simolian loan and place a natural gas power plant in a corner. It costs $9,000 simolians and produces "moderate pollution", so residential and commercial zones won't develop well nearby. But your city won't grow quickly without a lot of power.

not on my project (3, Informative)

superstick58 (809423) | more than 7 years ago | (#16580752)

I didn't see any delays on a project I am working on. We have 4 Gens in the 2MW range plus 3 gas/diesel gens in the 30MW range. Of course, this isn't for a datacenter, but I wonder if the generators would be the same.

Re:not on my project (2, Informative)

AKAImBatman (238306) | more than 7 years ago | (#16580796)

We have 4 Gens in the 2MW range plus 3 gas/diesel gens in the 30MW range.

What are the 4MW units if not diesel? If you're using Gas Turbines, that may explain why you didn't have the same supply problems.

Re:not on my project (1)

AKAImBatman (238306) | more than 7 years ago | (#16580876)

s/4MW/4 2MW/g

You get the idea.

Re:not on my project (1)

superstick58 (809423) | more than 7 years ago | (#16581030)

The smaller gens are diesel. Only the big boys run on gas.

Re:not on my project (1)

MadEE (784327) | more than 7 years ago | (#16581076)

What are the 4MW units if not diesel?
Possibly petrol or more likely it could use Bunker Oil.

Re:not on my project (1)

ThosLives (686517) | more than 7 years ago | (#16581092)

Impressive...3 x 30 MW plus 4x2 MW!? What kind of 'project' other than municipal power or maybe an airport or hospital would require that? 98 MW is substantial generation capacity.

Generator Delays (2, Interesting)

kilodelta (843627) | more than 7 years ago | (#16580910)

You would think that companies that make larger generators (I'm talking > 100kW) would understand that demand goes up during hurricane season, and things of that nature.

We only encountered a one month delay on delivery of our 125kW natural gas fired generator. Our delay was mostly because of hurricane Katrina having struck the gulf coast. We had to pull some serious string but since we wouldn't be moving in until November of 2005, it didn't really impact us.

Two Words.... (1)

Lunch2000 (701764) | more than 7 years ago | (#16580962)

Fuel Cell

I believe they can produce Natural Gas fuel cells that can produce this kind of output. I think I heard that most of Times Square runs off of two units that reside on the mid floors of a nearby building. Its a great idea becuase usually something incredibly catastrophic has to happen before the natural gas supply is cut. If that happens you probably have bigger problems than your data center going down.

magic number? (0)

Anonymous Coward | more than 7 years ago | (#16580964)

Why is 2,000 kW the magic number? I've worked on several Data Centers, and used anything from a single 150 kW to a few 1,700 kWers. What about 1,000 kW or 3,000 kW? What about gas vs diesel?

Cogeneration (0)

Anonymous Coward | more than 7 years ago | (#16580980)

I would hope that these are cogeneration type plants. There is no sense not using all that waste heat. Besides, most of those data centres need cooling, so you easily have another thermal load you could feed off the cogen plant. The decision as to Otto (spark ignition), Diesel, Rankine (gas turbine), Stirling or other prime mover depends on the mix of heat to electricity needed, but I wouldn't be surprised if nobody thought about cogeneration and just assumed otto/diesel generator sets.

alternate solution (1)

Quadraginta (902985) | more than 7 years ago | (#16581046)

Wrap the entire building in foil. Add a layer of thin dielectric, then another layer of foil.

Must remember to put rubber mats in all the entryways, of course.

Bonus benefit: no need for a security system. Just give the phone number of the county coroner to the janitor, so that if someone tries to break in overnight, the bodies will be cleaned up before anyone arrives for work in the morning.

Skywest Airlines can tell you... (4, Interesting)

DigitalRaptor (815681) | more than 7 years ago | (#16581214)

I was having dinner with an IT guy from Skywest Airlines the other night and he told me in the last big power outage in St. George, Utah where they're based (and I live) their battery backups ran out quickly and there was a hardware problem between their generators and their IT department.

The end result is that all of their servers and network equipment went out for hours, and they had to cancel a whole lot of flights costing the company well over $1,000,000.

You can buy a lot of crappy gas generators at the Lowes across the street for $1M. I think I would have sent two guys there and two guys to the gas station to keep the essentials online.

There is a lot to be said for redundancy. Redundancy is very important. You can't talk about redundancy enough. Seriously, it's better to be twice as redundant than only half as redundant. And three times as redundant is even better than twice!

Re:Skywest Airlines can tell you... (1)

ivan256 (17499) | more than 7 years ago | (#16581350)

The idea of getting a lot of crappy gas generators from Lowes to run in phase gives me a $1million headache.

Re:Skywest Airlines can tell you... (1)

DigitalRaptor (815681) | more than 7 years ago | (#16581506)

Who said anything about running anything in phase?

The last time I checked most of their HPUX server run on 110 volts, or ~maybe~ 220.

If you have 10 mission critical servers, get 10 cheap generators and hook them up on a 1-to-1 basis.

Very cheap, very effective, very easy.

WAY cheaper and easier than canceling $1M in flights.

Re:Skywest Airlines can tell you... (1)

Rob T Firefly (844560) | more than 7 years ago | (#16581398)

Gas stations are pretty useless during a power outage. Their pumps are electric.

Re:Skywest Airlines can tell you... (1)

DigitalRaptor (815681) | more than 7 years ago | (#16581570)

Yes, but very seldom is a power outage so far spread that you can't get to a gas station with power within 10 minutes or so. At least not around here. We have 3 different power companies within 10 miles, and I've never known them to go off at the same time.

Just as bad .... (1)

nbvb (32836) | more than 7 years ago | (#16581244)

One of the biggest delays we had building a new data center was in acquiring the transfer switches .... even once the generators and UPS systems were installed, they're useless without the transfer switches. There was about an 8-month lead time on those, and heaven help you if one of them arrives DOA. :)

from what department? (0, Offtopic)

farker haiku (883529) | more than 7 years ago | (#16581248)

Is it just me or did anyone else read that as so-that's-why-moms-can't-expand-to-meet-needs dept.

I was confused, because I know that Anonymous Coward's mom expands to meet my needs :)

What about the dams... (1)

creimer (824291) | more than 7 years ago | (#16581260)

Seems like the general environmental trend is to tear down the dams built in the 1930's (at least in California). Some of these dams also provide hydro-electric power. While most data centers are looking for on site power, you have to wonder if we're shooting ourselves in the foot by reducing the amount of power being generated from the dams to the grid. New natural gas and nuclear power plants are great if they get built but usually the NIMBY's will come out in force against a new plant.

Re:What about the dams... (1)

Iphtashu Fitz (263795) | more than 7 years ago | (#16581332)

Some of these dams also provide hydro-electric power

I forget where I read it but I recall something about this being one of the key reasons Google was building a huge datacenter complex in Oregon on the Columbia river. Not only does it provide cheap electricity from hydro power dams but the river also provides cooling for the datacenter.

It's about engines... (1)

zipthink (943185) | more than 7 years ago | (#16581274)

Caterpillar & Cummins are the two main sources for diesel generator engines. Caterpillar is a favorite of the US Government and the gov is busy lighting up most of the middle east right now. Both Cummins & Caterpillar are also busy lighting up China. Possible reason why they can't convert used engines... Cummins K series engines, which are popular in gensets... you can't find any used ones right now, because China is sucking up every one on earth. Cat 3516s - KBR is sucking them up to use them for Government projects in Africa & the middle east. Takes a lot of electricity to catch terrorists I guess.

Contra -diction (1)

SuperStretchy (1018064) | more than 7 years ago | (#16581314)

"'This will be a big issue for the next wave of data center builds'"

Interesting.. considering not too long ago an article in /. spoke about how the Datacenter was doomed [slashdot.org]

Steve Ballmers Psyche Profile (0, Troll)

Foofoobar (318279) | more than 7 years ago | (#16581348)

I used to work for a health care provider and managed to see Steve Ballmers psyche profile (before adding into a DB). Naturally I read it. Basically you could guess the issues he has just from reading Slashdot.

October Surprise (1)

hckrdave (588951) | more than 7 years ago | (#16581406)

Think that this could have anything to do with the storm here in buffalo? People here are in a mad dash do get there homes and businesses on backup power ASAP.

These problems do not apply to all vendors (0)

Anonymous Coward | more than 7 years ago | (#16581460)

Typical US centered article on /. ignoring the European vendors that can send you such 2MW generators in couple weeks of a notice. Stop buying only from the 2nd best, people..

Relaying! (1)

NineNine (235196) | more than 7 years ago | (#16581492)

I agree with the article. I certainly wouldn't want to relay on small generators. I might rely on them, but I certain'y wouldn't relay on them.

Slashdot editors: What do they do? Do they really exist? Do people actually get paid for those jobs?

Hamsters! (0)

Anonymous Coward | more than 7 years ago | (#16581602)

This problem is easy to fix, just switch to hamster powered generators.

I know of one company that isn't worried... (1)

xoundmind (932373) | more than 7 years ago | (#16582000)

Sun [sun.com]
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>