Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Water Main Break Floods Dallas Data Center

Soulskill posted more than 4 years ago | from the five-thousand-hats-worth dept.

The Internet 230

miller60 writes "IT systems in Dallas County were offline for three days last week after a water main break flooded the basement of the Dallas County Records Building, which houses the UPS systems and other electrical equipment supporting a data center in the building. The county does not have a backup data center, despite warnings that it faced the risk of service disruption without one."

cancel ×

230 comments

Silly rabbit. (4, Insightful)

migla (1099771) | more than 4 years ago | (#32489456)

There should always be duplication of critical components of a system with the intention of increasing reliability of the system, usually in the case of a backup or fail-safe.

Re:Silly rabbit. (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32489488)

And the funding comes from where?

Re:Silly rabbit. (2, Insightful)

migla (1099771) | more than 4 years ago | (#32489542)

>And the funding comes from where?

If Dallas having a data center is not that critical, then never mind. Otherwise, from taxes, where civilized civilization usually comes from. But actually I just put that first post there to spoil it for prospective first posters. It was a copy/paste from the start of the wikipedia article on redundancy(engineering).

Re:Silly rabbit. (2, Insightful)

Anonymous Coward | more than 4 years ago | (#32489658)

And the funding comes from where?

A redundant source of income that is.

Sadly, no matter how you design a system there is always a single point of failure. Just depends on how much you want to spend and what you want to risk going south.

Re:Silly rabbit. (0)

Anonymous Coward | more than 4 years ago | (#32489676)

Oooh you're like a geeky Kwai Chang Caine who wears a star trek shirt and quotes The Highlander.

Re:Silly rabbit. (5, Funny)

jo_ham (604554) | more than 4 years ago | (#32489520)

This is Texas - God is their backup solution.

Every night they pray for no hardware failures.

Re:Silly rabbit. (5, Funny)

Mordok-DestroyerOfWo (1000167) | more than 4 years ago | (#32489628)

God is their backup solution.

If this is true, shouldn't they have been prepared for a flood?

Re:Silly rabbit. (1, Funny)

vlm (69642) | more than 4 years ago | (#32489666)

If this is true, shouldn't they have been prepared for a flood?

Their mistake was preparing for the burning bush, from the oil spill, instead of the flood.

Re:Silly rabbit. (3, Funny)

gyrogeerloose (849181) | more than 4 years ago | (#32489720)

Their mistake was preparing for the burning bush, from the oil spill, instead of the flood.

They should have known better, having already passed off their burning Bush to the rest of the United States back in 2000.

Re:Silly rabbit. (2, Funny)

bertoelcon (1557907) | more than 4 years ago | (#32490052)

You weren't supposed to send him back though.

Re:Silly rabbit. (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32490266)

Hey, give us some credit... we TRIED to send him back 4 years earlier, but we couldn't get everyone to agree.

Re:Silly rabbit. (2, Funny)

DigiShaman (671371) | more than 4 years ago | (#32489772)

Supposedly, they have an employee named Noah responsible for off-site backups. Ask him.

Re:Silly rabbit. (1)

FooAtWFU (699187) | more than 4 years ago | (#32489874)

No, the big floods are officially off since the Noah incident. Fire from heaven, on the other hand... now we're cookin'.

Re:Silly rabbit. (2, Funny)

theshowmecanuck (703852) | more than 4 years ago | (#32490290)

They didn't care. Most of the data referred to Thomas Jefferson.

Re:Silly rabbit. (1)

plopez (54068) | more than 4 years ago | (#32490358)

Even that would have only rescued two servers of each kind.

Re:Silly rabbit. (-1, Troll)

Anonymous Coward | more than 4 years ago | (#32489636)

if (politics == LIBERAL) then stereotype();

Helen Thomas, is that you?

Re:Silly rabbit. (1)

jo_ham (604554) | more than 4 years ago | (#32489836)

if (poster == HUMOURLESS) then fuddyduddyresponse();

To be "Fair and Balanced" (tm), I'll throw in "I asked Obama for change and he gave me a quarter, a nickel and two pennies".

Re:Silly rabbit. (1)

plopez (54068) | more than 4 years ago | (#32490374)

More than I ever got from a Republican. BTW, I know you are trolling.

Re:Silly rabbit. (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#32489788)

Are you just trying to insult sincere believers with this smear? Because that is NOT how praying works. No SERIOUS Christian actually tries to backup data by praying to God, okay? So I don't even know what you're trying to prove with your bigoted smear against sincere Christians.

In Texas, we are proud of our faith. But that does not mean we use fath to backup data centers. What we're proud of is the work that God has done in our lives, and the healing power He has brought to those in need.

Let me just ask you this: have you ever read any serious works by any serious theologians about the nature of God an prayer? Heck, if you had read C. S. Lewis's Mere Christianity, you wouldn't be spreading this toxic lie about Christians and data backup methods. (Of course, I wouldn't recommend disbelievers read that today -- Mere Christianity has been superceded by much better, more solid theology.)

How about taking Christians seriously for once instead of making stupid, uninformed jokes about them?

Posting AC because of the massive anti-Christian bigotry on Slashdot.

Re:Silly rabbit. (5, Interesting)

Anonymous Coward | more than 4 years ago | (#32489848)

In Texas, we are proud of our faith.

Which is incredibly funny, especially considering that pride is considered one of the deadly sins (along with lust, greed, gluttony, and several others that Texans are well known for). Apparently too many of you are too busy thumping your chests about how you are such good Christians to even stop to consider what it really means to be a good Christian.

LOL AC (1)

DriedClexler (814907) | more than 4 years ago | (#32490000)

So, when that joke went over your head, did it make a WHOOSH!, or was it more of a buzzing sound?

Re:Silly rabbit. (4, Interesting)

jo_ham (604554) | more than 4 years ago | (#32490004)

It is quite possible to take Christians seriously and still make jokes about them. Some of the most devout Christians I know make jokes about their faith and God all the time.

One of the defining human characteristics is being able to laugh at yourself.

I make similar jokes about Apple users, Linux users, Football fans, people who drink Mountain Dew, atheists, rock fans, sci fi nerds, people who watch reality TV...

Humour is part of human character. You appear to be missing yours. Perhaps you should pray for it to return.

Re:Silly rabbit. (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32490214)

I'd like to see your joke about the atheist, drinking Mt Dew, while watch his favorite rock band during a football halftime show, while using his iPad to schedule his Linux DVR box for the next Stargate.

Re:Silly rabbit. (3, Informative)

Crudely_Indecent (739699) | more than 4 years ago | (#32490032)

This is Dallas, where the City Council is doing such a good job, they're trying to get 270% raises and to double the length of their terms.

Re:Silly rabbit. (0)

Anonymous Coward | more than 4 years ago | (#32490328)

I know you're joking but Dallas is actually one of the few places in Texas with a strong Democrat majority

Re:Silly rabbit. (4, Interesting)

vlm (69642) | more than 4 years ago | (#32489522)

There should always be duplication of critical components of a system with the intention of increasing reliability of the system,

Yeah that might be the intent, but it only works if the combined reliability is higher than individual reliability. Transfer switches I'm looking at you! I have worked at numerous facilities with data centers, and inevitably the transfer switch is less reliable than either wall AC power, or the diesels (youch!). Yes I know exactly what I'm saying, that at every facility I've worked at, power reliability would have been higher without the transfer switch and the generators. But its politically incorrect as the rare wall AC power failure would be unacceptable unless we spent money on switches and gens. As long as you spend money on switches and gens, any low level of reliability is acceptable.

Your mileage may vary, maybe coastie cities have less reliable power. Don't know.

Re:Silly rabbit. (2, Interesting)

headhot (137860) | more than 4 years ago | (#32489856)

Transfer switches are the bane of my existence. I work for a major MSO, and every site we have lost has been to a transfer switch problem. Equipment with 2 ps is the best solution, so you have have 2 sets of transfer switches, UPS, Generator, and Mains.

Re:Silly rabbit. (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32489912)

You no doubt have a UPS hold the data-center while the generators come up to speed. I know it might not be fun but what about a manual cut switch and just size the UPS to last long enough for whoever is on call to get there? Obviously you have to have at least one other person who also gets the page a responds if the primary on call does not indicate they have gotten to site after a set amount of time; and you want to make sure the jars in the UPS are tested often.

This is what we do and what most of the bigger shops I have seen do. We had an auto-switch but after its repeated failures determined it was best to remove it.

Re:Silly rabbit. (0)

Anonymous Coward | more than 4 years ago | (#32489532)

Thanks asshole. And I should be getting a blowjob right now from a hot blonde with big tits and an ass that tastes like french vanilla ice cream. Unfortunately, should doesn't get you very far...

Texas doesn't need backup. (4, Insightful)

copponex (13876) | more than 4 years ago | (#32489644)

In a state as blessed as Texas, they were told that God would provide protection against acts of God. I imagine many of the faithful are confused, especially when Jesus day is only a few days away.

Maybe they didn't execute enough retarded people this year?

http://en.wikipedia.org/wiki/Jesus_Day [wikipedia.org]

Re:Texas doesn't need backup. (2, Funny)

Anonymous Coward | more than 4 years ago | (#32489674)

Maybe they didn't execute enough retarded people this year?

They'll get around to you, don't worry.

OMG (1)

copponex (13876) | more than 4 years ago | (#32490096)

Jesus, is that you?!?!

Re:Texas doesn't need backup. (4, Funny)

PopeRatzo (965947) | more than 4 years ago | (#32489818)

While Governor of Texas, Former President George W. Bush, signed a bill into law proclaiming June 10, 2000 to be Jesus Day

Wow.

This year, they decided to re-write textbooks to eliminate the "liberal bias".

You know what? Maybe the Dallas datacenter going out without a backup is a blessing in disquise.

You think we could get Texas to secede if we asked them real nice? We can give them Arizona and a state to be named later, as long as they promise to never tell anyone that they were once part of the United States.

Re:Texas doesn't need backup. (0)

Anonymous Coward | more than 4 years ago | (#32489906)

You think we could get Texas to secede if we asked them real nice? We can give them Arizona and a state to be named later, as long as they promise to never tell anyone that they were once part of the United States.

That wouldn't work -- existing historical records show that Texas has been part of the United States, and people would always be available to read those, and they wouldn't ignore them just because Texas claimed something in contradiction of all the evidence.

By the way, what's wrong with re-writing the textbooks to eliminate the hardcore pro-materialist bias?

Re:Texas doesn't need backup. (2, Insightful)

copponex (13876) | more than 4 years ago | (#32490188)

By the way, what's wrong with re-writing the textbooks to eliminate the hardcore pro-materialist bias?

Pro-materialist bias? Is that what you ingrates are calling science and history now?

Re:Texas doesn't need backup. (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32490316)

Hey, I don't have to listen to spewing your blatant pro-reality propaganda. I like my fantasy world perfectly fine, thank you very much.

Re:Texas doesn't need backup. (1)

besalope (1186101) | more than 4 years ago | (#32490346)

You think we could get Texas to secede if we asked them real nice? We can give them Arizona and a state to be named later, as long as they promise to never tell anyone that they were once part of the United States.

That wouldn't work -- existing historical records show that Texas has been part of the United States, and people would always be available to read those, and they wouldn't ignore them just because Texas claimed something in contradiction of all the evidence.

Just send a note to Mini-Truth and they'll take care of it. Because he who controls the present controls the past, and he who controls the past controls the future.

We are at war with East Asia, we have always been at war with East Asia.

Re:Texas doesn't need backup. (1)

couchslug (175151) | more than 4 years ago | (#32490390)

"You think we could get Texas to secede if we asked them real nice? We can give them Arizona and a state to be named later, as long as they promise to never tell anyone that they were once part of the United States."

That's actually not a bad idea. The US is a bit large to be one country, political polarization means people who are dire enemies must fight for power (producing results no one likes), and it could be cut in two or three sections with no great loss. It would be less menacing to the rest of the world.

We've gotten used to being one country, but now we don't have anything in common. "Diversity" isn't bad, but there is no reason everyone should be forced to yield to everyone else when there are such profound differences in opinion, politics, and culture. The place has gotten too huge to manage well, so it is managed badly.

Re:Silly rabbit. (1)

QuantumRiff (120817) | more than 4 years ago | (#32489796)

They probably have lots of tapes, and backups. the data was not harmed, it was safe, it was just not available.

Re:Silly rabbit. (1)

couchslug (175151) | more than 4 years ago | (#32490204)

People often refuse to learn absent being bitch-slapped by outcomes they chose not to prepare for.

When "what to do" is so well-known, refusal to do it merits scorn, contempt, and frequent use as an example to others.

Re:Silly rabbit. (0)

Anonymous Coward | more than 4 years ago | (#32490432)

Really? Always?

What the heck does critical mean? Was there loss of life when this failed? Was there data loss?

How much did it cost? What was the chance of this disruption happening? Of any disruption of its type?

Was it worth it to buy a backup and keep it ready, maintained, and on standby 24/7/365?

I'm just saying...the attitude that every time I buy a new server, I need to buy two more is a bit...ridiculous. And the cost of keeping backups warm and good to go is HUUUGE compared to what it costs me just to keep a server in a rack. Especially with the extra testing and maintenance.

Yes--it's part of the price of enterprise business--but saying "always"...well...you know what... Some customers only want to pay $4 a month for their service. Other will pay $25. Guess which ones get their data backed up...

Shit happens (2, Interesting)

Anonymous Coward | more than 4 years ago | (#32489462)

Sometimes it's cheaper to deal with it when it happens than to take precautions.

Re:Shit happens (0)

Anonymous Coward | more than 4 years ago | (#32489494)

Maybe now the idea of outsourcing their work another Data Center company doesn't sound half bad.

Re:Shit happens (2, Insightful)

KevinKnSC (744603) | more than 4 years ago | (#32489512)

I don't disagree with you, but I strongly suspect this will be one of those times that it really would have been worth it to take precautions.

Re:Shit happens (1)

deprecated (86120) | more than 4 years ago | (#32489530)

The video from your family reunion in 1992 doesn't have a high availability requirement so you are correct for a teeny-tiny instance of cases. The criminal justice system of Dallas - maybe not in the same category.

Re:Shit happens (2, Interesting)

vlm (69642) | more than 4 years ago | (#32489604)

The criminal justice system of Dallas - maybe not in the same category.

No. Real high availability is like the AC power to the respirator in a hospital surgical room.

The criminal justice system of Dallas was closed exactly one week ago today for the holiday. Giving the parole officers, judges, juries, attorneys off is no big deal. The inmates, unless they had a trial scheduled today, frankly probably won't know the difference.

Re:Shit happens (0)

Anonymous Coward | more than 4 years ago | (#32489620)

The data center was offline from Monday to Friday. Everybody but the data center staff and the PR guys takes a week off, all deadlines are extended by one week and things are back to normal in at most two weeks. What's the obsession with high availability? It's not like the public administration is known for snappy response times anyway.

Re:Shit happens (1)

Kingrames (858416) | more than 4 years ago | (#32489546)

Yeah, like that oil spill.

Re:Shit happens (2, Interesting)

Monkeedude1212 (1560403) | more than 4 years ago | (#32489640)

I have not seen a case with a Data Center where this has EVER been the case.

Essentially, look at it this way:

Everything gets destroyed. You have to buy new ones. Then you have to start all over. It took you 3 days to get operational again, and now you've got to some-how start over from scratch.

Or what they could have done:

Everything gets destroyed. You already bought a spare system just in case, and had it stored off-site. For about a negligable amount, you went through the effort of backing up the data once a month, to either a spare hard drive, or to the old computer your IT admin has in his basement that he never uses. It'll take you 3 days to get operational again, but at least you'll have that data somewhere and you don't have to play catchup.

Or if they wanted to go crazy

Everything gets destroyed. They had a spare system set up with full redundancy in case this kind of disaster. The cost was high when setting it up, but you have essentially no down-time. Ideally, the 3 days you saved will cover the cost of the damaged equipment. Or if the data is particularily sensative, that is the cost you pay to keep it safe.

Re:Shit happens (1, Informative)

Anonymous Coward | more than 4 years ago | (#32489804)

No data was destroyed, they regularly made backups to an offsite facility. Additionally the flood only affected the "mechanical" room, no servers were destroyed or damaged. Getting back on their feet from that incident isn't "starting over from scratch" by a long shot. Also, the data center users were reminded of the convenience that the electronic data processing affords them. Think of it as an unscheduled admin appreciation week. All the urgent stuff was handled manually and the rest, well, wasn't urgent.

Re:Shit happens (1)

Monkeedude1212 (1560403) | more than 4 years ago | (#32489828)

Ah, well if the data is backed up than no problem. My mistake

Re:Shit happens (1)

Flea of Pain (1577213) | more than 4 years ago | (#32489692)

Tony Hayward? Is that you?

Re:Shit happens (1)

treeves (963993) | more than 4 years ago | (#32489902)

No, he still doesn't "have his life back" [huffingtonpost.com] yet, so I'm sure he's not posting on Slashdot today.

Re:Shit happens (1)

couchslug (175151) | more than 4 years ago | (#32490414)

"Sometimes it's cheaper to deal with it when it happens than to take precautions."

It certainly makes for enhanced Schadenfruede.

Backups (1)

vlm (69642) | more than 4 years ago | (#32489464)

The county does not have a backup data center

Traditionally, facilities with that level of management oversight don't even have backups. No not backup centers or facilities or hardware, I'm talking about backup tapes. Am I right or wrong?

I know rotating hard drives don't like immersion. Are SSDs any better, or do they die from galvanic corrosion? It might be an interesting race, which will survive longest.

Where where there heads? (0)

Anonymous Coward | more than 4 years ago | (#32489466)

Not in the cloud

Sewer line (4, Funny)

Anonymous Coward | more than 4 years ago | (#32489470)

At least it wasn't a sewer line break, that would be real shitty.

Seriously? (1)

Monkeedude1212 (1560403) | more than 4 years ago | (#32489478)

The county does not have a backup data center, despite warnings that it faced the risk of service disruption without one.

That alone spells disaster. If I had a nickel for everything that could go wrong in a data center, I could buy a new data center.

Re:Seriously? (1)

Kingrames (858416) | more than 4 years ago | (#32489540)

No, because inflation would render those nickels worthless.

Re:Seriously? (1)

vlm (69642) | more than 4 years ago | (#32489646)

No, because inflation would render those nickels worthless.

Recently, the value of the nickel content in a nickel briefly exceeded 5 cents. If the cost of everything, including nickel metal, exploded up by a factor of ten, then you'd be able to buy nickels at the bank or whatever for 5 cents and melt them into nickel ingots worth 50 cents. You'd be OK. Metals "always" hold value during inflation. Even hyperinflation, in that case especially if the metal happens to be cast and jacketed lead...

Re:Seriously? (1, Insightful)

Monkeedude1212 (1560403) | more than 4 years ago | (#32489672)

The problem is that not all nickels are full nickel anymore, just like not all pennies are made from pure copper.

I know people who go searching for the older coins simply so they can melt them down into ingots worth more than the cash value of the coins.

The newer stuff just isn't worth it though.

Re:Seriously? (1)

DigiShaman (671371) | more than 4 years ago | (#32490012)

It might get to the point where cheap metal is worth more than the Penny itself. When that happens, I expect the Nickel (5 cents) to be the lowest denomination issued from that point forward.

I'm not sure how financial institutions will feel about the new rounding off to the nearest 5 cents though.

Re:Seriously? (1)

SydShamino (547793) | more than 4 years ago | (#32490298)

The metal in a penny is already worth more than a penny. Congress solved this dilemma by making it illegal to melt down pennies to resell the metal.

There's a plan to move all cash transactions to 5 cents by making them all round up or down appropriately, while having all electronic transactions remain accurate to the penny. But, as you can guess, certain members of Congress who are more attached to sentimental ways vowed to save the American penny, and so they instead outlawed melting them down as a I mentioned above.

Meanwhile (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#32489536)

Linux's marketshare falls by 99%. Windows Rocks, -1, troll approved!

Tested Backups? (1, Redundant)

Ichijo (607641) | more than 4 years ago | (#32489538)

Officials stressed, however, that no data was lost during the shutdown because of automatic nightly backups at an off-site location.

Next question. Has the backup and restore process been tested?

Re:Tested Backups? (2, Insightful)

Fallen Kell (165468) | more than 4 years ago | (#32489570)

Probably not....

Re:Tested Backups? (2, Funny)

NevarMore (248971) | more than 4 years ago | (#32489758)

It has now.

Re:Tested Backups? (1)

eulernet (1132389) | more than 4 years ago | (#32489846)

And more importantly: are they waterproof ?

Re:Tested Backups? (1)

SydShamino (547793) | more than 4 years ago | (#32490304)

Without taking the production system down, what would they practice restoring to? The backup system?

Something like that happened to us... (2, Insightful)

snowraver1 (1052510) | more than 4 years ago | (#32489548)

About a year ago...

At the time, we had all our WLAN connections carried through Bell Canada VIA Frame circuits. I guess many of these circuits went through a facility in Edmonton. This facility was being rennovated, and some poor worker drilled through a pipe that they thought was empty... As it turns out, that pipe was filled with pressurized water, and so the water started spraying everywhere/everything and ended up taking down all our frame services north of Edmonton (about 30 sites). It took about 2 (very stressful) days for Bell to route our frame circuits through another data center.

It sucked, but I really feel bad for the poor guy that drilled through the wrong pipe.

Re:Something like that happened to us... (1)

snowraver1 (1052510) | more than 4 years ago | (#32489610)

BTW... WLAN - Our Greater LAN that spans thousands of KMs, not the wireless...

Re:Something like that happened to us... (2, Informative)

molo (94384) | more than 4 years ago | (#32489810)

I think you mean WAN.

-molo

Re:Something like that happened to us... (2, Interesting)

vlm (69642) | more than 4 years ago | (#32489762)

At the time, we had all our WLAN connections carried through Bell Canada VIA Frame circuits.

Was it ever possible to buy a frame relay switch that doesn't have an automatically rerouting ATM backbone as the underlaying technology? As far as I know, the answer is no.

ended up taking down all our frame services north of Edmonton

Somebody's got a single point of failure in Edmonton. Huge design mistake, not inherent technological limitation.

I worked for a carrier for many years that was properly designed. PVCs would drop over the dead trunk and reroute over the live trunk transparently, assuming you were using a real packet based protocol like TCP/IP and not SNA/SDLC where a dropped packet means reset the SNA communications controller...

It sucked, but I really feel bad for the poor guy that drilled through the wrong pipe.

The dude with the drill might be in ice cold Edmonton water. But the dude whom designed in the single point of failure in Edmonton is probably in really hot water.

Re:Something like that happened to us... (0)

Anonymous Coward | more than 4 years ago | (#32489862)

>But the dude whom designed

Who.

Friendly Advice (1)

cpaalman (696554) | more than 4 years ago | (#32489562)

Roof -> generators Above water level (1st floor and up) -> Electrical stuff Basement -> rats and other Texas style vermin ??? profit

Re:Friendly Advice (1)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#32489726)

Unfortunately "other Texas style vermin" likely includes fire ants. Those bastards are mean, and attracted to electrical hardware [extension.org] ...

Re:Friendly Advice (0)

Anonymous Coward | more than 4 years ago | (#32489808)

Electrical stuff Basement -> rats and other Texas style vermin

Those Texas Republicans are a drag, no doubt about it.

Who's idea... (4, Insightful)

Pharmboy (216950) | more than 4 years ago | (#32489594)

Whose bright idea was it to put the UPS and backup systems in the portion of the building that is first to be flooded, and the most devastated in just about any natural disaster, AND the least accessible afterward? Sounds like something a government would do....

Re:Who's idea... (4, Insightful)

NewWorldDan (899800) | more than 4 years ago | (#32489688)

.... and it's also the part of the building that's easier to cool and isn't in demand for office space. A lot of businesses put their data centers in basements. I've seen a few places that built dedicated buildings for the data center, but usually, cost dictates that they stick it where they can.

Frankly, while it will be a pain in the butt for 2 weeks, they'll get through this just fine. If they had a redundant data center, people would be whining about the waste of money and so on. There's no right answer here.

Re:Who's idea... (0)

Anonymous Coward | more than 4 years ago | (#32490438)

.A lot of businesses put their data centers in basements.

Is this a US thing?

Contrary to popular belief, rack cabinets fit fine in the goods elevator. UPS systems come in pieces ("some assembly required").

Re:Who's idea... (3, Interesting)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#32489698)

It's a terrible plan; but putting the heavy stuff in the basement makes life easier(unless your building was purpose-built, or you have serious renovation funds, putting thousands of pounds of UPSes on one of the upper floors isn't always one of your choices). And, more generally, there seems to be this perverse part of human nature that clings to the atavistic belief that stuff you don't have to look at isn't an issue. We put the ugly, heavy, parts of the system in the basement, we bury dangerous chemical wastes(where it is virtually impossible to inspect them for leakage, and all leakage goes directly to the water table) rather than putting them on the second floor(where discovering leaks is as easy as walking through the first floor)...

Don't know (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32489740)

Maybe they were more worried about tornadoes than floods there.

Of course the police/para/military mindset is bunker/underground everything critical as much as possible as well. I know my county just built a new jail/police station a few years ago and more than half that sucker is underground, you can see it from the road so it was easy to see while it was being constructed. It's built more like some fort than what you might think of as a traditional jail or copshop facility. Well sort of, it is half underground now, what they did was build it on level ground, the first level, then they bermed that over and built the next level you can still see now on top of it.

Re:Who's idea... (1)

night_flyer (453866) | more than 4 years ago | (#32489766)

I would guess a gov't worker

Re:Who's idea... (0)

Anonymous Coward | more than 4 years ago | (#32489830)

Yeah. Now just envision a basement data center hosting all of the nation's computerized health care records. Hooray for petty half-wit autocrats!

Re:Who's idea... (3, Interesting)

Nemesisghost (1720424) | more than 4 years ago | (#32489868)

Actually most buildings in Texas are built where the basement is the most structurally sound part of the building. This is because the biggest 'natural' disaster are tornadoes. This is especially true of older buildings(like most government buildings). There's only one state run building I know of that was built any other way. It's one of the state backup centers in West Central Texas. They spent 2-3x the amount of a comparable building so that the top floor could withstand hurricane force winds. They didn't do it because it was easy or economical, they did it because if you want to remain safe during a major storm, you head to the basement.

How's the Dallas sewer system? (1)

twoallbeefpatties (615632) | more than 4 years ago | (#32490160)

Some of the replies are saying the basement is a fine place to put these things, I can appreciate that. I'm wondering if this is less a problem of basement and more a problem of sewer. I don't know about Dallas, but I live in Houston, and Houston has the most terrible sewers I've ever seen. Storm drains are located very far away from each other, and the amount of space underground for carrying water seems to be miniscule. Once a year or so, we get just a day's worth of heavy storms which result in flash flooding - which seems like it would have all just washed into the drains in any other major city I've ever been in. Maybe the greater issue here is whether or not those basements were properly protected by drainage to keep the waters out.

Re:How's the Dallas sewer system? (0)

Anonymous Coward | more than 4 years ago | (#32490326)

Some of the replies are saying the basement is a fine place to put these things, I can appreciate that. I'm wondering if this is less a problem of basement and more a problem of sewer.

The Dallas sewer system is a bit of a mix. Some parts are so awesome that people take tours of them (seriously; there's a group that goes out on Saturdays in the early Spring and late Fall to explore them), while other parts are inadequate and poorly built). Most people, even those that live in Dallas, aren't aware that there is also a significant "underground city" below the main city downtown (you can travel between just about any two downtown buildings without going outside into the sunlight, and the underground malls are nicely air conditioned, or at least used to be, with miles of walkways large enough to drive vehicles), but everyone once in a while (decade or so) there are flooding issues even there.

Re:Who's idea... (1)

Shivetya (243324) | more than 4 years ago | (#32490218)

Well many businesses locate such stuff at the lowest levels to protect it from severe weather.

Yet I have met many people in IT from small government IT setups who never had backups, some even had charged water sprinkler systems in their data centers. Many vendors sell systems without backups because if the cost of the backup systems were included they would be too expensive. Plus never overestimate the ability of people to put off to the next group to follow to finish their job or suffer from the incomplete state. Standard fare from politicians at local and upper levels.

Prepared government or small government pick one (4, Insightful)

zerofoo (262795) | more than 4 years ago | (#32489750)

Data center floods, Katrina, and the BP oil spill the Gulf. All have one thing in common - a government that was not prepared.

I chuckle when I hear my more conservative friends complain about the level of competence or disaster response times we find in government. These are the very same people who want less government and lower taxes.

You can not have a small, cheap, government that is staffed by geniuses and prepared for every possible problem. Smart people cost money, resources cost money.

While this flood was an avoidable occurrence, do taxpayers really want to pay for redundant EVERYTHING at the local, state, and federal government? Probably not.

-ted

Re:Prepared government or small government pick on (2, Informative)

Anonymous Coward | more than 4 years ago | (#32489886)

In what world do you live in that the government is ultimately responsible for a massive oil spill that was caused by a corporation?

While the government may be responsible for the fact that the regulations weren't followed every step of the way, the company that is behind the actions on the rig are ultimately fully responsible for not following those rules and regulations. I'm glad that the government can help in any way possible, but BP should foot the bill.

Re:Prepared government or small government pick on (1)

SydShamino (547793) | more than 4 years ago | (#32490310)

Tell that to Bobby Jindal.

Re:Prepared government or small government pick on (1)

savanik (1090193) | more than 4 years ago | (#32489944)

It's entirely possible that the reason they're complaining is that they want a smaller government with less things they're responsible for, so they'll have fewer things they're capable of screwing up, and want the things that government does taken over by private industry. Having a larger government does not necessarily mean having a more prepared government - it generally means having more layers of bureaucracy and more people who's jobs are not directly tied to their performance.

While I don't promote handing over the control of our jails over to private industry, you could certainly hand over the server management to people who actually know what they're doing and have a vested interest in making sure that there's a stable, solid datacenter behind the servers. You could get rid of some government sysadmins, and it'd be cheaper than buying a brand new datacenter as well.

Re:Prepared government or small government pick on (1)

SydShamino (547793) | more than 4 years ago | (#32490322)

Having a larger government does not necessarily mean having a more prepared government - it generally means having more layers of bureaucracy and more people who's jobs are not directly tied to their performance.

And this is different from larger industry how?

Re:Prepared government or small government pick on (1)

shipbrick (929823) | more than 4 years ago | (#32490200)

money != productivity [youtube.com] "

Well I have this here deed.. (1)

gearloos (816828) | more than 4 years ago | (#32489974)

Well sirs I have this here deed that says I own the Cowboys Stadium. What do you mean it looks like I typed it up? You got any proof I don't own this here stadium? OK Then. lol can see it now.

Someone forgot the rules... (3, Insightful)

U8MyData (1281010) | more than 4 years ago | (#32490042)

From the movie Contact: "First rule in government spending: why build one when you can have two at twice the price?" Load balanced and replicated no less. Basement + critical systems = bad idea. Not only for flooding potential, but bad sewer systems as well. Yes been there done that. Curious, is there a systems engineer that could make a good argument for building data center infrastructure in a basement. Two points already for shielding from severe weather; anything else?

Re:Someone forgot the rules... (4, Insightful)

Renraku (518261) | more than 4 years ago | (#32490216)

Climate control is easier in the basement. You can build big fuckoff heat exchangers that go under ground level and surface however far from the building you want them to surface.

Simpler wiring plans because you don't have to run big industrial power cables up to the top floor and the data lines don't have to go far to get to the basement.

All that being said, below-ground server rooms should have some method to be able to seal themselves off from the rest of the world in case of flooding. Perhaps the elevator or hallway door can form a decent seal, whereas everything else is already as sealed as it can be. Perhaps sealing everything also cuts power so nothing overheats.

Two out of three ain't bad, right? (1)

peacefinder (469349) | more than 4 years ago | (#32490210)

Confidentiality? Check!
Integrity? Check!
Availability? ... Can I borrow a pen?

The basement? (1)

fotoguzzi (230256) | more than 4 years ago | (#32490320)

That's where all the Kennedy Assassination records were stored!

Where to locate you data center (1)

plopez (54068) | more than 4 years ago | (#32490342)

I was at a $BigSwitchMakerResearchSite and they located their servers on the second floor of a 3 story building. They had a flood in the area in the past which wiped out their first floor data center. Since this place was also in tornado country some one made a sane and sensible decision to locate all the important servers on the second floor away from the outer walls.

Take heed all you junior sysapes.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...