Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Vendors Take Blame For Most Data Center Incidents

samzenpus posted more than 2 years ago | from the who's-to-blame dept.

IT 57

dcblogs writes "External forces who work on the customer's data center or supply equipment to it, including manufacturers, vendors, factory representatives, installers, integrators, and other third parties were responsible for 50% to 60% of abnormal incidents reported in a data center, according to Uptime Institute, which has been collecting data since 1994. Over the last three years, Uptime found that 34% of the abnormal incidents in 2009 were attributed to operations staff, followed by 41% in 2010, and 40% last year. Some 5% to 8% of the incidents each year were tied to things like sabotage, outside fires, other tenants in a shared facility. But when an abnormal incident leads to a major outage that causes a data center failure, internal staff gets the majority of blame. 'It's the design, manufacturing, installation processes that leave banana peels behind and the operators who slip and fall on them,' said Hank Seader, managing principal research and education at Uptime."

cancel ×

57 comments

Sorry! There are no comments related to the filter you selected.

Anonymous Cowards takes credit for most (-1, Offtopic)

Anonymous Coward | more than 2 years ago | (#39201945)

FIRST POSTS!

I BLAME OBAMA (-1)

Anonymous Coward | more than 2 years ago | (#39202271)

Barack Obama is a stuttering clusterfuck of a miserable failure.

If stupid hurt, Obama would need a morphine drip.

Re:Anonymous Cowards takes credit for most (1)

eternaldoctorwho (2563923) | more than 2 years ago | (#39202759)

The headline said "blame", not "credit". I think the former is more relevant here.

First Outage (0)

Anonymous Coward | more than 2 years ago | (#39201975)

Let's remember that the first outage was caused by an external force also--a moth [wikipedia.org] .

Whelp. (1)

GmExtremacy (2579091) | more than 2 years ago | (#39202009)

I think it's time to switch to Gamemaker. Have to face the music some day, yes?

correlation is not causation (4, Insightful)

Karmashock (2415832) | more than 2 years ago | (#39202019)

I'm sure outside forces installing things are disruptive. But then are they the primary forces doing installations in general? And if that's the case, then it would be more appropriate to call them simply installation related issues... and that's both common and to be expected.

Install anything new and teething issues tend to crop up.

Re:correlation is not causation (1)

Anonymous Coward | more than 2 years ago | (#39202377)

I think you are 100% correct. I would also consider it the responsibility of the operational staff of a datacenter to properly monitor any external vendors granted access to the datacenter to ensure they don't break anything. I mean, I'm a customer of Datacenter A, I don't really give a crap if the person who took down my network works for Datacenter A or Vendor B that Datacenter A hired. The fact is, my SLA is with Datacenter A, and they are 100% responsible for maintaining the integrity of that datacenter, regardless of what outside forces are involved.

Re:correlation is not causation (1)

Imrik (148191) | more than 2 years ago | (#39204287)

What happens when the person who took down the network works for Vendor C that you hired?

Re:correlation is not causation (1)

starfishsystems (834319) | more than 2 years ago | (#39204163)

Speaking only anecdotally, I've found, on many occasions when installing server applications, that the vendor's installation mechanism breaks the system in some way. These products can't be trusted in their default configuration, yet the nature of software installation entails an elevated degree of trust.

A characteristic example would be a network service which inserts its own startup behavior into one of the standard Unix scripts in /etc/init.d rather than providing its own standalone script. There's really no excuse for this; it's just a mediocre hack by someone who doesn't know or care to do it the right way. But it's exactly here, where the application is provisioned to the system, that the greatest opportunity lies for breakage.

It's also been my experience that contractors who perform installations on site, as well as vendors acting in that capacity, are generally not motivated to do the installation in a clean manner. Sure, it's nice to have someone to blame when things go wrong, but the kind of things that go wrong are often not encountered until the next system upgrade or patch interacts with whatever the installation broke, by which time the outside party is long gone. So usually the local staff take the blame, and in any case they're the ones who have to identify and fix the issues.

Ideally, the remedy for these scenarios lies with the operating system vendor. If the system provided and enforced its own defensive installation API instead of relying on third parties to play nice, then the system vendor and the application vendor could duke it out on their own turf instead of dragging site administrators into the battle.

Can anyone think of an example where this approach is strictly not possible?

Re:correlation is not causation (1)

Karmashock (2415832) | more than 2 years ago | (#39204919)

It's also very very easy to blame the guy that was there for a day and then not there the next to defend himself.

I've run into a few situations where a coworker blamed mistakes they made on someone that was recently fired or only came to the office occasionally.

Why not after all?... what are they going to say in their defense?

Internal staff IS to blame (1, Interesting)

nurb432 (527695) | more than 2 years ago | (#39202029)

They need to monitor and control their vendors/contracts/etc better.

Re:Internal staff IS to blame (2)

chuckinator (2409512) | more than 2 years ago | (#39202053)

but, but, but, THAT'S TOO HARD when passing the buck is so much easier!

Re:Internal staff IS to blame (1)

houstonbofh (602064) | more than 2 years ago | (#39202085)

Is there a category for warning the PHB that it won't work and being told "I don't need excuses! Just get it done!" as that might be the missing 102%...

Re:Internal staff IS to blame (5, Insightful)

Archangel Michael (180766) | more than 2 years ago | (#39202231)

over worked, understaffed, added three projects this month and only closed one that was already in the works. It isn't too hard, or that it can't be done, it is also we don't have the time to do it right because we're still cleaning up the mess from the last three projects that were "critical" and were over budget and late. We'd be outsourced, but the cost of hiring outside vendor is about 10x what in house staff costs, and they would charge more for each project added.

Which is why I no longer try to do things on "low budget" and why everything I look at is Enterprise level. Enterprise level allows me to blame the vendor, because THEY are the ones that are selling this shit to the PHB who doesn't know how ridiculously over simplified the vendor makes it sound.

Re:Internal staff IS to blame (0)

Anonymous Coward | more than 2 years ago | (#39202645)

wise man, i have been in this exact same boat before.

Re:Internal staff IS to blame (1)

houstonbofh (602064) | more than 2 years ago | (#39205385)

wise man, i have been in this exact same boat before.

I think it is a rather large boat...

Re:Internal staff IS to blame (1)

ArsenneLupin (766289) | more than 2 years ago | (#39209399)

I think it is a rather large boat...

With Schettino at the helm...

And in the enterprise world, he's got a golden lifeboat that he can fall into when things turn sour....

Re:Internal staff IS to blame (4, Informative)

sociocapitalist (2471722) | more than 2 years ago | (#39202319)

Nice try -

The reality is that whenever something goes wrong, the vendors/contractors are almost always blamed regardless of who is at fault. It's standard business practice for the customer to bring in a vendor for just this reason - if something goes wrong, they can point at the vendor. The bigger the vendor name the better this works. If you can bring the manufacture(s) in that's best of all. Who can blame you then?

The 'abnormal' incidents where an internal employee is blamed are probably instances where there was absolutely no way for that employee to escape responsibility (ie the syslog entry shows that user logged in, using a one time password token in his possession so that there's no chance of "the vendor has my username and password bullshit", and entering the command 'reboot').

I'm not saying that vendors, contractors and manufacturers don't make mistakes - they're human and from the manufacturer standpoint there are always bugs that are going to cause problems. I'm just saying that the employee / external aspect should be taken into account and thus these statistics taken with a very large grain of salt.

Re:Internal staff IS to blame (1)

eclectus (209883) | more than 2 years ago | (#39202475)

As a vendor, I will attest to this. I have 'fallen on the sword' more than once for employees to save face.

Re:Internal staff IS to blame (0)

Anonymous Coward | more than 2 years ago | (#39270557)

I'm working one now where we were involved as much as Best Buy corporate offices for some guy walking in off the street and buying a refrigerator, then complaining that it doesn't fit in his nook. He didn't ask. He didn't mention it. The stats for the fridge were public knowledge and he could have measured it himself. But he wanted the sales guy to ask "how much space do you have" and recommend the best one. But no, it's my fault because I "should have known" what they meant when they lied to me. I am getting tired of being lied to then being responsible when the lies are uncovered. Is "I don't know" so hard? I just want to get hired into an internal IT department and not have to deal with this anymore.

Re:Internal staff IS to blame (0)

Anonymous Coward | more than 2 years ago | (#39202339)

Sounds like the QA isn't up to par on both ends. Where the contractor does shoddy work(not like bad just not to a specific standard) and the client signs off without checking it.

Re:Internal staff IS to blame (0)

Anonymous Coward | more than 2 years ago | (#39205603)

I see QA being the issue more and more.

Fewer and fewer people have the ability to analyze and test systems that are becoming more and more complex.

Then there are "the shirts" that always want it done yesterday because missing the deadline or install quota messes up the bonus for people "high up".

This business attitude of "bottomline management" seems to drive the "mad rush" to get stuff turned up on schedule no matter how bad it might actually be. That directly impacts the QA process since proper QA takes time and people that understand the systems. What is one of the first things to go when management wants something done now, QA. Sometimes just the time is slashed. Sometimes the people are slashed with the time.

It seems like most business management types today are over-educated money-grubbing bonus-driven hypocrites.

Re:Internal staff IS to blame (3, Insightful)

s73v3r (963317) | more than 2 years ago | (#39202359)

I honestly wonder how many of these incidents blamed on outside vendors are actually the result of something the outside vendor did, and not the result of some manager yelling and screaming loud enough to make the vendor do something to shut him up and not lose business.

Re:Internal staff IS to blame (0)

Anonymous Coward | more than 2 years ago | (#39203451)

> They need to monitor and control their vendors/contracts/etc better.

Ha.

As Einstein once said "No problem can be solved from the same level of consciousness that created it."

There are things which are not in the contracts. When things go wrong, it's very important to have someone to take the blame for you. If said someone is protected by a network of lawyers and has millions to throw at lawyers, well... that is a free pass to irresponsibility IMHO.

Couple that with public service where there's no real accountability to the real boss (the citizen) and it's easy to know why people don't use Linux desktops in organizations, when everybody and his dog knows it's cheaper than the alternatives.

Blame game? (2, Insightful)

swb (14022) | more than 2 years ago | (#39202041)

It sounds like this is just some kind tool to show that "it's not our fault, really" -- but at the end of the day, aren't the internal staff responsible for managing the "outside forces" up to and including setting standards, supervision, etc?

Or is this one of those deals where so much it oursourced that it's easy for everyone to deny culpability?

Re:Blame game? (0)

Anonymous Coward | more than 2 years ago | (#39202219)

The first. It's sometimes said that the difference in cost between store bought and homegrown is what you pay for a convenient punching bag outside your organization.

Re:Blame game? (1)

sociocapitalist (2471722) | more than 2 years ago | (#39202561)

Depends - if you bring in HP or IBM to do provide a solution for you all the way from business requirements, applications development, systems, networking, security, etc, etc, and then at implementation you have IBM, Juniper, Cisco, etc on site to support and something happens...you've covered yourself perfectly. No one can blame you because you got 'the best in the business'.

Re:Blame game? (1)

trevelyon (892253) | more than 2 years ago | (#39211405)

I think that's SAP's entire business model.

Re:Blame game? (0)

Anonymous Coward | more than 2 years ago | (#39203431)

And all I could read was 'deny capability'...

Well duh, cuz they outsource everything. (2, Insightful)

Above (100351) | more than 2 years ago | (#39202091)

Corporate America loves to outsource. Not because it's efficient or cheap, but because it provides someone to blame!

Outsource the network to one firm, the generator to another, the HVAC to a third. Hire temp contract lackeys to staff the place, and rent-a-cops to "guard" it. Then, when something goes wrong, blame them. If it's a big enough issue fire them and replace them with the next batch of people who won't be trained, won't care, and will eventually screw up.

This article isn't illuminating, it's simply restating the design parameters of the system!

Re:Well duh, cuz they outsource everything. (1)

swb (14022) | more than 2 years ago | (#39202663)

This is what I really wanted to post.

And it's not like its even done "intentionally" to find someone to blame, it's just that there is SO much outsourcing and the buck stops...nowhere. Everybody does the least amount they possibly can to keep something from going wrong, because they (well, hell, *I*) know it will because there's inadequate training, documentation, testing, PHBs and Suits screaming about how late projects are, nobody bought enough storage/CPU/bandwidth/amperage and the aforementioned suits/PHBs wont' spend "any more".

It's just fucking endless and the blame just gets shipped downstream, rather than someone wondering if maybe somebody with a brain bigger than the shift knob on their BMW sedan should be in charge.

Re:Well duh, cuz they outsource everything. (1)

dkf (304284) | more than 2 years ago | (#39206451)

Corporate America loves to outsource. Not because it's efficient or cheap, but because it provides someone to blame!

Outsource the network to one firm, the generator to another, the HVAC to a third. Hire temp contract lackeys to staff the place, and rent-a-cops to "guard" it. Then, when something goes wrong, blame them. If it's a big enough issue fire them and replace them with the next batch of people who won't be trained, won't care, and will eventually screw up.

They're forgetting that the one thing they cannot outsource is the overall responsibility for having things working enough to support their business, for if they get rid of that then they've eliminated the need for them to exist at all (and their supplier will simply cut them out of the equation with no ill-consequences). If things keep failing horribly because the people they're outsourcing to suck, it's Corporate America's fault for outsourcing to the wrong people (or outsourcing at all).

Mind you, it might be better to deal with problems through conventional insurance than trying to make the system infallible.

Re:Well duh, cuz they outsource everything. (0)

Anonymous Coward | more than 2 years ago | (#39206575)

Corporate America loves to outsource. Not because it's efficient or cheap, but because it provides someone to blame!

Outsource the network to one firm, the generator to another, the HVAC to a third. Hire temp contract lackeys to staff the place, and rent-a-cops to "guard" it. Then, when something goes wrong, blame them. If it's a big enough issue fire them and replace them with the next batch of people who won't be trained, won't care, and will eventually screw up.

God you're an idiot. First, that's not outsourcing. Second, you're saying that corporations should be massive generalized operations which do everything instead of focusing on delivering a specific product.

For example, internet providers don't build their own routers and switches. Why the fuck would they? Nobody does that.
As another example, they don't build generators. Usually they'll have someone in charge of doing routine maintenance, but why the fuck do you want your Carrier to be in the business of designing and building electrical generation equipment? How about having a company which specializes in making that equipment take care of it, hmmm?
Same with HVAC. You're saying you want your ISP to be in the business of making HVAC systems. Why stop with the HVAC and not mention the building itself? From the sounds of your post, you'd like to see them do the architecture for the facility and have their employees out pouring the foundations and hanging drywall... instead of hiring a qualified structural engineering firm to do it.

I don't understand how the fuck you got up to a +3 Insightful, I guess it just shows how many fuckwads have mod points to burn these days. You obviously have your head square up your ass.

Re:Well duh, cuz they outsource everything. (0)

Anonymous Coward | more than 2 years ago | (#39207029)

Corporate America loves to outsource. Not because it's efficient or cheap, but because it provides someone to blame!

Outsource the network to one firm, the generator to another, the HVAC to a third. Hire temp contract lackeys to staff the place, and rent-a-cops to "guard" it. Then, when something goes wrong, blame them. If it's a big enough issue fire them and replace them with the next batch of people who won't be trained, won't care, and will eventually screw up.

Why is this so? This is not a formula that works. Why do they care that they can "blame" someone when things go wrong? Blame is worth nothing, unless the one blamed can cover the costs. A boss can blame an uneducated cheap tech for a massive failure, and fire him too. But that won't recover the money lost. The cheap tech has nothing worth taking. How come they don't see this?

You can try to blame IBM/HP/etc who has money, but they will surely be able to point out that the failure was not covered by contract, or that your cheap techie didn't follow their manual somehow. They may even sell a solution to the current mess, but they won't cover the cost because you can't pin the blame on them.

Dive deeper... (1)

Anonymous Coward | more than 2 years ago | (#39202119)

80-90% of abnormal incidents caused by vendors was the previous vendors fault.

Banana peels? (3, Funny)

Chemisor (97276) | more than 2 years ago | (#39202171)

It's the design, manufacturing, installation processes that leave banana peels behind and the operators who slip and fall on them

When a company tries to get around minimum wage laws by hiring low-paid monkeys to do their design, manufacturing, and installation, they get exactly what they deserve.

UPS datacenter testing (3, Insightful)

DigiShaman (671371) | more than 2 years ago | (#39202195)

My favorite is getting notifications that all our servers went offline. Now typically, that would be at the network (ISP) level. So come to find out later that the entire facility lost power. Apparently they performed an internally scheduled UPS test without letting us know before hand. Well, they completed the test alright. It was a failure.

In that whole event, we ended up with dirty NTFS volumes that needed to have chkdsk ran and one or two servers with a failed drive in their respective RAID5 arrays. Not happy!

Re:UPS datacenter testing (1)

thebeige (2555996) | more than 2 years ago | (#39202743)

Justified rant!

It's part of the shell game. (2)

L3370 (1421413) | more than 2 years ago | (#39202213)

If you let them in your datacenter, it's your fault if anything goes wrong in there.
If your vendor botched a deployment or delivers a functionally useless product, it's your fault for buying into their marketing campaign and not understanding what you just got yourself into.

But mostly, I think the blame system was by design here...Hire someone else to do the job for everything possible. Fire them/drop contracts when they don't work for you, then file insurance claims to compensate (plus extra if you do it right) for the damages. The trick is to keep the damages rolling as expected--enough to keep insurance revenues up, but not enough so that your premiums adjust to make it unprofitable.

rotating contracts leads to people with no knowled (1)

Joe_Dragon (2206452) | more than 2 years ago | (#39202487)

rotating contracts leads to people with no knowledge of the site and more errors as people get up to speed.

Re:rotating contracts leads to people with no know (1)

L3370 (1421413) | more than 2 years ago | (#39203805)

Which is PERFECT if you don't want things to work correctly in the first place! Good products delivered efficiently become cheap.

Its about profit maximization. More fuckups = more billable hours and expenses to pass on to the customer :)~

I think her name was Abby... (1)

neo-mkrey (948389) | more than 2 years ago | (#39202229)

Abby Normal

Vendors are SUPPOSED to take the blame (0)

Anonymous Coward | more than 2 years ago | (#39202275)

That's why we pay service contracts. If covering your ass wasn't a part of business, we'd all be using free or OSS tools.

Well (1)

M0j0_j0j0 (1250800) | more than 2 years ago | (#39202371)

i don't know but i have been taking these pills, my wife is happy, and they were recommended to me by the Uptime institute as well, so this study must be close.

contractors and sub contractors add middle man (2)

Joe_Dragon (2206452) | more than 2 years ago | (#39202467)

contractors and sub contractors add middle man and overhead.

Some times to the point where a sub may get a job with little to documentation or a job with poor or bad documentation.

Or a sub may hit a issue and have to work though alot middle man off site managers to get things fixed or just be told do as the documentation says and we will have to get a other contract to fix it.

Re:contractors and sub contractors add middle man (1)

Anonymous Coward | more than 2 years ago | (#39204217)

As a contractor I experienced this exact thing in a datacenter. During installation of fiber infrastructure I noticed some anomalies on the scope of work. On site contact pretty much said to do exactly what it said (which was wrong/incomplete) because it was designed by the corporate level 1 engineers. Of course this contact retires and the replacement comes in and blames me for an unfinished job.

You Fail %IT?! (-1)

Anonymous Coward | more than 2 years ago | (#39202689)

fucking percent of Corpse 7urned over previo0sly thought OS I do, because

Lowest Common Denominator (0)

Anonymous Coward | more than 2 years ago | (#39202693)

My experience working at a technical support center for a major OS software vendor is with outsourcing IT staff overseas. These so-called IT professionals were barely functional and created the vast majority of their own problems. I saw major outages in corporations in North America caused by mis-steps in trying to rectify what started as a minor issue. It was always annoying to deal with an IT admin and their issues when his manager constantly whispering into his ear and monitoring the call. Very often asked the tech to ask his manager to go for a coffee break :) as they were not helping the issue.

Problem is outsourcing to the lowest cost without qualifying the abilities of staff.

W!OOET.. FP (-1)

Anonymous Coward | more than 2 years ago | (#39203053)

Deer in the headlights... (2)

billybob_jcv (967047) | more than 2 years ago | (#39203207)

Back in the prehistoric days a group of us were sitting in a bull-pen outside the datacenter. There were big windows on the datacenter wall so we could all ooh & ahh at the blinky lights on the servers and switches. Suddenly, my workstation froze - and when I (and every other person in the bullpen) yelled and looked up, we saw our network admin standing in the datacenter looking back at us with a "What?" look on his face. In his hand was the Ethernet cable he had just pulled out of a core switch...

   

They do the risky work (2)

hawguy (1600213) | more than 2 years ago | (#39203803)

Is this surprising? The vendors/contractors do more of the risky work. When it comes time for UPS maintenance, our vendor comes in to take the UPS offline and do the work. If they screw up when they bypass the UPS, they can take down the datacenter. Likewise, when it comes time to add a new disk tray to the storage system or replace a failed controller board, instead of having our staff do it (who may add one tray every year if that), we have the vendor do it, so there's more chance of him doing the wrong thing and bringing down our storage system -- but there's less chance of the vendor causing a problem than our own staff since the vendor's engineer does this twice a week.

Re:They do the risky work (0)

Anonymous Coward | more than 2 years ago | (#39204101)

Is this surprising? The vendors/contractors do more of the risky work. When it comes time for UPS maintenance, our vendor comes in to take the UPS offline and do the work. If they screw up when they bypass the UPS, they can take down the datacenter. Likewise, when it comes time to add a new disk tray to the storage system or replace a failed controller board, instead of having our staff do it (who may add one tray every year if that), we have the vendor do it, so there's more chance of him doing the wrong thing and bringing down our storage system -- but there's less chance of the vendor causing a problem than our own staff since the vendor's engineer does this twice a week.

I add more than 2 disk trays a week :P as a vender. The point is valid however. The company I work for is trying to get the customers to do more and more of the work. However the customers don't want to take the risk of doing the work. Some one like me might install 2-3 new storage systems a week where the customer might install 2-3 new systems a year.

A matter of control (1)

PPH (736903) | more than 2 years ago | (#39203817)

Quality in a data center, or any facility for that matter, depends on controlling the processes within that facility. If vendors have signed on to working within the procedures developed by the data center operators, fine. There should be minimal problems. But if vendors are allowed on the property to do work not covered by these plans and controls, antics will ensue.

There is nothing inherently wrong with bringing in outside vendors. As long as their function has been planned for. And there is some means to hold those vendors to working within that plan. But all too often, data center managers overlook certain functions in their procedures. Like installation and commissioning new equipment, for example. So when these operations become necessary, people are brought in (or the task is handed to in house techs) with insufficient directions on how to proceed. The difference between vendors and your own techs is that vendors come in familiar with their own equipment, but unfamiliar with data center processes. In house technicians have the opposite problem. They know their way around the facility, but not so much the equipment. Either way, somebody is going to need training.

So, do you train your people on functions that they'll rarely have to perform? Or do you expect vendors to learn your processes when they may not return for months or years.

Who to blame (1)

Skapare (16644) | more than 2 years ago | (#39203939)

But with the possible exception of a meteor strike, there's always someone to blame for a data center problem.

I always blame Anonymous Coward. He's the one that failed to order the meteor sheilds.

Tech support war story here. (1)

sirwired (27582) | more than 2 years ago | (#39210849)

Several years ago, I was working a support case with a major bank. Their remote storage mirroring between BFE, [Southwest State Here] and BFE, [Flyover Country State here] failed, and they wanted to know why. I obtain SAN switch logs from both fabrics and attempt to troubleshoot the issue. The logs revealed that the network ports dropped offline one by one, about 5-7 seconds apart, and then the problem hit the other switch. They came back online one-by-one about three minutes later. The ports were scattered all over the respective switches.

I inform the customer of my findings and am informed that there happened to be somebody working on the cabling in that exact same rack cabinet, but he swears he didn't touch anything having to do with these cables and that the problem MUST be within our hardware. I inform the customer that hardware or software issues do not spread to random ports within a switch, and then to a switch that has NOTHING in common besides a nearby rack cabinet, and ONLY affect a particular group of ports that are otherwise completely randomly spread throughout the switch. (We are talking good old-fashioned light loss here... not some esoteric failure that could be caused by software.)

The customer replies: "We'll be having a "discussion" with that cabling contractor."

I can contest... (0)

Anonymous Coward | more than 2 years ago | (#39211273)

Just last year we had a team of painters come in and paint the inside of the data center.(I have no idea why) One morning we get a call from the monitoring team saying a server went down. Shortly after, another server randomly went down. We go back to the computer room to figure out what was going on and immediately see the painters had covered a server rack with plastic sheets from head to toe. We quickly uncover the rack and hear every fan on all severs screaming for mercy. Later on, watching the security videos, we even saw them walking on top of the covered racks to paint.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?