Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Emory University SCCM Server Accidentally Reformats All Computers Campus-wide

Soulskill posted about 4 months ago | from the that-qualifies-as-a-bad-day dept.

Security 564

acidradio writes: "Somehow the SCCM application and image deployment server at Emory University in Atlanta accidentally started to repartition, reformat then install a new image of Windows 7 onto all university-managed computers. By the time this was discovered the SCCM server had managed to repartition and reformat itself. This was likely an accident. But what if it weren't? Could this have shed light on a possibly huge vulnerability in large enterprise organizations that rely heavily on automated software deployment packages like SCCM?"

cancel ×

564 comments

Sorry! There are no comments related to the filter you selected.

Cool (5, Funny)

rossdee (243626) | about 4 months ago | (#47024905)

Sounds like a good way to get rid of Malware

Re:Cool (1)

TemperedAlchemist (2045966) | about 4 months ago | (#47024931)

inb4 the whole system was compromised

Re:Cool (5, Interesting)

Anonymous Coward | about 4 months ago | (#47024977)

Unfortunately, SCCM [wikipedia.org] also supports Linux and Mac OSX clients. I wonder whether it tried to install Windows 7 on them also? Users would be really pissed to discover their Mac/Linux box was now lurching under Windows...

Re:Cool (2)

_Shad0w_ (127912) | about 3 months ago | (#47025327)

Only if their machine was part of the University's Active Directory infrastructure, as far as I know. Just being on the same network wouldn't be enough.

Re:Cool (0)

Anonymous Coward | about 3 months ago | (#47025389)

SCCM supports PXE, so anything on the network with the capability to boot from the network may have received a stock Windows image. Hopefully they had the network properly VLANed, or it may very well have tried to reformat students' personal computers.

Re: Cool (5, Funny)

Anonymous Coward | about 3 months ago | (#47025331)

I worked at Emory for years and I have no doubts this was sheer incompetence not sabotage.

mac systems may not even boot with Partition types (2)

Joe_Dragon (2206452) | about 3 months ago | (#47025367)

mac systems may not even boot with the old Partition tables that are needed for older NON EFI systems that windows runs on.

also the Mac os Recovery Partition may even be wiped out.

Re:Cool (0)

Anonymous Coward | about 3 months ago | (#47025393)

This is the very example needed to illustrate why "re-imaging" machines should not be done without a confirmation by the user of the machine.

I'd really hate to come to work one day and see that all the stuff I've been working on has been lost... because we were supposed to save it to the H (home) drive on the server that... also got wiped.

And speaking from experience, this is exactly what happens at call centers as well. When Microsoft rolls out a patch tuesday, they test it and then have all the machines re-imaged when the user logs off.

Re:Cool (1)

Knightman (142928) | about 3 months ago | (#47025401)

AFIK the heterogeneous client on Linux/Mac only handle installation/removal/inventory/monitoring of applications and not pre-loading of a complete OS.

Of course, this may have changed since I last worked with it but there are some inherent problems to get that to work under Linux/Mac et al.

Re:Cool (1)

Anonymous Coward | about 4 months ago | (#47025017)

Sounds like a good way to get rid of Malware

What? By installing Windows?

Re:Cool (0)

Anonymous Coward | about 4 months ago | (#47025027)

That is the definition of MALWARE!!

Rid of Malware (0)

Anonymous Coward | about 3 months ago | (#47025265)

So why then install the Malware 7?

backups (2)

Wonda (457426) | about 4 months ago | (#47024909)

Time to test those backups!

SCCM server reformats itself? (5, Funny)

Rick Zeman (15628) | about 4 months ago | (#47024913)

Kind of sounds like a snake eating its tail....

Re:SCCM server reformats itself? (2)

jon3k (691256) | about 3 months ago | (#47025383)

Someone must have pressed the Ouroboros button.

Configuration deplorement (4, Funny)

K. S. Kyosuke (729550) | about 4 months ago | (#47024925)

The configuration deployment server apparently upgraded itself into a configuration deplorement server.

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 4 months ago | (#47025059)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

Re:K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025287)

Spam reported. Ignored.

Re:K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025317)

Shouldn't have called apk names KSKyosucky like you did and ran.

Re: K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025329)

That is because they installed APK's shitty hosts application.

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 4 months ago | (#47025065)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025191)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025201)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025219)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025225)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025235)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025247)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025261)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025267)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025273)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025283)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025297)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025309)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025323)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025341)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025349)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025363)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke gets called out & ran (-1)

Anonymous Coward | about 3 months ago | (#47025399)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

K. S. Kyosuke = "Run, Forrest: RUN!" (-1)

Anonymous Coward | about 3 months ago | (#47025409)

From a fair challenge like a chickenshit blowhard http://slashdot.org/comments.p... [slashdot.org]

Re:Configuration deplorement (-1)

Anonymous Coward | about 3 months ago | (#47025425)

Everyone gets to see how little you know KS despite all your crap talk in coding forums along with your ad hominem attacks on apk that were completely illogical and off topic KSKyosucky but best of all we all get to see you run like a scared little beyotch who got run off by his betters.

Did the backup and restore work? (1)

tqft (619476) | about 4 months ago | (#47024929)

Be interesting to know how much actual data was lost

Re:Did the backup and restore work? (0)

Fuzzums (250400) | about 4 months ago | (#47025053)

Are you telling me you still keep all your data on your C-Drive instead of an other partition, an other drive or NAS or even on a cloud?

Re:Did the backup and restore work? (2, Informative)

wisnoskij (1206448) | about 4 months ago | (#47025067)

This is University. Based on my experience with University IT, there will be loads and loads of important data that you are legally obligated to NOT do that to. It cannot leave one specific room, in any form.

Normally, the computers are still contacted to the network and the Internet, but everyone using them must know NOT to copy any of these files off of C.

Re:Did the backup and restore work? (2)

scottbomb (1290580) | about 3 months ago | (#47025151)

So there are laws which dictate which hard drives and/or appliances store data relative to the OS? They can still be in the same room if that's the concern but if there are laws that actually say "x, y, and z must be stored on the same partition as the operating system" then I say they get what they deserve and perhaps those laws need to be re-examined.

Re:Did the backup and restore work? (1)

Joe_Dragon (2206452) | about 3 months ago | (#47025379)

People may keep big files local and the servers where also being formed and trying to install windows 7 as well.

Centralized Control... (2)

mhkohne (3854) | about 4 months ago | (#47024941)

The problem with centralized control is that the center can give any commands it wants...

Re:Centralized Control... (1)

lagomorpha2 (1376475) | about 3 months ago | (#47025149)

The problem with centralized control is that the center can give any commands it wants...

Somebody probably should have told that to the Bolsheviks...

Re:Centralized Control... (0)

Anonymous Coward | about 3 months ago | (#47025313)

They did , just very few people actually obeyed them.

Re:Centralized Control... (1)

Old Fatty Baldman (3630557) | about 3 months ago | (#47025431)

A long time ago I worked in a place that had a lab full of test machines and a team web server, which were on adjacent shelves. About once every year or so, they would hire a new contractor and say, "those are your test boxes over there," then be utterly shocked when he flattened the web server later that day. Switching from SCCM to a guy with sneakers and a LiveCD isn't a magic bullet.

Sounds like IT incompetence (4, Insightful)

areusche (1297613) | about 4 months ago | (#47024961)

SCCM is pretty good. It makes my desktop techs jobs significantly easier to deploy assets company wide. In this case, it sounds like someone pressed some buttons without being 100% clear as to what was going on. Unfortunate someone will not be working in IT ever again.

Re:Sounds like IT incompetence (5, Insightful)

BitZtream (692029) | about 4 months ago | (#47025061)

Assuming it was just a mistake and not malicious ...

Probably not. This shit happens, and that person who did it will never do something like this again. Have you ever made a massive, expensive mistake?

I have, I was 19 years old and cost my company nearly a million dollars due to a silly misconfiguration. After I discovered it, corrected the error and notified my boss, I spent most of the night throwing up. The next morning, after everyone in the company (only 15 people or so) knew what happened, and I walked through the halls on the way to the meeting with the owner and my boss, I thought I'd pass out. As I walked into the Owner's office I didn't even bother to sit down, expecting a fairly short conversation. I was asked to sit down while my boss had this very stern look on his face. So I did, cost them that much money, I can do what they ask.

The owner than proceeded to tell me the story of how, when working for a certain Germany car company doing CICS programming, he made a mistake that screwed up a production line and cost the company several million dollars. He knew exactly how I felt, and he knew that it would never happen again because I had already punished myself more than he possibly could.

If they fire the person who did this, they just wasted the whole event. The person learned their lesson and will be extremely cautious in the future. Firing them now just means someone else will get to reap the benefits of this experience, and thats pretty stupid.

People make mistakes, and in this case the software is at least partially responsible. The SCCM server should have aborted during the preflight checks when it realized it was going to take itself out in the process. The best thing this IT department can do is for the manager/director to keep the specific employe's name under wraps, stop shit from flowing down hill from above and move on. Nothing will benefit anyone if all of Emory treats the person responsible as if he deserves to pay for all the time lost in repairing the damage, he simply can't.

The hard lesson has been learned by everyone, nothing else will make anyone any better off.

Re:Sounds like IT incompetence (3, Insightful)

Richy_T (111409) | about 3 months ago | (#47025117)

Or he could just be an incompetent shit.

Don't get me wrong, I've made mistakes myself, perhaps not quite to the same level. Hopefully he is someone who can take a lesson but there are many who can't.

Only a good manager could tell the difference (5, Insightful)

Anonymous Coward | about 3 months ago | (#47025211)

It sounds like the commenter above was teachable - he no doubt learned his lesson.
It also sounds like the company's owner knew he could learn this lesson. That's the mark of a great manager.

Whether the Emory staffer responsible for this mistake is teachable or not, I hope his boss can tell the difference. Some folks aren't teachable, some are. If the Emory boss is worth his paycheck, he should be able to tell.

Re:Sounds like IT incompetence (5, Insightful)

bitt3n (941736) | about 3 months ago | (#47025123)

Have you ever made a massive, expensive mistake?

Glances woefully down at wedding ring...

The person learned their lesson and will be extremely cautious in the future.

Thinks back on previous three weddings...

Re:Sounds like IT incompetence (1)

lagomorpha2 (1376475) | about 3 months ago | (#47025157)

Have you ever made a massive, expensive mistake?

Glances woefully down at wedding ring...

+5 insightful

Re:Sounds like IT incompetence (1)

Anonymous Coward | about 3 months ago | (#47025231)

This is different, as you have an internal program that incessantly tells you to gain access to fertile women.

Re:Sounds like IT incompetence (2)

iggymanz (596061) | about 3 months ago | (#47025259)

Fourth unhappy one? you're not making mistakes, you have a problem

Re:Sounds like IT incompetence (4, Funny)

rastos1 (601318) | about 3 months ago | (#47025147)

Also known as:

Harrisberger's Fourth Law of the Lab:
Experience is directly proportional to the amount of equipment ruined.

Re:Sounds like IT incompetence (0)

Sir Holo (531007) | about 3 months ago | (#47025175)

You should be fired.

Re: Sounds like IT incompetence (0)

Anonymous Coward | about 3 months ago | (#47025207)

They deserve it for letting a 19 year old have that much privilege.

Re: Sounds like IT incompetence (1, Interesting)

Aethedor (973725) | about 3 months ago | (#47025227)

Your story isn't about you messing up. It's about your boss failing hard at proper risk management. He's the one who should be fired for allowing a process in the company in which one person could do so much damage. Unfortunately, this happens still too often. Companies in which the mistake of one single person, the error of one single machine or the failure of one single process starts a chain reaction which causes heavy damage. Just take a look a look at the company you work for. I'm sure everybody can point out a machine or a person that will cause serious problems if that machine or person is not available for a certain amount of time.

Re:Sounds like IT incompetence (2)

ccguy (1116865) | about 3 months ago | (#47025307)

Did you get a raise? I mean, with all that extra experience you didn't have when you signed up...

Re:Sounds like IT incompetence (2)

Kjella (173770) | about 3 months ago | (#47025315)

I think it depends on the type of person you are, if you're usually a dutiful and reliable employee who made one mistake and it's a huge one that's different than your hotshot wonder boy who always does things the quick and dirty way and has caused minor outages and bugs before but gets away with it because the quick turnaround time is making him popular. Then I'd be a lot more inclined to say your reckless behavior finally blew up in everyone's face, there's the door. Admitting to your own screw-up is also a lot better than someone else finding out or worse trying to cover it up or pin the blame on someone else, many people won't say anything until shit hits the fan or hope that by some miracle nobody will notice or find out it's you. Everybody makes mistakes, but some quite a few more and with higher consequences than others.

punished myself more than he possibly could (0)

Anonymous Coward | about 3 months ago | (#47025405)

That line really caught me since most people think that punishment works. Punishment pushes you down towards criminality not up towards better behavior. The reason people at all do better inspite of punishment is because they inherintly want to do it right and are only too happy to learn how to do so.

Our prisons are doomed since it's the best possible way to degrade and debase you to the point where you give up on society and any efforts to contribute positively to it. Instead you try to live outisde society and it's downhill from there.

Very street smart boss! You must have done something right to have him as your boss.

Re:Sounds like IT incompetence (5, Interesting)

jd2112 (1535857) | about 3 months ago | (#47025221)

SCCM is pretty good. It makes my desktop techs jobs significantly easier to deploy assets company wide. In this case, it sounds like someone pressed some buttons without being 100% clear as to what was going on. Unfortunate someone will not be working in IT ever again.

Or perhaps someone decided that having a testing environment for deployment packages was an unnecessary expense combined with personnel who aren't properly trained. Just think how much money they saved by eliminating training and a test environment!

Sounds like IT incompetence (0)

Anonymous Coward | about 3 months ago | (#47025345)

SCCM is pretty good. It makes my desktop techs jobs significantly easier to deploy assets company wide. In this case, it sounds like someone pressed some buttons without being 100% clear as to what was going on. Unfortunate someone will not be working in IT ever again.

Agreed that this is a huge blunder but mistakes happen. When you have the human factor involved there are always risks and there will be mistakes made.
I don't blame the person executing this, Ultimately it's managements responsibility to ensure the person was trained enough to not make this mistake. Someone dropped the ball??

Re:Sounds like IT incompetence (1)

fermion (181285) | about 3 months ago | (#47025369)

Obviously. I have few issues with my machines that are remotely managed, and appreciate the ease that policies can be managed. However, it is not a perfect system and did recently have a machine go down because an error occurred during such an update. Any machine I have that is managed this way either is not used for real work, or is backed up constantly under the assumption that it will be unusable at any minute.

Re:Sounds like IT incompetence (1)

Joe_Dragon (2206452) | about 3 months ago | (#47025387)

if they have tenure then you can't get rid of them just like that and even then the logs may of been wiped out as well.

Surprisingly Infrequent (4, Interesting)

crow (16139) | about 4 months ago | (#47024967)

I think the big surprise here is that this doesn't happen more often.

Consider how many corporations, universities, and such have huge PC deployments with automated updates. I've seen updates that drop all the PCs off the network, but I've never seen one where everything is wiped.

I'm also surprised that I haven't heard of malware that accidentally wiped a network of 100K or more machines when someone sent the wrong command.

Or maybe the news here is that it was in a more open environment where people hear about it. If a publicly traded company wiped a thousand PCs at its headquarters, you bet they would try to keep it quiet.

Oh man (2)

50000BTU_barbecue (588132) | about 4 months ago | (#47024979)

I bet the IT department is changing each other's diapers now! And updating their resumés....

Re:Oh man (5, Funny)

Anonymous Coward | about 4 months ago | (#47025025)

In a résumé, "Watched in horror as images were accidentally deployed" becomes "Supervised the deployment of images on university-managed computers".

Re:Oh man (1)

Richy_T (111409) | about 3 months ago | (#47025127)

site-wide.

Maybe (1)

Anonymous Coward | about 4 months ago | (#47024983)

A desperate attempt to delay taking finals?

Re:Maybe (1, Interesting)

Anonymous Coward | about 3 months ago | (#47025253)

A desperate attempt to delay taking finals?

No, finals were from 1 May to 9 May; graduation ceremonies were on Monday, two days before this incident began.

I'd like to point out that this had essentially no effect on the academic part of the university; the computers involved
belonged mostly to the libraries and administrative offices. Here in the Math/CS department, I didn't even hear about
this until today (we run our own servers, mostly Linux and (gasp) Solaris).

Re:Maybe (0)

Anonymous Coward | about 3 months ago | (#47025321)

Same thing here (different university). Only Solaris for home directories nowdays though. Yet another reason for when big IT wants us to stop running our own systems, which they do every now and then.

Not so sensational... (2)

urbanriot (924981) | about 4 months ago | (#47024987)

Considering how easy it would be for a less-than-savvy IT person to accidentally encourage this situation, I doubt this is a sensational case of some huge vulnerability.

As someone who regularly provides consultation to IT staff, I know full well that there's plenty of 'administrators' that wade into waters they don't understand. We often encounter the aging IT staff member that's forced to interact with software they don't quite understand or we have the younger IT staff that impulsively click on what they don't understand, both occasionally leading a company to some manner of pandemonium level disaster. Or you simply have a dysfunctional IT department that doesn't communicate and, "oh, I'll just move this server into this container right here..." Just another day in IT.

Re:Not so sensational... (0)

Anonymous Coward | about 4 months ago | (#47025019)

Not sure what age has to do with either scenario. Unfamiliarity with tools can result in mistakes being made. There, shortened that for you.

Fire everyone from orbit, just to be safe. (1)

Anonymous Coward | about 4 months ago | (#47024993)

No such instruction should be actionable without being compulsorily deployed to at least one test system, and the actions manually checked and confirmation given to the SCCM.

No instruction which affects a significant number of systems should be possible without manual confirmation to continue with the next batch of systems.

An instruction which is applied across several classes of systems should be treated as if separate instructions for each system.

I've seen enough mid-level sysadmins think that being able to write puppet scripts elevates them to god-like status, but being a good IT janitor is hard - it's primarily not about automating your job as much as possible, but about having such an intimate understanding of your system that it ends up running so smoothly that nobody thinks you're needed. (Yes, this creates a problem if you work for THAT sort of company, but nothing will change unless you fight.)

sooooo (0)

Anonymous Coward | about 4 months ago | (#47024997)

i work at a larger, private university in the USA. I really pooped my pants when i read this

An...accident..? (5, Insightful)

geekmux (1040042) | about 4 months ago | (#47025009)

Knowing that people have been running various kinds of centralized update services, perhaps across multiple OSes, and spanning several years now, listening to a story about an update server literally going rogue and nuking everything attached to it, and then for the coup de grace, basically committing suicide at the end by reformatting itself, does not sound like an accident.

If it truly was, I'd hate to see what the hell purposeful intent looks like.

Re:An...accident..? (0)

Anonymous Coward | about 4 months ago | (#47025033)

intent would be a lot more subtle. An accident is just going to be big and obvious.

Re:An...accident..? (2)

geekmux (1040042) | about 4 months ago | (#47025073)

intent would be a lot more subtle. An accident is just going to be big and obvious.

I guess that would depend on what the intent was.

In this particular case, if the intent was maximum damage and disruption, I've got a two word summary.

Nailed it.

Re:An...accident..? (1)

Brett Buck (811747) | about 3 months ago | (#47025165)

Systems with central administration has always been absolutely wide open to insider sabotage. Distributed systems can be made at least somewhat damage-limiting

As with others, I am amazed it doesn't happen either accidentally, or on purpose, far more than it does. You are basically one bit, or checkbox, away from it more-or-less all the time.

      BTW, I note that the result was installing Windows 7. If it was doing that, does it mean they were running XP or Vista until just now?

Re:An...accident..? (5, Funny)

Anonymous Coward | about 3 months ago | (#47025213)

Might be interesting to see how the Emory Board files this away.

Re:An...accident..? (1)

flux (5274) | about 3 months ago | (#47025353)

Maybe the malware was deeply employed in all the install images, and now it is guaranteed that all the systems have it, even after re-deploying ;).

And NOTHING of value was lost (0)

Anonymous Coward | about 4 months ago | (#47025013)

So it goes, so it has been, so it will be.

Lets wait for investigation. (1)

jacekm (895699) | about 4 months ago | (#47025029)

In my company IT tests all upgrades on the isolated small network before deploying. You would imagine that this should be standard practice.

Re:Lets wait for investigation. (1)

arth1 (260657) | about 3 months ago | (#47025357)

In my company IT tests all upgrades on the isolated small network before deploying. You would imagine that this should be standard practice.

But there's no saying that this would have prevented the problem, so it's irrelevant. You can fatfinger just as easily in production as in test and staging.
And you can even introduce new problems if not careful. Like a script washing the test deployment environment, like clearing the test target IP by mask, and inadvertently leaving the network address in its place.
With no information about what the real cause was, advocating a cure is not very helpful. Even if it's best practice and something that should be done, it is premature and can derail the investigation and fixing of the real root cause.

Time to look at FOG Project (1)

clifffton (912293) | about 4 months ago | (#47025043)

FOG requires a small amount of skill. And Linux. It wouldn't be impossible to do this in FOG, but it wouldn't happen by mistake!

Wrong OS (5, Funny)

Air-conditioned cowh (552882) | about 4 months ago | (#47025075)

It reformatted the drives and put Windows on them. Eeewww! That's gross!

Backups (3, Insightful)

wisnoskij (1206448) | about 3 months ago | (#47025095)

Bad news most likely on this front. I have worked University IT, and I can guarantee they are going to have problems.

For one, no matter how many layers of backups you have, when you are working with a bunch of 90 year old academics, they will always find a way to miss every single one.

And more grievous, Universities tend to have important data that absolutely cannot be backed up in any normal way. Data that is legally obligated to stay on one specific computer in one specific room and never leave; under penalty of legal action.

Re:Backups (0)

Anonymous Coward | about 3 months ago | (#47025245)

It did say this happened on all university managed computers; my professor's computer is not managed by the university, nor is mine. Our data would be OK.

Re:Backups (2)

jc42 (318812) | about 3 months ago | (#47025411)

... my professor's computer is not managed by the university, nor is mine. Our data would be OK.

I hope you verified this before posting. ;-)

Since a reformat and reinstall was done, the permissions involved were presumably handled at a lower level (BIOS?) than the installed OS. So it could easily have hit any Intel-based machines accessible via the network. Such low-level operations are rarely done by software that understands subtleties like ownership and organizational structures.

It might be interesting to know whether non-Windows and/or non-centrally-managed machines were affected by this event. So far, comments on the topic sound like guesses or conjectures or assumptions based on reasonability; i.e., i nteresting, but probably not reliable information.

It's not a bug Its a feature. (0)

Anonymous Coward | about 3 months ago | (#47025097)

I Simple answer is that the underlying software must have been designed to do this. A resent software update to the SCCM server could have caused it.

centralized user data... (1)

fisted (2295862) | about 3 months ago | (#47025121)

This is why centralized data storage and automated installation are invaluable for managing larger sets of desktop/office computers.
If at my workplace a computer breaks, gets stolen, catches fire, whatever, I fetch a new one from the basement, tell the PXE server to load the installation image. 15 minutes later, the user can resume their work.

Then again, it's probably much more complicated to achieve this with Windows.

most successful Win7 deployment... (0)

Anonymous Coward | about 3 months ago | (#47025137)

...ever!

Common at colleges for shared computers. (2)

bongey (974911) | about 3 months ago | (#47025161)

When I worked in the IT in my work study program, shared computers would be re-imaged often. We would usally re-image 300-400 computers at time. Often it was just some professor wanting certain program, it was just easier and safer just to wipe the machines. Malware was big problem at the time, sasser hit all of are computers, that sucked.

Well, that's why you store data a fileserver... (0)

Anonymous Coward | about 3 months ago | (#47025205)

LOL...wait, it wiped WHUT?

Umm, offline backups?

research university likely has a big theory based (0)

Joe_Dragon (2206452) | about 3 months ago | (#47025291)

research university likely has a big theory based CS program and hires people only with cs degrees and not people who went to tech schools / learned on there own / on the job.

"Somehow"??? (3, Insightful)

Tony Isaac (1301187) | about 3 months ago | (#47025407)

"Somehow" makes it sound mysterious and inexplicable. I'd be willing to bet that the truth is far less sensational. I could see a student tech assistant doing something like this on a dare, or a low-skilled admin just clicking OK one too many times, without actually reading the warning message.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>