Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

MS Says All Sidekick Data Recovered, But Damage Done

timothy posted about 5 years ago | from the small-favors dept.

Data Storage 279

nandemoari writes "T-Mobile is taking a huge financial hit in the fallout over the Sidekick data loss. But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation. As reported earlier this week, the phone network had to admit that some users' data had been permanently lost due to a problem with a server run by Microsoft-owned company Danger. The handset works by storing data such as contacts and appointments on a remote computer rather than on the phone itself. BBC news reports today that Microsoft has in fact recovered all data, but a minority are still affected (out of 1 million subscribers). Amidst this, Microsoft appears not to have suffered any financial damage. However, it seems certain that its relationship with T-Mobile will have taken a major knock. The software giant is also the target of some very bad publicity as critics question how on earth it failed to put in place adequate back-ups of the data. That could seriously damage the potential success of the firm's other 'cloud computing' plans, such as web-only editions of Office."

Sorry! There are no comments related to the filter you selected.

Cloud computer (3, Funny)

sakdoctor (1087155) | about 5 years ago | (#29760873)

Not just buzz, it's the future bro.

Re:Cloud computer (1)

Philip K Dickhead (906971) | about 5 years ago | (#29760883)

Forcris'sake!

The outfit was called "Danger".

What. It that like "break a leg"?

Microsoft and Danger (4, Interesting)

mollog (841386) | about 5 years ago | (#29761509)

At this point, the name Microsoft is pretty much a synonym for danger.

But the damage is not limited to Microsoft's reputation, the damage extends to the concept behind 'cloud computing', whatever that is. I think it is safe to say that Microsoft will recover from this incident, after all, it's record is already pretty suspect, but cloud computing will have this example hanging over it from now on.

I doubt that people will take this as a lesson that Microsoft is not to be trusted or believed since they are the public face of computing, but that computing generally, and 'cloud computing' is what's untrustworthy. Microsoft can abandon this particular project, coin a new term to replace 'cloud computing', and move on.

This is an opening for Google or other competitors. Will they step up and displace Microsoft as the public face of computing? We can be rid of monolithic operating systems if someone can make a system that boots a minimal browser/front-end that connects to the internet. A combination of BIOS and replaceable flash drive. Sell flash drives with the kernel and the drivers for the display/keyboard and network interface.

Re:Microsoft and Danger (1)

commodore64_love (1445365) | about 5 years ago | (#29761681)

This just in:

Microsoft's share of the OS market has dropped from 89.5% to 89.4%. MSFT stock plummets; the NASDAQ fell. (I kid.) People tend to have short memories. They've already forgotten the mess that was the DTV Coupon program, prevented many from using the coupons to get DTV converter boxes, and affected ~50 million Americans. This Sidekick story about a million users almost-losing data will be forgotten by next week.

Re:Cloud computer (4, Insightful)

fahrbot-bot (874524) | about 5 years ago | (#29761141)

it's the future bro

Perhaps for people who don't care about their data... Privacy, security, accountability and reliability cannot be ensured by a third party. I'll keep my data in-house thank you.

Re:Cloud computer (4, Informative)

Krneki (1192201) | about 5 years ago | (#29761369)

it's the future bro

Perhaps for people who don't care about their data... Privacy, security, accountability and reliability cannot be ensured by a third party. I'll keep my data in-house thank you.

If you can setup offline synchronization and data encryption, there is no reason to not use cloud computing.

If your provider does not support this, then it's time to change it.

Re:Cloud computer (2, Interesting)

commodore64_love (1445365) | about 5 years ago | (#29761731)

One reason not to use Cloud Computing is that I can avoid Ribbon Interface crapola (as was in Office 2007), and just keep using my older software. Or I can ignore Vista/ME and just keep using older XP/98 operating systems. With cloud computing using older programs won't be an option, because it will be forced upon you.

Re:Cloud computer (3, Insightful)

elnyka (803306) | about 5 years ago | (#29761651)

it's the future bro

Perhaps for people who don't care about their data... Privacy, security, accountability and reliability cannot be ensured by a third party. I'll keep my data in-house thank you.

Dude, organizations use third party data centers (or data centers that they physically own but are managed by a 3rd party) all the time w/o a glitch. Unless you are a software giant (like ebay or amazon) that can build your own data center, or are a minor/midsize operation (or are just a guy with a home computer), you will inevitably have a large part of your stuff either running on someone else's infrastructure or having it operate on someone else's watch.

It is done all the time, by many, for years now. Almost no glitches that can be directly attributed by the fact that a 3rd party was involved. In order to have a meaningful opinion on IT operations, you need to differentiate problems that occur because things are not run by you (things that are inevitable in computing) vs problems that occur because of lack of safeguards or wrong procedures (which can and will happen under your watch or someone else's.)

Re:Cloud computer (2, Funny)

4D6963 (933028) | about 5 years ago | (#29761653)

Yes, if I'm gonna lose personal data I want it to be to my own flawed backup strategy! To hell with professionals whose job and business is to do just that!

Re:Cloud computer (5, Funny)

corbettw (214229) | about 5 years ago | (#29761313)

Storing all of your data and the lion's share of your processing on a remote machine, with only the bare minimum stored and run locally? Sounds a lot more like the past to me.

Re:Cloud computer (1)

oldhack (1037484) | about 5 years ago | (#29761519)

Don't buzz me, bro.

Trusting in Microsoft's servers? Hah! (-1, Flamebait)

Anonymous Coward | about 5 years ago | (#29760879)

I have a feeling that Microsoft-owned also means powered by Windows, hah.

Re:Trusting in Microsoft's servers? Hah! (-1)

Anonymous Coward | about 5 years ago | (#29761245)

Last I checked, Hotmail still ran on FreeBSD - there's even a document somewhere that tries to twist into a win for windows.

Re:Trusting in Microsoft's servers? Hah! (2, Insightful)

Disgruntled Goats (1635745) | about 5 years ago | (#29761381)

Last I checked, Hotmail still ran on FreeBSD

Which was what? 8 years ago?

Re:Trusting in Microsoft's servers? Hah! (0, Troll)

Anonymous Coward | about 5 years ago | (#29761253)

Yeah, it's not as if Google's Loonix servers have ever had downtime... Oh wait. Haven't there been like 3 or 4 downtimes on Gmail just in the last few months? LOLOLOLOLOL

Re:Trusting in Microsoft's servers? Hah! (-1, Flamebait)

Anonymous Coward | about 5 years ago | (#29761359)

Ever lose any data from Google? If so state it and stop being a jackass. Everyone has occasional downtime but when we are talking about data loss it's a different ball park. Hey a new reality series is coming on. Why don't you spend your time watching who wants to fuck a midget and less time talking with the grown-ups. Ok?

Re:Trusting in Microsoft's servers? Hah! (2, Insightful)

Disgruntled Goats (1635745) | about 5 years ago | (#29761395)

In fact, yes, people have lost data from Google [webpronews.com] . That isn't even the only example one can find.

Re:Trusting in Microsoft's servers? Hah! (2, Informative)

Disgruntled Goats (1635745) | about 5 years ago | (#29761409)

Oh and if you had actually read the summary you'd see there wasn't any data loss in this case:

BBC news reports today that Microsoft has in fact recovered all data

Re:Trusting in Microsoft's servers? Hah! (1)

techiemikey (1126169) | about 5 years ago | (#29761473)

and if you finished your quote:

BBC news reports today that Microsoft has in fact recovered all data, but a minority are still affected (out of 1 million subscribers)

Re:Trusting in Microsoft's servers? Hah! (1)

Disgruntled Goats (1635745) | about 5 years ago | (#29761557)

Yes, the "but a minority are still affected" was referring to a previous line talking about them getting all the data full restored.

Microsoft Corporate Vice President Roz Ho says that all data will be restored, beginning with personal contacts.

She believes that only a minority of Sidekick users are still affected.

This is the quote in full context. There was no data loss.

Re:Trusting in Microsoft's servers? Hah! (1)

Mister Whirly (964219) | about 5 years ago | (#29761607)

Just because customers may still be affected by the outage does not automatically mean that they lost data. As a matter of fact, a statement like "recovered ALL data" should tip you off to the fact.

Anonymous Coward (-1, Flamebait)

Anonymous Coward | about 5 years ago | (#29760887)

I bet apple never would have made this mistake.........

Teenagers everywhere: REJOICE! (0)

Anonymous Coward | about 5 years ago | (#29760891)

OMG, Totally Awesome, My SK is back in the clouds!

As Rob Pegoraro of The WaPo points out (5, Informative)

wiredog (43288) | about 5 years ago | (#29760905)

here [washingtonpost.com] the damage to T-Mobile is compounded by their tone deafness on customer support.

Uh, T-Mobile, can I offer a hint here? This is not the time to nickel-and-dime cranky customers. Let them go now, and maybe they won't spend the next nine months telling everybody they know to avoid your service -- instead, if you're lucky, they'll find a new hobby after only two months.

Gotta PATCH SQL 2008 NOW !! (-1, Offtopic)

Anonymous Coward | about 5 years ago | (#29760909)

Get to it M$FT !!

This is why you have press people (5, Informative)

SuiteSisterMary (123932) | about 5 years ago | (#29760911)

Well, to be fair, whoever said 'All data is lost' to the press should have been dragged out back and shot. They should have said 'We're looking in to how long it will take to restore data, and to see if there will be any problems' and left it at that for a few days.

Re:This is why you have press people (1)

Icegryphon (715550) | about 5 years ago | (#29760951)

So they had actual backups of the data?
Still a sad state it is taking so long.

Re:This is why you have press people (3, Insightful)

rtfa-troll (1340807) | about 5 years ago | (#29760985)

From stories circulating it looks as if they are doing this by recovering the structure of the database, not restore from backup. Note that they say that most customers should have all data restored. Not just "data up to last week" or something similar. Of course this could all just be misplaced speculation and misunderstandings.

Re:This is why you have press people (1)

SuiteSisterMary (123932) | about 5 years ago | (#29761341)

That's kind of my point. As was pointed out in the original /. discussion, the data wasn't 'lost' per se; it was pointed out even then that lots of it could likely be recovered, though it would be very inconvenient and possibly not worth the effort.

Don't blame t-mobile for Danger's failure (5, Insightful)

cookie23 (555274) | about 5 years ago | (#29760921)

It is hard for me to blame T-Mobile for the MS/Danger server / backups failure. Danger both makes the phones and runs the service, where as T-Mobile appear to be little more than common carriers and the customer service department. It is a bit unreasonable to suggest that T-Mobile could have prevented the outage. I mean it not like they could host the data somewhere else right? Sure they could have done a much better job handling the failure after it happened, much much better, but I just don't think they could have prevented it.

Re:Don't blame t-mobile for Danger's failure (0, Troll)

rtfa-troll (1340807) | about 5 years ago | (#29761093)

but I just don't think they could have prevented it.

You are responsible for who you choose as a supplier. Perhaps T-mobile had never heard of Microsoft's reputation, but they should have audited it. They are 100% responsible. Specifically the person who did the service acquisition should be fired. They obviously never verified that the backup strategy was tested and they never agreed a plan for disaster handling. You can tell that from the bad information in the press releases. SuiteSisterMary's post [slashdot.org] was probably wrong about who should be "shot" but right about the releases.

Re:Don't blame t-mobile for Danger's failure (5, Insightful)

outZider (165286) | about 5 years ago | (#29761117)

T-Mobile and Danger were partners long before Microsoft ate Danger up. It's not like Microsoft had a history of failed backups and horrible transitions.

Re:Don't blame t-mobile for Danger's failure (2, Informative)

6ULDV8 (226100) | about 5 years ago | (#29761155)

Except Hotmail

Re:Don't blame t-mobile for Danger's failure (1)

EricTheGreen (223110) | about 5 years ago | (#29761715)

Oh, sure, they've got a history of horrible transitions. HotMail comes to mind immediately.

See "dogfooding" within the following:

http://www.appleinsider.com/articles/09/10/12/microsofts_sidekick_pink_problems_blamed_on_dogfooding_and_sabotage.html [appleinsider.com]

Re:Don't blame t-mobile for Danger's failure (3, Informative)

clem (5683) | about 5 years ago | (#29761457)

SuiteSisterMary's post [slashdot.org] was probably wrong about who should be "shot" but right about the releases.

Hey, now, let's not fight over who should be shot. There's plenty of bullets for everyone.

Re:Don't blame t-mobile for Danger's failure (2, Insightful)

LMacG (118321) | about 5 years ago | (#29761109)

But Johnny SidekickUser can't contract directly with Danger, he has to deal with T-Mobile. T-Mobile has some responsibility for making sure the service they're reselling operates as advertised. This shouldn't be a "best-effort" service.

It was T-Mobile's name on the contract and device. (5, Informative)

sirwired (27582) | about 5 years ago | (#29761219)

If T-Mobile plasters their name on the contract, the device, and the service, then the buck stops there. Period. Internally, T-Mobile can choose to blame the Easter Bunny if they like, but ultimately, it was T-Mobile's responsibility to ensure that their customer's data was properly protected. This absolutely could have been prevented by audits of Microsofts/Danger's operations, checks of backup integrity, tighter contracts, etc. T-Mobile can go try and sue MS to get their damages back, but in the meantime, customers can, and should, be blaming (and suing) T-Mobile.

SirWired

Re:Don't blame t-mobile for Danger's failure (1)

InlawBiker (1124825) | about 5 years ago | (#29761295)

They are ultimately responsible. Personally, I want to hear a recording of the conference call that went on in this maintenance window. I bet the "oh shit" moment was pretty intense.

Re:Don't blame t-mobile for Danger's failure (1)

Thing 1 (178996) | about 5 years ago | (#29761317)

Danger both makes the phones and runs the service [...]

You'd think that their name would at least give customers pause as to the safety of their data... (Or, perhaps their name gives them some legal wiggle room?)

Huh? (0, Flamebait)

recoiledsnake (879048) | about 5 years ago | (#29760935)

What's up with all the editorializing in the summary? Danger was bought by MS only 18 months ago. What the heck has this got to with Office and cloud computing except wishful thinking by the submitter?

Oh sorry, it's the bash MS article of the day. Please continue.

Re:Huh? (3, Funny)

Etrias (1121031) | about 5 years ago | (#29761051)

What? That's not so bad. I mean if you really wanted a conspiracy theory you could surmise that MS bought Danger with some knowledge of how in bed they were with T-Mobile. AND seeing that one of the major carriers that Google's Android is T-Mobile, MS purposefully destroyed data to strike out against T-Mobile for partnering with their sworn enemy. Right now, Ballmer is sitting in his evil lair over an active volcano, cackling fitfully while stroking a white cat. Now THAT is a conspiracy theory.

Now that I said it, it doesn't seem unpossible. I better call Hollywood.

Re:Huh? (1)

fuzzyfuzzyfungus (1223518) | about 5 years ago | (#29761549)

Don't forget: the other half of the evil plan is to turn Danger into a fuckup so massive that it spreads fear, uncertainty, and doubt over the entire enterprise of cloud computing, thus saving MS' client and server software from Google....

Re:Huh? (5, Insightful)

mcrbids (148650) | about 5 years ago | (#29761053)

What's up with all the editorializing in the summary? Danger was bought by MS only 18 months ago. What the heck has this got to with Office and cloud computing except wishful thinking by the submitter?

So... in a year and a half they shouldn't have toured their new acquisition and checked for basic things like:

1) Updated server software

2) Firewalls

3) Backups

And other "yer an idjit if you don't do this" kinda stuff?

For *any* kind of hosted service, having backups measures just slightly below "is it turned on" in terms of importance. And for a year and a half, NONE WERE DONE? Further, they did a major update to a SAN and didn't backup first?

This isn't about bashing Microsoft - highly successful businesses have had to close shop forever due to glaring, horrid oversights like this. This is gross incompetence.

Re:Huh? (1)

schon (31600) | about 5 years ago | (#29761107)

So... in a year and a half they shouldn't have toured their new acquisition and checked for basic things

Or maybe have done all that before buying them... what the hell kind of mickey-mouse outfit would buy another company without examining their operations?

Re:Huh? (0)

Anonymous Coward | about 5 years ago | (#29761179)

I'll buy your slashdot account for 47 dollars.

Re:Huh? (1)

Applekid (993327) | about 5 years ago | (#29761337)

Or maybe have done all that before buying them... what the hell kind of mickey-mouse outfit would buy another company without examining their operations?

The kind of mickey-mouse outfit that's desperate for market share, particularly if that outfit isn't exactly late to the party (*cough* Windows CE / Mobile *cough*) and hasn't managed to capture a significant portion of it.

Re:Huh? (1)

Disgruntled Goats (1635745) | about 5 years ago | (#29761635)

How would buying Danger get more market share for WinCE or WinMo?

Re:Huh? (2, Insightful)

wizardforce (1005805) | about 5 years ago | (#29761071)

What's up with all the editorializing in the summary? Danger was bought by MS only 18 months ago. What the heck has this got to with Office and cloud computing except wishful thinking by the submitter?

Er... because it is a form of cloud computing which failed? When a failure like this occurs, it rightfully raises doubt as to the reliability of other cloud computing services, one of which happens to involve office.

As reported earlier this week, the phone network had to admit that some users' data had been permanently lost due to a problem with a server run by Microsoft-owned company Danger. The handset works by storing data such as contacts and appointments on a remote computer rather than on the phone itself.

Re:Huh? (4, Funny)

sopssa (1498795) | about 5 years ago | (#29761077)

Dear Sir or Madam,

The responsible Anti-Microsoft Troll that should have replied to this post by now is on sick leave and was unable to prepare a custom flaming reply to this particular post. In lieu of that, attached is our generic template which we use to write all our flaming responses.

1. Make a general anti-Microsoft jab
2. Blame Microsoft for it's stance against Free Software (and also for lack of network neutrality, the current state of patent laws, the Iraq war, and the extinction of the dinosaurs)
3. Accuse the poster who wrote something positive about Microsoft of being either a fanboy or a Microsoft employee. If the poster in question made a comment about Microsoft's actual support of Free Software in a particular instance, accuse the poster of being an oblivious idiot unable to see through their Embrace-Extend-Extinguish approach
4. State that the Linux revolution is inevitable
5. Finish off with another outpour of flames

We hope you will be able to infer the potential content of the post that should have been done by the respective Troll. Please accept our apologies.

Sincerely,

Assistant Secretary,
Anti-Microsoft Trolling Association, Ltd.

Re:Huh? (0, Troll)

Yvan256 (722131) | about 5 years ago | (#29761283)

Oh yeah? Well, the jerk store called and they're running out of you!

Re:Huh? (4, Insightful)

MightyMartian (840721) | about 5 years ago | (#29761101)

Well, it's a bit of a non sequitur, to be sure. But the whole incident spells out in stark detail the dangers of "cloud computing", or as us folks who actually have worked with computers for more than than ten minutes call it; the client-server model. When explained as what it really is, it's a matter of ensuring adequate and timely backups. When described in some pathetic marketing term, it sounds like some magical new way of computing, no longer constrained by those old-fashioned good practices.

Quite frankly, I would never ever ever put any mission critical data or apps on a system that I couldn't back end the data on my own out of. If I can't move my data out of the app, then my data never gets there in the first place.

well of course it is... (0)

Anonymous Coward | about 5 years ago | (#29761131)

At least bother to read the summary you dolt.

Re:Huh? (5, Insightful)

mcmonkey (96054) | about 5 years ago | (#29761139)

You really don't see the connection?

Yesterday, you put all your cell phone contacts and calendar data up in the "cloud".

Today, your data is lost.

Tomorrow, the same companies responsible for losing your cell phone data now want to take over all your Office documents.

Well, since this is /., you take your car in for a routine oil change. The mechanic botches the job.

Are you going to go back to the same mechanic for a transmission rebuild?

Re:Huh? (0)

Anonymous Coward | about 5 years ago | (#29761619)

Erm, actually, 'today', your data is recovered after an extended outage.

I mean, I agree with the point, but I agree with that point REGARDLESS OF COMPANY INVOLVED. You don't see me storing important stuff on Google Docs...

Re:Huh? (3, Interesting)

mcgrew (92797) | about 5 years ago | (#29761277)

What's up with all the editorializing in the summary?

You must be new here.

Danger was bought by MS only 18 months ago.

A year and a half later and they don't have a handle on it? Someone's getting paid WAY too much.

What the heck has this got to with Office and cloud computing

Nothing to do with office (unless they're using Access, which would explain the data loss), but "cloud computing" is what a couple here have more logically and less buzzwordily renamed "OPS" -- Other People's Servers. This is EXACTLY what "cloud computing" is.

Re:Huh? (5, Insightful)

Col. Klink (retired) (11632) | about 5 years ago | (#29761347)

They have not had this problem in their first 8 years. Then, 18 months after Microsoft acquires them, they have a critical failure. You think that's all coincidence?

I suppose it's possible for one company to buy another and leave the company alone, but Microsoft certainly didn't do this. They moved most of the developers to Project Pink (and most of them have left MS entirely by now). I think it's pretty clear that the new MS was responsible. They managed the company. The data was stored at Microsoft's data centers.

Meanwhile, Microsoft is trying to sell people on the idea that their data should be hosted at Microsoft data centers. Am I not supposed to be skeptical about this now?

Re:Huh? (1)

jeffyboz (1027792) | about 5 years ago | (#29761543)

What's up with all the editorializing in the summary? Danger was bought by MS only 18 months ago. What the heck has this got to with Office and cloud computing except wishful thinking by the submitter?

Oh sorry, it's the bash MS article of the day. Please continue.

Ugggh, if you can't stand the bash-wagon, get off the bus.

Re:Huh? (1)

Frosty Piss (770223) | about 5 years ago | (#29761741)

But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation.

I don't see it. MS is one of those companies people either love or hate. The lovers will say "shit happens, move on" and the haters will say "I told you so". Sum tot = zip.

AT&T says thanks (5, Funny)

elrous0 (869638) | about 5 years ago | (#29760939)

Hey, at least this fiasco took the heat off their crappy network for a while.

Microsoft's reputation (5, Funny)

Aurisor (932566) | about 5 years ago | (#29760941)

But Microsoft, which bears at least part of the responsibility for the mistake, is paying the price with its reputation.

Wow, this is a terrible blow for Microsoft. This might make people think that they produce unreliable products!

Re:Microsoft's reputation (2)

Finallyjoined!!! (1158431) | about 5 years ago | (#29761147)

This whole sorry saga just puts the two companies in silhouette. Data loss is directly caused by Microsoft & their shoddy stuff

T-Mobile, who only sells this Microsoft stuff, on hearing of the problems, immediately issues a statement & offers advice & compensation.
Microsoft, who caused this, "Not me Guv! 'onest!"

Re:Microsoft's reputation (0)

Anonymous Coward | about 5 years ago | (#29761297)

Danger wrote and created the solution. Unfortunately for Microsoft, they walked into a terribly run company though.

Not only did they screw this up, but they also screwed up the reason that Microsoft bought them--their "Pink" project. It's probably about time to cut that entire company loose. Even AppleInsider claims it may be sabotage [appleinsider.com] , and AppleInsider takes every chance to knock Microsoft down a few pegs (and does in the paragraphs leading up to it).

I don't know what went wrong, but something tells me that a poor implementation was at the root of it and the "Danger" division was to blame. I imagine they are run like a separate division, rather than one big happy family.

Re:Microsoft's reputation (0)

Anonymous Coward | about 5 years ago | (#29761491)

Riiiiight . . . MS has owned Danger for 18 months and put their Pink project in Danger's hands and walked away and said, "Hey guys create an iPhone Killer and call Mr. Ballmer when you are done!" I don't believe for a minute that MS would take the hands off approach on this project or the operation of the company. If they did they screwed up BIG TIME. This is an huge 'fail' by MS one way or the other.

Re:Microsoft's reputation (2)

johnny cashed (590023) | about 5 years ago | (#29761673)

Wow, this is a terrible blow for Microsoft. This might make people think that they produce unreliable and shoddy products!

There, fixed that for you.

Backups are unimportant; restore is everything. (5, Insightful)

rtfa-troll (1340807) | about 5 years ago | (#29760947)

Worth repeating every time. Nobody cares if you back up your data. Take a blank server; take whatever it is that you store offsite. If you can turn the blank server into your production system then you are fine. If you can't then your strategy is failing. If you never try it then you are an amateur.

This incompetence is something far beyond serious for MS. T-mobile is a much bigger customer than almost anyone short of vodafone can ever hope to be. MS have been moving strategically into hosting servers such as exchange for many customers. If you're a CEO you should be calling your CIO in and asking him when he plans to be free of MS services. If you are a CIO you want to be able to answer "there's nothing business critical relying on MS services" by the time that meeting comes.

Re:Backups are unimportant; restore is everything. (0)

Anonymous Coward | about 5 years ago | (#29761021)

This would be true to ask your CIO, no matter who is hosting your applications or data for that matter.

Understand its Bash M$ day, but they are not the only ones out there that do this service.

Cloud computing... Wonder what that will do to Compliance like PCI, HIPAA, and SOX....

Re:Backups are unimportant; restore is everything. (1)

Aurisor (932566) | about 5 years ago | (#29761079)

I think you're overstating your point. Unless you are saving your data in a truly useless format, having a practiced procedure for getting that data back into production only lets you get the data back up faster. We have one backup system in particular at my office - although we have never built a production machine from it, we do (manually and automatically) test the data to ensure that everything from production made it in. Will restoring that data be slow and sketchy? Sure. Is it fair to say that nobody will care if we have the data backed up? No.

That being said, though, if a system is capable of losing this much data without an act of god, then a lot of people need to be fired. With incremental backups, tests, and enough redundancy, it is nearly impossible to actually lose more than a couple days worth of data.

I agree with you about MS, though. People really need to get it through their heads that Microsoft is one company among many. They make great hardware (typing this on a Microsoft Natural keyboard), and excel is still best in class; on the other hand, they make a couple products I wouldn't be caught dead using.

On the bright side, I guess this should put the adage "Nobody gets fired for buying Microsoft" to bed, eh? :)

Re:Backups are unimportant; restore is everything. (1)

rtfa-troll (1340807) | about 5 years ago | (#29761185)

There are astounding stories of whole databases where it turned out that database had never been written from memory to disk. There are many people who make the mistake of believing that their MySQL files on disk are consistent (you are supposed to dump the database). Even applications like office can have corrupt files on disk if a document is open. I know of situations where it turned out that the heads in the backup system were misaligned and so the tape only read back on the system they were backing up on (and where they tested the backup tape). I'm not really interested in how fast you do the restore. I'm interested in the fact that you have tested that it really _does_ restore on a blank default unrelated server. Even just that you do it once a month for one random system out of five hundred will put you so far ahead of the rest of the people out there that you will be happy.

Re:Backups are unimportant; restore is everything. (1)

rtfa-troll (1340807) | about 5 years ago | (#29761465)

Sorry and I should have said; not testing breaks the fundamental principle of KISS. If you have to think about whether your backup is correct, then your system is too complex. You should know it's correct because you know it works.

Re:Backups are unimportant; restore is everything. (1)

timster (32400) | about 5 years ago | (#29761567)

I know of situations where it turned out that the heads in the backup system were misaligned and so the tape only read back on the system they were backing up on

As I recall, this was essentially true all the time for DDS-3 drives. Remember kids, Just Say No to helical tape.

Re:Backups are unimportant; restore is everything. (4, Insightful)

timster (32400) | about 5 years ago | (#29761209)

No, he's NOT overstating his point. Unless your data is a bunch of flat text files or Word documents or whatever the restore is a critically difficult process.

Enterprise data like this often has never been in a flat or "dead" state since the original implementation. Complex applications frequently have delicate interactions between the live application and the contents of the database at any particular moment. Having a bunch of database tables on a tape somewhere doesn't do you much good if the application can't actually start from the state contained on the tapes, and it's a two-week manual process to clean up the issues.

If you can afford a "slow and sketchy" restore process, or your application is just not that complicated, then by all means, don't test your restore, and don't create a department with responsibility for backups and nothing else. It's still amateur work.

Re:Backups are unimportant; restore is everything. (2, Informative)

Ephemeriis (315124) | about 5 years ago | (#29761211)

I think you're overstating your point. Unless you are saving your data in a truly useless format, having a practiced procedure for getting that data back into production only lets you get the data back up faster. We have one backup system in particular at my office - although we have never built a production machine from it, we do (manually and automatically) test the data to ensure that everything from production made it in. Will restoring that data be slow and sketchy? Sure. Is it fair to say that nobody will care if we have the data backed up? No.

The point isn't to have a practiced procedure that your technicians can run through with their eyes closed... The point is to actually test your backups and know whether they are working, whether the data is usable, and whether it is possible to get a production server up and running from that backup.

Most backups aren't going to be as easy as insert tape, walk away, come back to a working production server an hour later. Most backups will involve some kind of re-pointing or importing or configuration or whatever. That's kind of expected.

But if you never test your data, you don't know if there's anything being written to the tape (disk, cloud, whatever). Sure, the backup program (script, monkey, whatever) claims the task was completed successfully... But you don't know. The data could all be corrupt. Or you could have skipped some innocent-looking database that turns out to be truly essential. Or you might have re-named a directory since the backup was configured, and now you aren't getting something that you need.

The point is that you need to test your backups periodically.

Re:Backups are unimportant; restore is everything. (1)

Slashdot Parent (995749) | about 5 years ago | (#29761541)

Will restoring that data be slow and sketchy? Sure.

So what is supposed to happen while "slow and sketchy" is taking place? All business stops?

And what is meant by "sketchy"? "Sketchy" is not the adjective I like to hear used to describe the accuracy and consistency of my data.

Even if you're just backing up a bunch of flat files, how do you know that your backup is a consistent snapshot? Or are you OK with your data just being invalid in unpredictable ways?

Where are your backups located? On-site? I sure hope not. Fires happen. Floods happen.

Backups and restores are tricky to get right, and I don't think it's possible to overstate this.

Re:Backups are unimportant; restore is everything. (3, Interesting)

cptdondo (59460) | about 5 years ago | (#29761153)

This incompetence is something far beyond serious for MS. T-mobile is a much bigger customer than almost anyone short of vodafone can ever hope to be. MS have been moving strategically into hosting servers such as exchange for many customers. If you're a CEO you should be calling your CIO in and asking him when he plans to be free of MS services. If you are a CIO you want to be able to answer "there's nothing business critical relying on MS services" by the time that meeting comes.

Hehe. I raised this issue when this broke. We have a huge amount of critical data outsourced to a hosting company. I sent this fiasco up the food chain asking what is our backup strategy should this happen to our host.

I got back some pablum about "well, they have 2 geographically separate datacenters, blah blah blah" from the guy who administers the contract.

Maybe they did at one point but I know the folks we use fired most of their devs, including the lead developer, back in March as a cost cutting measure. and I wouldn't be surprised if one of the "two data centers" disappeared along with the developers. Regardless, no one on our end seems to be concerned and no one is taking any precautions (like local backups.)

Maybe one day I'll get to say, "I Told You So."

Re:Backups are unimportant; restore is everything. (0)

Anonymous Coward | about 5 years ago | (#29761593)

Maybe one day I'll get to say, "I Told You So."

Don' forget to back up those e-mails! ;)

Re:Backups are unimportant; restore is everything. (1)

onyxruby (118189) | about 5 years ago | (#29761533)

The issue has nothing to do with Microsoft. The issue has to do with a failure by T-Mobile with a vendor (that happens to be Microsoft). How T-Mobile ever approved a contract with the appropriate backup software, hardware, DR plan and testing is the bigger issue. Microsoft likely provided exactly the level of support that T-Mobile paid for, and I'm willing to bet that T-Mobile balked at these proposed charges from Microsoft and went with the cheaper option without the backup expenses. If your a CIO you use this as an example of why you pay for backup and disaster recovery services.

There are many backup and recovery products that work with Microsoft products just as their are for the various flavors of *nix. Best practices can and should be vendor neutral and your post is completely misguided. This should be a lesson learned for those involved in best practices and reckless management decisions. All that being said, Microsoft never should have agreed to a contract without the appropriate backup clauses in place. If microsoft did have those clauses in the contract, than they violated their contract in a very public way and you will be reading about the lawsuit from T-Mobile all too soon.

Re:Backups are unimportant; restore is everything. (2, Insightful)

Slipped_Disk (532132) | about 5 years ago | (#29761627)

This point absolutely cannot be overstated: A backup that has never been through a restore/recovery test is just as bad as having no backups at all.
Your admin team or hosting company should be able to tell you what is involved to get from your backup to a fully functioning production system (a truly well thought-out backup scheme will have a step-by-step recovery checklist), and they should be able to provide a worst-case data loss estimate based on your backup scheme.

This isn't a failure of "cloud computing" or any other buzzword-of-the-day but rather a failure of basic competence in information management: an unforeseen event coupled with broken, inadequate or nonexistent backups lead to a catastrophic data loss that should never have happened.

Not likely (3, Interesting)

pongo000 (97357) | about 5 years ago | (#29760973)

"That could seriously damage the potential success of the firm's other 'cloud computing' plans, such as web-only editions of Office."

I can't tell whether this is spin put on the summary by the submitter or some other third-party (because we all know submitters are, absent any editorial constraints on /., free to post what they want without attribution). That said, it's highly unlikely Microsoft will suffer from this. Wisely, they offloaded all responsibility the moment they created this entity known as Danger. They've effectively washed their hands of the entire affair, because it wasn't really a Microsoft problem in the end, but a problem with an affiliated company.

It is simply wishful thinking on the part of the submitter (or whomever) that Microsoft will be tainted by this deal. In all likelihood, Microsoft will simply walk away from their relationship with Danger, and it will be business again as usual.

Re:Not likely (3, Insightful)

nedlohs (1335013) | about 5 years ago | (#29761343)

Except that people make decisions and don't really care if something is just "affiliated".

Microsoft and Google bid for the "cloud computing" "office" contract at some company. Do you really think Google isn't going to mention, with a bunch of references, this screw up?

With quotes from press releases like:

We have determined that the outage was caused by a system failure that created data loss in the core database and the back-up.

Roz Ho
Corporate Vice President
Premium Mobile Experiences, Microsoft Corporation

in big bold blocks.

Seriously (0)

Anonymous Coward | about 5 years ago | (#29760997)

I think in the end that you get what you deserve if you actually bought a sidekick.

said it before and will say it again (4, Interesting)

jollyreaper (513215) | about 5 years ago | (#29761027)

The worrisome part about cloud computing is putting your trust in someone else's hands. But keeping your backup process internal to the company is no panacea either. Bad management practice is what led to the cloud screwing up, just like bad management practice led to in-house data losses at other companies.

How many of you guys generate your own power 24x7? C'mon, you're really going to place the face of your business in the hands of people running off the wire? Wire power. Feh! That wire could be going anywhere. Real men run their own generators!

Sounds silly, right? Of course, that's only because we're used to power companies running like utilities, government-regulated monopolies allowed to exclusively service the public with a healthy, dependable profit in return for low rates and universal service. In such an environment having your own generators for anything other than emergencies is paranoia. But wow, you start deregulating things and let the businessmen go nuts and it almost seems like you'd have to.

The real question with cloud computing is whether the companies are going to operate in a fashion that brings to mind steady, sober, dependable service like a local utility, like a giant rapacious corporation uncaring of human concerns, or like a fly-by-night dotcom. My personal opinion is that I don't trust these fuckers. Current company's situation is that we have a major software product we run our business on and the publisher got gobbled up by a bigger company and that company got gobbled up by a bigger one. The big company has decided to discontinue the product and have been slowly dismantling the team that supports it. We know we're going to have to make a jump eventually but the conglomerate could pull the plug tomorrow and we'd still be in operation. If it was a cloud app, we could be dead in the water.

Re:said it before and will say it again (1)

vertinox (846076) | about 5 years ago | (#29761291)

The worrisome part about cloud computing is putting your trust in someone else's hands.

I don't get this part. (privacy issues aside) Cloud services aren't generally touted as incremental backup solutions.

I mean its the same exact thing as running a RAID server in your own home. Simply having a system that is designed to mitigate downtime a backup system does not make.

Even if you have a system that separates data across multiple disks in different locations, user error or a database bug could wipe the "live" data just as much as a cloud hosting provider mistake can.

I think its much as the user being at fault as accepting a single solution as an acceptable backup solution. If it is that important, you will keep it local, on the cloud, and then on an external drive and tape backup in a fire safe or in a safety deposit box off site.

If its too complicated or difficult for you to use multiple solutions, then the data must really not be that important to you.

One other fair point... Your data is always technically in someone elses hands but on different levels. I mean how much do you trust seagate or western digital to not make crappy hard drives? Or that your RAID controller isn't going to eat your data?

Just because the hardware is sitting next to you completely in your control doesn't make it any inherently better than one on the cloud. Backup includes using multiple solutions over multiple locations... And that its incremental!

And if you are worried about privacy issue... 256 bit Truecrypt that sh** before you upload.

Re:said it before and will say it again (1)

Todd Knarr (15451) | about 5 years ago | (#29761479)

Well, actually my employer does in fact generate their own power. Or rather, they can. Our data center runs off a UPS bank, which is fed by the mains feed and a generator. If mains power fails, the UPS has enough capacity to keep the data center running until the genny starts up and starts supplying power. It's more convenient to run off mains, but we don't assume mains power is totally reliable (or even completely clean, the UPS bank filters and stabilizes it).

And yes, they run monthly tests of the genny to make sure it'll start when it's needed. They also run an annual test that involves shutting off the mains feed upstream of us to make sure everything works when the mains feed goes dead for real, followed by a fail-over to our disaster-recovery data center to make sure that part of things works the way it's supposed to.

Re:said it before and will say it again (4, Insightful)

Chris Mattern (191822) | about 5 years ago | (#29761501)

The real question with cloud computing is whether the companies are going to operate in a fashion that brings to mind steady, sober, dependable service like a local utility, [or] like a giant rapacious corporation uncaring of human concerns

Man, what fantasyland are your utilities located in? I wanna move there! In my experience, utilities *are* "giant rapacious corporation uncaring of human concerns".

Re:said it before and will say it again (1)

jeffasselin (566598) | about 5 years ago | (#29761717)

If you trusted the power company that much, you wouldn't have UPS and power generators in data centers.

Stormy weather (3, Interesting)

surfdaddy (930829) | about 5 years ago | (#29761059)

Given how much of our internet access is being spied on by the government, how could ANYBODY want to trust their critical data to a cloud service? Sounds like Microsoft has Cumulonimbus clouds.

Re:Stormy weather (1)

TheGratefulNet (143330) | about 5 years ago | (#29761137)

but that can work in your favor!

'talk' about sensitive stuff in the middle of a file upload (so to speak) and you'll be sniffed.

then simply file a FOIA to get your data back from the government 'backups'.

(yeah, right.)

Critics only *NOW* questioning MS's competence?!?! (4, Insightful)

Anonymous Coward | about 5 years ago | (#29761087)

Years of BSODS.

Years of viruses.

Years of trojans.

Yet THIS "damages Microsoft's reputation"?!?!?!

Re:Critics only *NOW* questioning MS's competence? (0)

Anonymous Coward | about 5 years ago | (#29761617)

I was with you until "Years of Trojans", since, in my experience, Trojans are much better than Durex (broken too many Durex to count).

Re:Critics only *NOW* questioning MS's competence? (2)

rtfa-troll (1340807) | about 5 years ago | (#29761645)

It's kind of like having a reference customer. It's all very well showing that they are incompetent in theory. It's good to be able to set up the production servers and run load tests. Here we have a real life demo that MS can really damage loads of customer's data. There are always cynics who say "yes, but they won't be able to do it in production". Now nobody will be able to claim that MS can't do an up to date full scale cloud screw up.

Simon Says The Moon Is Made Of Green Cheese (1, Funny)

Anonymous Coward | about 5 years ago | (#29761163)

"a server run by Microsoft-owned company Danger."

rename Microsoft to DangerOUS.

Yours In Ashgabat,
K. Trout

One thing and another (5, Insightful)

Teun (17872) | about 5 years ago | (#29761165)

Cloud computing and remote storage are not necessarily the same.

What we see here is a small device storing it's data remotely and I wonder why.
Considering how cheap a couple of GB of memory are and how precious wireless bandwidth is this can mean only one thing, having and thus exploiting that data is worth more than the cost of the bandwidth.

Danger, Will Robinson, Danger! (3, Interesting)

pubwvj (1045960) | about 5 years ago | (#29761235)

A company called Danger? Responsible for data and servers? Yowsa! Red alert time!

MS has a reputation? (1)

minstrelmike (1602771) | about 5 years ago | (#29761239)

How could Microsoft damage their reputation? That's like saying George W Bush could be more inept. Once you're at rockbottom, you cannot go lower.

Microsoft? No. (4, Insightful)

snspdaarf (1314399) | about 5 years ago | (#29761281)

I don't see this as having a big effect on Microsoft. T-Mobile on the other hand....
I don't believe that customers care if your services providers have problems. They have an agreement with you, not your providers.

Re:Microsoft? No. (1)

sten ben (1652107) | about 5 years ago | (#29761461)

Well, my guess is that a lot of other carriers etc. interested in Microsoft products and storage solutions might be a bit more ... ehm ... critical before they go to bed with them. But on the other hand most people I've met who have used Windows Mobile hates it with a passion, so maybe most carriers aren't looking that way anyway.

All Hail Cloud computing!! (0)

Anonymous Coward | about 5 years ago | (#29761331)

This is what makes "cloud computing" so great! It's not your fault you don't have a backup, because you can't backup your own data.

All data recovered? (3, Interesting)

Carik (205890) | about 5 years ago | (#29761415)

So here's what confuses me... "BBC news reports today that Microsoft has in fact recovered all data, but a minority are still affected." If all the data has been recovered, wouldn't NO ONE still be affected? I mean... being affected by this means your data was lost in such a way that it couldn't be recovered. So...

Re:All data recovered? (1)

Disgruntled Goats (1635745) | about 5 years ago | (#29761683)

No, the "but a minority are still affected" has to do with some people still experiencing outages.

Confirmation? (0)

Anonymous Coward | about 5 years ago | (#29761711)

Has anyone confirm that their data has been recovered?

Maybe I'm jaded, but I can imagine that this is just a way to settle down the s-storm until the media cycle moves on to another politician being found "on the appalacian Trail".

Anyone?

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?