×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Confirmed Gmail / Google App Outage

timothy posted more than 4 years ago | from the were-you-there-when-it-happened dept.

Google 189

mbone writes "Earlier today there was a confirmed Google outage which got a lot of attention from network operators. From a post to NANOG after everything calmed down: 'Google ack'd a maintenance on their core network did not go as planned-Forced traffic to one peer link that was unable to handle all the traffic. Maintenance has been rolled back. Issue has been restored.' This is exactly what makes me nervous about cloud computing and data storage. It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once?" Several readers also point to CNET's coverage of the outage. Update: 05/14 19:25 GMT by T : CWmike adds this: "Steven J. Vaughan-Nichols writes that what may be happening is a massive DDoS attack. Based on the size of the attack that would be needed to interfere with Google, I believe that it's quite likely to be the result of an attack from the controllers of the Windows worm, Conficker. Another theory that has been put about — that the problem was due to AT&T NOC routing problems — does not appear to hold water, writes Steven." Update: 05/14 21:01 GMT by T : Google's put up a low-detail explanation on their blog that says "An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam. As a result, about 14% of our users experienced slow services or even interruptions."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

189 comments

comments from google (5, Funny)

JWSmythe (446288) | more than 4 years ago | (#27954283)

    In comments from Google Admins, they said "oops." :)

Re:comments from google (5, Funny)

TommydCat (791543) | more than 4 years ago | (#27954371)

We apologise for the fault in the comments. Those responsible have been sacked.

Re:comments from google (-1, Redundant)

bev_tech_rob (313485) | more than 4 years ago | (#27954407)

Heh....good Monty Python reference there... ;)

Re:comments from google (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#27954751)

wtf are you talking about noob? thats a digg and 4chan thing.

Re:comments from google (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#27955207)

Nope.

Re:comments from google (-1, Troll)

Anonymous Coward | more than 4 years ago | (#27955345)

yeah it is, faggot. 4chan is the source of all memes worth repeating. you must be a nigger.

Re:comments from google (4, Insightful)

Anonymous Coward | more than 4 years ago | (#27954859)

Thanks for ruining it. Cultural references are MUCH funnier when not immediately followed up by some smartass going "I got the joke, guys! It's funny, see, he's paraphrasing a line from !"

Don't do that anymore. Please.

Re:comments from google (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#27955289)

Movie quotes are only funny when you're actually watching the movie, not when some nerd tries to appropriate it for his own lack of cleverness.

Re:comments from google (3, Informative)

Red Flayer (890720) | more than 4 years ago | (#27954543)

Yes, but the Admins' "Oops" comments were a cover-up. This outage was not due to an error during maintenance.

This outage was a sign that Gatus and Joba are having some success in their quest to thwart Googol the Destroyer [slashdot.org] ; apparently, not only are they getting converts in their efforts to have all the world's sorcerors collaborate on the One True Operating System with Global Search, but they (or someone else) has launched an effort to delay the work on the Rite of a Million Targeted Ads by Googol the Destroyer and his acolytes.

Who is responsible for the guerrilla attacks on Googol the Destroyer, his acolytes, and his crack team of evil underlords? How are Gatus and Joba faring on their recruitment of the world's sorcerors? What has Stallmanx been working on in his secret laboratory*? Will we ever find out what lies beneath his Beard of Druidic Prowess? Answers to these questions and more will be revealed in the next two episodes!

Remember, kids, tune in to next week's episode (look for it Tuesday or Wednesday) of Googol the Destroyer!

Re:comments from google (1)

Red Flayer (890720) | more than 4 years ago | (#27954635)

Oh, one more thing... I promise not to write advertisements for next week's episode again... just in case it was bothering anyone.

Google Voice Issues (4, Informative)

0100010001010011 (652467) | more than 4 years ago | (#27954317)

My Google voice account went all sorts of haywire.

1) Text messages sent from the web got duplicated. One person got near 10 duplicates in quick succession. I also got duplicate messages back.
2) My number doesn't work. If you call it you get a "Currently unavailable"
3) A few calls that came in before the outage aren't showing up in the Received/Missed calling list.

Re:Google Voice Issues (4, Interesting)

MozeeToby (1163751) | more than 4 years ago | (#27954587)

I've noticed some inconsistencies on my companies finance.google page. It seems to be giving two different values for gains and losses for the day, the one on the graph is correct but the one at the heading is not. It also lists our company as one of the related companies, something that it has never done before.

I've got to wonder just what the hell happened here. Major and unusual issues across nearly all of Google's services? This isn't going to be good for Google's brand image.

Re:Google Voice Issues (5, Funny)

MickyTheIdiot (1032226) | more than 4 years ago | (#27955065)

I noticed that when I did an images.google.com search for "hot anime chicks" it showed me a picture of Cowboy Neal.

So yeah, it did cause issues on all of Google's services. It also soiled the inside of my trash can with puke.

Re:Google Voice Issues (1, Insightful)

Anonymous Coward | more than 4 years ago | (#27955973)

Looks like it screwed up your typeface too.

Phone... Home... (2, Funny)

N!NJA (1437175) | more than 4 years ago | (#27955977)

strange. my Firefox 3.0.10 got somehow affected by this outage. it just refused to open! it loaded about 30Mb of data to RAM but went nowhere from there. the browser window never appeared. and i tried to re-launch it several times, but for no avail! very odd.... anyone else had problems with it? Opera -- although not able to open Google.com -- opened fine!

Is Firefox tied to Google like E.T. was tied to Elliot?

We have to take an deep breath... (5, Funny)

WarwickRyan (780794) | more than 4 years ago | (#27954319)

...and take an stroll to the great big place known as "outside".

Re:We have to take an deep breath... (4, Funny)

Just Some Guy (3352) | more than 4 years ago | (#27954579)

Wow, that's literally what we did. "Hey, Internet's broken. Walk?", and we went outside and got a little sun and fresh air. Thanks, Google!

Re:We have to take an deep breath... (4, Insightful)

Red Flayer (890720) | more than 4 years ago | (#27955091)

Or you know, do what most people do when systems go down... work on things not requiring those systems.

What's that? All your work systems are dependent on a single outside vendor?

Heh. Heh heh. Brilliant.

Re:We have to take an deep breath... (1)

PriceIke (751512) | more than 4 years ago | (#27955449)

The OUTSIDE MMO .. great graphics (if your video hardware can handle it), like AWESOME resolution, and there's billions of players. You can even make real money if you know the right cheats to use. But it's an expensive monthly subscription ...

e-mails down? (5, Funny)

Anonymous Coward | more than 4 years ago | (#27954325)

call me....

Re:e-mails down? (0)

Anonymous Coward | more than 4 years ago | (#27954427)

Just don't be using Google Voice...

Re:e-mails down? (1, Funny)

Anonymous Coward | more than 4 years ago | (#27955271)

I can't! I use the google phone!

Hey Chicken Little (5, Funny)

llZENll (545605) | more than 4 years ago | (#27954327)

And yet somehow miraculously we are all still alive. The sky is not falling!

Re:Hey Chicken Little (0)

Anonymous Coward | more than 4 years ago | (#27954717)

Why isn't this upvoted more?

I can't believe the amount of blather this is getting.

Everyone at once is better though (3, Insightful)

Nick Ives (317) | more than 4 years ago | (#27954347)

When it's just your mail server down, everyone else gets annoyed at you because you're not {gett,receiv}ing mail they're {sending, expecting from} you. When the cloud is down, everyone can just chill and be thankful that they're not going to log on to find a whole stream of new emails.

This sucks for docs though but using a completely cloud based doc solution is a bit mental. Even if you're mobile it's best to have a local copy to save on battery life.

Re:Everyone at once is better though (3, Insightful)

rho (6063) | more than 4 years ago | (#27955367)

It also sucks for the Web in general.

Google was so fucked that a lot of pages that had Google ads, or Google Analytics were slow to load or not loading at all.

Re:Everyone at once is better though (4, Insightful)

Nick Ives (317) | more than 4 years ago | (#27955635)

Browsers should be smarter about that. Maybe if they remembered that certain hosts are down and so stop trying to load scripts from them? They could periodically retry unreachable script-hosts in the background and then ask the user if they wanted to reload all relevant tabs.

The problem with remotely hosted scripts isn't just limited to Google or cloud apps, it's a more general issue and browsers should be able to handle it with grace.

Re:Everyone at once is better though (1)

rho (6063) | more than 4 years ago | (#27955991)

Browsers should be smarter about that. Maybe if they remembered that certain hosts are down and so stop trying to load scripts from them? They could periodically retry unreachable script-hosts in the background and then ask the user if they wanted to reload all relevant tabs.

What does the unicorn burger taste like in this mystical land in which you live?

Re:Everyone at once is better though (1)

RLiegh (247921) | more than 4 years ago | (#27955853)

Google was so fucked that a lot of pages that had Google ads, or Google Analytics were slow to load or not loading at all.

Which is different from business as usual how, exactly? There's a reason googleanalytics has a place in my adblock file (and I'm far from alone in that).

single point of failure (1)

Ilgaz (86384) | more than 4 years ago | (#27955711)

e-mail is supposed to be reliable because of its distributed nature. It is not supposed to be on single "cloud", distributed machines should be caring for it. It is just like XMPP vs. old fashion MSN/AIM etc. junk.

Let me show what I see with the "cloud" (which is one of the worst abused terms) right now:
(wget)
s3.amazonaws.com[72.21.207.242]
Saving to: `423.dmg'

10% [===>                                   ] 4109203     5,54K/s  eta 79m 33s

So, highly successful mac shareware which I love couldn't deal with bandwidth issues and offloaded the downloads to Amazon S3. Amazon S3 on the other hand, showing it works perfectly (on status page) has 450 ms ping response and I am back to 56K speed on a 4 mbit ADSL line. It looks like something wrong with Level3 hops.

Cloud is not offloading all mail to one central server nor putting all files to Amazon S3, it doesn't even exist yet. When people do 10x realtime h264 encoding with their Xgrid enabled portable Macs running Snow Leopard and store the file anonymously to thousands of other machines, that would be some kind of "cloud". Right now, Cloud is just an icon for that overpriced me.com (dotmac) service :)

Re:single point of failure (1)

Nick Ives (317) | more than 4 years ago | (#27956079)

e-mail is supposed to be reliable because of its distributed nature.

You seem to be missing the point of what I'm saying. When your email server is down, you can't send or receive mail. This leads to lots of irate phone calls about why you haven't replied to or sent some email. When everyone's email is down, you get the occasional call about how it sucks that email is down because so-and-so wanted you to do <trivial task>.

Even better, more complicated things that involve moving attachments have to be postponed which leaves you to catch up with your real work! Plus nobody gets irate because everyone has the same problem.

Cloud is not offloading all mail to one central server nor putting all files to Amazon S3, it doesn't even exist yet ... Right now, Cloud is just an icon for that overpriced me.com (dotmac) service :)

Yea, we've all noticed that "The Cloud" doesn't exist yet. TFS was made a comment about how scary the thought of running email on the cloud was due to the fact of it being a single point of failure, I'm making the point that's not such a bad thing.

Incidentally, the organisation I work for has a centralised email service and completely prohibits the use of all external mail services for security reasons (all the popular ones are blocked for good measure). The situation I outline above about how the occasional email downtime is a nice little break to get on with real work is from personal experience.

I look forward to a future where The Cloud makes those moments possible for all :)

It's a feature, not a bug (3, Insightful)

ColdWetDog (752185) | more than 4 years ago | (#27954349)

It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once

If everybody goes down, nothing happens and you just go outside (beyond the doors, out into the bright white light) and enjoy your day until 'they' fix it.

What's not to like?

Re:It's a feature, not a bug (1)

calmofthestorm (1344385) | more than 4 years ago | (#27954429)

What's not to like?

If you're they;P

Re:It's a feature, not a bug (1, Funny)

Anonymous Coward | more than 4 years ago | (#27954875)

If you are the "they" you better get back to work!

Re:It's a feature, not a bug (0)

Anonymous Coward | more than 4 years ago | (#27955663)

Ha... hahahahahahaha. Work... yeah. I'd rather sit here eating my sandwich, drinking my Mountain Dew, and /.ing. :)

Re:It's a feature, not a bug (0)

Anonymous Coward | more than 4 years ago | (#27955245)

Don't you know geeks are closely related to vampires? Pasty, drinking caffine as their lifeblood, up at odd hours. Forcing exposure to the sun could decimate the population!

Time sensitive work... (1)

msimm (580077) | more than 4 years ago | (#27955531)

Notes or contacts causing important meetings to be missed or leaving attendees un/less prepared. It's easy to say back everything up, but in the real world under stress (or laziness, or stupidity) you tend to stick with simpler work-flows. I like Saas for non-critical applications, maybe it's an age thing or maybe critical service/hosted solutions are simply still new enough that the kinks in reliability haven't been fully worked out.

Re:It's a feature, not a bug (0)

Anonymous Coward | more than 4 years ago | (#27956177)

It's great! While investigating the complaints from users, I checked the routers in the machine room. Pretty quickly saw google was down again and the net was fine. One of the young receptionist joined me, and now I have somewhat less heavy balls. Thanks DoSers, same time again tomorrow?

Then download your google mail (4, Insightful)

LingNoi (1066278) | more than 4 years ago | (#27954357)

This is exactly what makes me nervous about cloud computing and data storage. It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once?"

If it bothers you then use a mail client to download your mail from Google. As someone that has been using my gmail account all week I didn't even notice a problem, the whole thing seems overblown.

Re:Then download your google mail (1)

lothos (10657) | more than 4 years ago | (#27954479)

POP access wasn't working for the affected users. I didn't notice any problems, but a lot of people did.

Re:Then download your google mail (4, Informative)

Genocaust (1031046) | more than 4 years ago | (#27954689)

You don't even need to run POP to do this; just use the gears widget for offline access.

Re:Then download your google mail (2, Insightful)

Botia (855350) | more than 4 years ago | (#27954791)

The problem wasn't just mail. Any site that used Google for web statistics, mapping, or other services that Google offers was affected. For example, certain online banking systems use Google Analytics. These were affected.

Re:Then download your google mail (1)

Ash Vince (602485) | more than 4 years ago | (#27955113)

The problem wasn't just mail. Any site that used Google for web statistics, mapping, or other services that Google offers was affected. For example, certain online banking systems use Google Analytics. These were affected.

Strange, I am responsible for several sites that use Google analytics and I had a nice quiet day. They are fairly intensively monitored so if this had affected us I would have heard horrible alarms going off and clients ringing us.

Maybe the issue was more localised than people making it sound.

Re:Then download your google mail (1)

Mister Whirly (964219) | more than 4 years ago | (#27955221)

I didn't notice any outage in Google services either. But I only use it for mail, maps, and searching. I checked my email at least a dozen times today with no problems.

Re:Then download your google mail (3, Informative)

Ash Vince (602485) | more than 4 years ago | (#27955237)

I know it is poor form to reply to your own posts but I have just read the full article above and discovered that us in the UK seemed to be ok. Also not affected was the West coast apparently.

Maybe someone told Google I was on holiday tomorrow and needed a nice quiet day to clear my desk :)

Re:Then download your google mail (1)

jadin (65295) | more than 4 years ago | (#27955093)

Not only that but if your personal mail server goes down, there's just you fixing it. When google's does, how many hundreds if not thousands of people are scrambling on red alert to fix it?

Re:Then download your google mail (1)

MobileTatsu-NJG (946591) | more than 4 years ago | (#27956009)

If it bothers you then use a mail client to download your mail from Google. As someone that has been using my gmail account all week I didn't even notice a problem, the whole thing seems overblown.

I've had a lot more lost Google downtime caused by power outages or ISP service interruptions than I've had with Google being down. So, yeah, I agree with you, very overblown. Doesn't matter how dependent we are on the cloud, we still cannot take the internet for granted.

Big Deal (1)

mandark1967 (630856) | more than 4 years ago | (#27954387)

Considering the amount of usage google sees, a minor interruption like today's issue is nothing that worries me much at all.

It's not like oh, say, Comcast, who left me without an internet connection for a month because their technician was drunk and rammed his truck into the large metal junction box where my apartment's internet connection tied into everyone elses. It only took them a month to replace the box and re-wrire everything

Re:Big Deal (4, Insightful)

westlake (615356) | more than 4 years ago | (#27954865)

Considering the amount of usage google sees, a minor interruption like today's issue is nothing that worries me much at all.

But usage is precisely the point.

I lost access to Search, News, E-Mail...

Everything Google.

To a casual user at home this doesn't matter - but try explaining a global blackout of Google to your boss.

Google is the poster child for the web-based app.

Computing in the cloud.

Re:Big Deal (4, Insightful)

rAiNsT0rm (877553) | more than 4 years ago | (#27955211)

Finally someone commenting with some sense. It kills me to read all the "Great Job! Google!" and "Bravo!" comments. This exposes a serious flaw in planning, design and change management of a very heavily relied upon resource.

There is nothing to give kudos for here. Gotta love blind loyalty.

Re:Big Deal (2, Insightful)

OverlordQ (264228) | more than 4 years ago | (#27955759)

But usage is precisely the point.

I lost access to Search, News, E-Mail...

but try explaining a global blackout of Google to your boss.

Well that sounds like your problem really. Guess you shouldn't sold him on that single point of failure idea then huh?

Mail Servers (5, Insightful)

Aladrin (926209) | more than 4 years ago | (#27954419)

Having run my own mail server, and used mail servers run by companies I work for, I'll -gladly- take GMail's track record for reliability. Even with no 'guarantee', it's been a hell of a lot better than anything else I've experienced.

And what's -really- the difference between a server going down locally that affects you and a server going down globally that affects you? Nothing.

Re:Mail Servers (5, Informative)

ACMENEWSLLC (940904) | more than 4 years ago | (#27954627)

>>And what's -really- the difference between a server going down locally that affects you and a server going down globally that affects you? Nothing.

Actually, I disagree. There is a difference. If it's local and I own it, I have to fix it. If it's outsourced and Google owns it, I sit back and let Google fix it. Which is nice.

ThePlanet.com had a bad switch install a few days ago which brought down part of our cloud. Our website was down, as was our access to Google DNS gave an IP down there for Google. If you look at the last year, the cloud solution has had a better uptime than what I was providing computing in planned maintenance, patching, updates and all.

It was nice to leave at 5pm, knowing ThePlanet would fix the switch and get us back up. And they did. It's a lot easier to gripe about the cloud being down and sit back, than to manage and fix your own local servers switches and such. When you get to managing hundreds of servers, it becomes time to know what to outsource.

Re:Mail Servers (5, Insightful)

Cro Magnon (467622) | more than 4 years ago | (#27955891)

The flip side is, if it's local and important to me, I'll make fixing it a priority. If it's important to me and I DON'T have control, I just have to hope that it's reasonably important to whoever can fix it, or I'm screwed.

Google vs. Twitter...Once in an e-lifetime. (4, Funny)

geekmux (1040042) | more than 4 years ago | (#27954499)

Take a good look kids. Google was down and Twitter was up. This only happens once in every 3,271 days. You probably won't see it again, at least in Twitters lifetime...

Re:Google vs. Twitter...Once in an e-lifetime. (0)

Anonymous Coward | more than 4 years ago | (#27955927)

Looks like Twitter is currently down...

The news is not that Google went down (4, Insightful)

cwgmpls (853876) | more than 4 years ago | (#27954581)

Anyone who has ever used or administered a mail server has experienced a mail server going down. This is not news.

What is news is that Google Mail has been up for so long until now. And current accounts seem to indicate the outage lasted about one hour.

One hour of down time after five years of steady service is good enough for me. It is better than any other mail server I have ever used.

Re:The news is not that Google went down (1)

JWSmythe (446288) | more than 4 years ago | (#27954765)

  Well, I've administered a lot of mail servers over the years, and even when I've announced an outage, it's pretty much guaranteed that I'll get a phone call within 30 seconds of taking the machine down.

    I've noticed Gmail having problems quite often lately. Mostly the inbox can't load, times out, whatever. Not that I'm complaining though. It's free, and I can keep a copy back here for when they go under. :) I just don't look forward to copying my mail back up to my own server. It took about a day to bring it all down from gmail.

Re:The news is not that Google went down (0)

Anonymous Coward | more than 4 years ago | (#27954805)

It wasn't just Mail. I couldn't load sites with Google Ads and I couldn't Google anything. RSS feeds failed, etc...

Re:The news is not that Google went down (1)

HikingStick (878216) | more than 4 years ago | (#27954841)

Not to mention that it didn't seem to affect all of their users. Bravo, Google! Bravo, again for restoring services so quickly.

Re:The news is not that Google went down (4, Interesting)

teknopurge (199509) | more than 4 years ago | (#27955129)

You mistakingly act as if this is the first time google has had an outage in 5 years. Try again. [google.com] Some more too [google.com] .

Over the years there have been countless issues with google - from gmail being down to apps not working, though it tends to to affect everyone, but subsets of users.

Some of the google issues have to do with mailboxes getting lost and reassigned, etc. If it doesn't happen to you, it doesn't count as an issue, according to your logic.

Re:The news is not that Google went down (1)

socketwiz (792252) | more than 4 years ago | (#27955347)

Anyone who has ever used or administered a mail server has experienced a mail server going down. This is not news.

google != gmail

It was more than just mail that went down it was all of Google's services. If it were just mail, only a few people would care. It would not have affected me at all.

Re:The news is not that Google went down (1)

drinkypoo (153816) | more than 4 years ago | (#27955659)

I was not affected at all by this outage (I have been using gmail all day, no lie) but that could be because I am using offline gmail... but I was sending and more importantly receiving mail. I guess it could have happened and been over before 6:08 pacific... no, it looks like it happened later. I have replied mails from all around that time.

I have seen gmail outages before, so I don't really know why this is allegedly news. None of them lasted long though. Maybe they were just rewriting my email or something :D

It's that reliable... (0)

Anonymous Coward | more than 4 years ago | (#27955701)

...and it's still in beta!

Re:The news is not that Google went down (1)

Cro Magnon (467622) | more than 4 years ago | (#27955987)

If my email server goes down for an hour, I probably won't even notice. But if the adservers are down, causing the whole internet to run slower than Vista on 512MB, I not only notice but get very annoyed.

blacklash against the cloud? (4, Interesting)

recharged95 (782975) | more than 4 years ago | (#27954659)

In the end, who the F* cares if a cloud service goes down?

If a life is not lost, there are no worries with cloud computing (hence, cloud computing should be used for non-life critical services, gmail is a perfect example).

Of course, VCs may have lost revenue, Capitalists may sweat from loss stock trades, teenagers may lose that one twitter about how cool Miley is to them, some adult may not get that date tonight from craigslist, you may miss that one Hulu commercial, some K-12 kid may not be able to send out his homework, some college kid can't access his pirate bay music lists, or the USPoTC may miss that extra minute to promote his stimulus bill.

In the end, I hope cloud services shows us that we are not slaves to time. The human race has advanced enough to know that already. And really, if "the cloud" is down for an hour, maybe you should go outside and enjoy the wonders of nature and peace for once, or talk to someone physically. It begs to ask the question: "can it wait?"

Re:blacklash against the cloud? (2, Funny)

levicivita (1487751) | more than 4 years ago | (#27955971)

It is amazing how you were able to post this brief message, sitting as you are, deep in the Amazon jungle, thousands of miles away from civilization surrounded by mystery and adventure, your copy of Thoreau's Walden lovingly ensconced in your chest pocket, close to your heart.

It's better when it's all of Gmail going down (1)

rduke15 (721841) | more than 4 years ago | (#27954703)

"It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once?"

That's much better for you. Instead of having to explain to everybody that the dog ate your homework or whatever, you can sit back and let them explain it to you...

Good Point (1)

Dodder (1410959) | more than 4 years ago | (#27954773)

It will suck when everything's on the Cloud because I won't be able to claim my server's been down all day while I'm out playing golf.

One service that was definitely affected (1)

AntiRush (1175479) | more than 4 years ago | (#27954929)

was Google Voice. I've been using it for a while as my non-mission critical line and starting this morning calls stopped going through. I was also unable to dial out through their service.

When things came back up this afternoon it was an old backup version and several of my settings had been rolled back.

I guess this is one instance where Google's perpetual beta status really applied - those using Voice for mission critical communications were up a creek.

Also shows dangers of electrical grid (1)

blamanj (253811) | more than 4 years ago | (#27954951)

Ah, we all get our power from the "electrical cloud". We all need private generators. Ah! Ah!

Forcing denial of service on unrelated sites (3, Interesting)

roc97007 (608802) | more than 4 years ago | (#27954979)

If we're talking about the same outage that caused google advertisements to hang forever this morning, it caused access to many unrelated websites to hang, including slashdot itself. This seems like a really bad single-point-of-failure issue. If a site can't display ads, shouldn't it come up anyway?

It's bad enough that I have to wait tens of seconds for Captcha content to pop up long after a login page has loaded.

This is starting to get annoying. If this is "cloud computing", I'd rather stay on earth.

this again proves (1)

anonymousNR (1254032) | more than 4 years ago | (#27955031)

why yahoo,hotmail never go down , because nobody is using them. Heck I haven't opened mine in months now.

I'm glad I use Hotmail (-1, Troll)

Anonymous Coward | more than 4 years ago | (#27955043)

Fuck y'all!

Sweet Jeebus you people miss the point (0, Troll)

rAiNsT0rm (877553) | more than 4 years ago | (#27955085)

I love all the fucktards who keep saying: "oop, the cloud's down I'll go for a stroll" or "welp, google's down, I'll go home." Where in the hell do you work? Your phone is going to be lit up like Times Square with all the user calls/complaints for hours. And just up and leaving offers zero customer service to users who rightfully don't know what is actually wrong.

Shit's down, whether it is your or not you are seen as responsible and at least have to offer some communication and support. The problem is you look bad because you can't tell anyone an actual ETA or valid explanation besides a shrug of the shoulders and a "hopefully CompanyX gets it fixed soon."

Cloud computing can be a great thing but this shows that there are fundamental flaws still. I have run systems that could have zero downtime and achieved it. Yes, it requires redundancy. Yes, it is expensive. Yes, it requires geographically and ISP independent sites. Yes, it requires planning. But it can be done, so stop all the bullshit praise that because it is Google and they are big, this is OK. It isn't. if anything they should NEVER have this kind of issue.

The Google-colored glasses need to be taken off.

Re:Sweet Jeebus you people miss the point (1)

drinkypoo (153816) | more than 4 years ago | (#27955707)

Hence the reason why we need a whole storm of clouds... and some APIs for submitting the same jobs to multiple clouds. If one goes down you start sending them off someplace else (maybe someplace slower or more expensive) for the duration of the outage.

Re:Sweet Jeebus you people miss the point (0)

Anonymous Coward | more than 4 years ago | (#27955833)

I love all the fucktards who keep saying: "oop, the cloud's down I'll go for a stroll" or "welp, google's down, I'll go home." Where in the hell do you work?

Not all of us fucktards do network support.

Jackass.

Re:Sweet Jeebus you people miss the point (0)

Anonymous Coward | more than 4 years ago | (#27956275)

I don't see what the big deal is. You could've easily had numerous virtual machines running at Rackspace, Amazon, wherever else and done the global load balancing externally. It's just compute power without the hassle of dealing with the hardware directly. I don't see anyone claiming the 'cloud' is the be all end all solution for all things, but it is extremely cost efficient from both a hardware/labor/uptime/cost perspective for tons of business solutions.

The "cloud" in cloud computing is the internet... (4, Interesting)

GPLDAN (732269) | more than 4 years ago | (#27955137)

When done correctly, the "cloud" is the internet itself. Google has network design issues, some of their key services only have a couple of ingresses into Tier-1 providers:

http://en.wikipedia.org/wiki/Tier_1_carrier [wikipedia.org]

I don't work for them, i don't hold their stock, and I am not (currently) a customer, so I have no skin in their game, but Internap as a BUSINESS MODEL, becomes more important.

If you are a major company that comes to rely HEAVILY on Cloud Services, you want to insure that you have on-ramps into several Tier-1 providers ALL AT ONCE, without having to contract individually with 4 or 5 of them yourself. I predict more companies will mimic this model of aggregation, essentially handling the business of BGP optimization for customers, and handing customers 2 redundant pipes and saying "hey, don't worry if San Fran has an earthquake and these peering points blow up, we'll get you out via this Tier-1 backbone over to your cloud computing provider's service via this backbone within seconds. Let us handle that."

Especially with ISPs that get into pissing matches, like when Cogent and Telia got into it, and cut each other off. If you had Cogent as your only ISP, you were screwed if you wanted to get to a bunch of Swedish sites, because Cogent's CEO was trying to play chicken over some tariff rates. The cloud computing model will no longer tolerate that, it's not just some website, it's a BUSINESS function.

that's my take at least.

It was like the whole net was down (1)

warrior389 (314070) | more than 4 years ago | (#27955201)

while I was trying to get work done today. This was pretty scary. I mean, besides not being able to search google and check my email, there are other sites that wouldn't work. Some apache projects and also nabble use google analytics apparently, so I couldn't even load those pages. Also, I couldn't load slashdot's main page because it apparently uses googleads or something like that. What suggestions to people have for this? What other sites were not accessible during the outage?

Re:It was like the whole net was down (1)

AngryK9 (1553903) | more than 4 years ago | (#27955511)

(Sorry, I really couldn't resist posting that other reply :D ) What I do is block the Google adsense servers on my Windows machine in the HOSTS file. Add the server name (pagead2.google.com or whatever) and redirect requests for that page to localhost (127.0.0.1) I would say try a Google search for directions on blocking ads with the HOSTS file, but perhaps you should try Yahoo Search instead. :P

Much more broad than it may seem... (1)

davecrusoe (861547) | more than 4 years ago | (#27955205)

Many sites rely on Google in ways that aren't immediately evident - for instance, during the outage, Google Analytics connections were lagged, which meant that all our our sites that incorporate Analytics were ALSO lagged.

What's amazing is the extent to which an outage on a single entity can bring down ALL of the other entities that surround it -- not just those who rely more visibly, e.g., Google Docs., on their services.

Yikes!

--Dave

Re:Much more broad than it may seem... (1)

saiha (665337) | more than 4 years ago | (#27955515)

Its because of very poor design. Its the same reason that slow loading adverts slow down a site. One small aspect of a site should not affect its performance the way it does today.

Re:Much more broad than it may seem... (1)

javaxjb (931766) | more than 4 years ago | (#27955781)

I thought maybe something had corrupted my Firefox session at the time... I suspected Google was having problems, so I went to E*Trade which was failing to load just about anything except text. I don't see any references to Google or ga.js on E*Trade's pages. But, E*Trade does rely heavily on Akamai servers. If it was a DOS attack, it may have affected Akamai, too (either as the subject of a separate direct attack or an indirect victim of traffic generated by Google's problems).

Both Google and E*Trade recovered at the same time.

Don't panic (1)

AngryK9 (1553903) | more than 4 years ago | (#27955285)

Everything is fine. It was simply a glitch in the holo-matrix. The Doctor has been tinkering with his program again and caused a feedback loop between the holo-emitters and EPS conduits on deck five. Seven has corrected the malfunction.

Cloud computing is better (1)

liam193 (571414) | more than 4 years ago | (#27955553)

It's bad enough when I screw up a config and it takes down my mail, but what about when it happens to the entire globe at once?

I was reading this comment and it occurred to me that the latter is actually preferred. With the first option, your systems are messed up, but everyone else wants you to continue to conduct business. With the latter situation, your systems are down and so are the people who would normally be trying to reach you.

humor? (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#27955661)

I'm pretty let down by you people commenting today. there are usually at least a handful of gems for me to laugh at, but you guys. you're taking this WAY too seriously. Make me laugh, that's what you're here for!

I was my fault (0)

Anonymous Coward | more than 4 years ago | (#27955669)

I think I did something wrong: I was going to open google, but wrote the address in the firefox search box instead of the address bar. Suddenly, the internet went mayhem. Any clues what went wrong?

Coining a new term, you heard it here first (0)

Anonymous Coward | more than 4 years ago | (#27955697)

The Fog: What happens when The Cloud is down.

Why added speculation in post? (1)

Jim Efaw (3484) | more than 4 years ago | (#27955699)

This speculation from the ComputerWorld blog doesn't belong in the post. Even the blog author says its conjecture. Especially ridiculous since the NANOG post in the second link already explained that the problem was a routing error at Google.

Re:Why added speculation in post? (0, Offtopic)

Jamie's Nightmare (1410247) | more than 4 years ago | (#27956171)

I second that. Even worse, it's written by Steven J. Vaughan-Nichols, a prolific Linux Troll who takes any opportunity to crap on Microsoft while hailing Linux as the greatest thing ever. In this case he's blaming the Conficker worm for a DOS attack on Google. No proof, but his articles never need any. You might wonder why he hasn't gotten fired yet, but angry readers are readers none the less. It's all about the ratings.

fir5T (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#27956137)

unpleasant 'this Mistake or Goals I personally
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...