US Amazon.com Website Down For Over 1 Hour 228
CorporalKlinger writes "CNET News is reporting that Amazon's US website, Amazon.com, has been unreachable since 10:30 AM PDT today. As of posting, visiting www.amazon.com produces an 'Http/1.1 Service Unavailable' message. According to CNET, "Based on last quarter's revenue of $4.13 billion, a full-scale global outage would cost Amazon more than $31,000 per minute on average." Some of Amazon's international websites still appear to be working, and some pages on the US Amazon.com site load if accessed using HTTPS instead of HTTP."
Beer on the server? (Score:5, Funny)
Re:Beer on the server? (Score:4, Insightful)
It's not like there are a lot of alternatives out there. Sure, some specialized places might fill part of the bill, but once you've become accustomed to Amazon, you more or less stick with em.
But... (Score:5, Informative)
Also now you are Slashdotting it!
Re: (Score:2, Funny)
This will surely help (Score:5, Funny)
Re:This will surely help (Score:5, Insightful)
Only exceptions would be if there was a lot of heavy content being served on each page turn, saturation of one's uplink is a possibility - 10Gb links to the backbone aren't that common as yet, and CDNs like Akamai helps alleviate a good portion of that traffic.
My totally unsubstantiated guess is there was some DNS fooage that directed sites to a down cluster or possibly a screwed up CDN leg, but I'll be interested to see what's truly up.
sloth jr
Re:This will surely help (Score:4, Insightful)
Re:This will surely help (Score:5, Interesting)
Re: (Score:3, Informative)
Re:This will surely help (Score:5, Insightful)
Digg sends far more traffic to a site than Slashdot does (obviously it wasn't always this way). And digg's traffic isn't particularly noteworthy to a site of any reasonable size. (Say, Ars Technica, nevermind amazon.)
Yahoo Buzz, on the other hand, sends *huge* amounts of traffic, noticeable to sites like, again, Ars but again no disruptions of service*. But I doubt that amazon would even hiccup. If you think slashdot would even be a blip on amazon's radar, you have some serious delusions about 1) slashdot's size 2) amazon's size or 3) both.
* According to one of the devs.
Patents (Score:5, Funny)
Re:Patents (Score:5, Funny)
Great move! (Score:2, Funny)
Re:Great move! (Score:5, Funny)
Which is cheesy awesome.
So, it finally happened... (Score:5, Interesting)
Believe me, if you've seen the code that runs that site, it's impressive it runs as well as it does. Try to imagine 900M static binaries that take almost an hour to link because of some tiny little code change, because they can't be fucked to make their deployment system deal with dynamic libraries reasonably.
Re:So, it finally happened... (Score:5, Interesting)
Believe me, if you've seen the code that runs that site, it's impressive it runs as well as it does. Try to imagine 900M static binaries that take almost an hour to link because of some tiny little code change, because they can't be fucked to make their deployment system deal with dynamic libraries reasonably.
Fuck up a dynamic library and you fuck everything. Fuck up one of those 900M programs and you've fucked 1/900M'th of everything.
What does Amazon's back end compile for? If it's Linux, that's an issue right there. The GNU linker has pathological behavior when linking large numbers of static libraries. I work on a relatively small (~1 million line) codebase and it takes about ten minutes to link. Link it on another platform (e.g. Solaris) and it links in about five seconds.
The problem isn't the huge number of libraries. The problem is that the linker blows.
Re:So, it finally happened... (Score:4, Informative)
GP is approximately 3 years out of date. See Gurupa [wikipedia.org]
Since I can't give any details directly, I'll let wiki [wikipedia.org] do it.
Re: (Score:3, Informative)
I think her point was that there's one gigantic binary, not an enormous number of tiny ones.
Re: (Score:3, Interesting)
Ian Lance Taylor said...
ralph: The main difference in gold is that it was designed from the ground up to work for ELF. The GNU linker was designed to work for a.out and COFF.
ELF conceptually requires three passes over the object files, a.out and COFF require two. A version of the third pass was hacked into the GNU linker by using a backpatch system on the symbol table, in which the GN
Re: if you don't like the GNU linker use Gold (Score:3, Informative)
Re: (Score:2)
Being able to avoid the incessant act of compiling/linking is probably what leads to so much sub-par PHP code in the first place - you're less likely to be moulded into a mindset of getting it right the first time
Re:So, it finally happened... (Score:5, Interesting)
Re:So, it finally happened... (Score:5, Funny)
Re:So, it finally happened... (Score:5, Funny)
Re: (Score:2)
Re:So, it finally happened... (Score:5, Informative)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Insightful)
Re: (Score:3, Informative)
Seriously. She didn't even say what language or what platform they're running on, which is more useful and easy for even a non-employee to figure out.
Re: (Score:3, Funny)
D&D did it. (Score:5, Funny)
Re: (Score:2)
Re:D&D did it. (Score:5, Funny)
Re: (Score:2)
Re: (Score:2)
I want my D&D!
(partially) works for me... (Score:3, Informative)
So, it seems to be working...at times.
D&D (Score:2, Funny)
Re: (Score:2)
Seriously, look at the DnD 4.0 thread... I don't think I've seen something go over so lackluster since the Segway came out.
How much lost? (Score:3, Insightful)
Even if accurate, that's assuming everyone who sees the error message will go somewhere else to buy their books.
I imagine some people would just wait to buy the book from amazon later when it is up again (probably very soon).
Re: (Score:2)
And Amazon, in my mind, doesn't quite have the nice comfy natural monopoly of network effects (think ebay, youtube or even slashdot). Nothing amazon sells cannot be bought elsewhere at about the same price. (That said, I'm a long-term and satisfied Amazon customer who tends to buy there unless I can save several bucks buying elsewhere).
Somebody write a book about this please (Score:2, Funny)
They Think I'm a Robot (Score:5, Funny)
Well I think THEY are the robot. I don't know if I can win this argument...
Re: (Score:2)
Elizamazon: Do you wish that you can win this argument?
Re: (Score:2)
Re: (Score:2)
We're sorry!
You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com.
We apologize for the inconvenience.
Frequently Asked Questions
Q: Why am I seeing this page?
A: This page is usually shown when we believe that the request is coming from a robot or other automated source of requests. If you are not a robot please contact us immediately by emailing ad-help-us@amazon.com and we will reinstate your access to our website.
Q: How can I operate a robot and not get this page?
A: We understand that there are many legitimate reasons for robots to access our website. We are happy to work with people trying to create robots so that they may do so safely and efficiently. If you are operating a robot and you are seeing this page we'd love to hear from you so that we may better understand your use case and help you to achieve your goals. Please email ad-help-us@amazon.com and we'll help you out - seriously, we aren't mad at you.
Q: What are some general tips for people writing robots?
A: First, you should see if there is a better method to get the information you need. For example, Amazon Web Services provides a rich set of APIs to retrieve the information displayed on many of Amazon's web pages (prices, reviews, sales rank, etc). Because Amazon Web Services exposes a stable set of APIs that provide structured data it is often much easier to retrieve information via this method. You'll be able to find out more about Amazon Web Services at http://aws.amazon.com./ [aws.amazon.com] Second, you should identify your robot using a unique user agent string that provdes a way we can get in touch with you if necessary. For example, here is Google's user agent string Googlebot/2.1 (+http://www.googlebot.com/bot.html).
Using Safari (not logged into an Amazon acct), I get that message. Using Opera (logged in with an acct that has been used for years), I get:
We're sorry!
An error occurred when we tried to process your request. Rest assured, we're already working on the problem and expect to resolve it shortly.
If you were trying to make a purchase, please check Your Account to confirm that the order was placed.
We apologize for the inconvenience.
They havin' problems. ;)
Re:They Think I'm a Robot (Score:4, Funny)
The problem isn't Amazon . . . (Score:5, Funny)
$31,000 per minute! (Score:5, Insightful)
Re:$31,000 per minute! (Score:4, Insightful)
Well, they will frequently come back, yes. But the site being down also affects consumer confidence in a big way and that will make fewer people likely to go to the site.
So, using the metric of exactly how much you sell in a given time period is likely inaccurate, but I suspect the actual impact is higher, not lower.
Re: (Score:3, Insightful)
C'mon, how many people are really going to stop buying from Amazon because their website was down for a few hours on June 6, 2008?
Re: (Score:2)
Opus:HSN::bughunter:Amazon.com
Re: (Score:2)
If you're buying a $40 tent, I hope that all you are planning on doing is camping in the basement. At least pray that it doesn't rain.
Re: (Score:2)
still not working AFAICT (Score:2)
We're sorry!
An error occurred when we tried to process your request. Rest assured, we're already working on the problem and expect to resolve it shortly.
If you were trying to make a purchase, please check Your Account to confirm that the order was placed.
We apologize for the inconvenience.
on the Amazon.com home page
thinks I am a robot (Score:3, Interesting)
Re: (Score:2)
I got this too, and I thought it was because I set up a little script that checked the homepage every 5 seconds to let me know when it came back up (because I wanted to purchase something today and get it ASAP).
I sent them an email to that address, and I had access again an hour later. I didn't get a response from them though, so I don't know if they did something to unblock my IP address or if it had nothing to do with my script and was just related to the outage.
Anonymous Coward (Score:2, Informative)
I received word about 30 minutes ago that Amazon has been the victim of a DDoS attack this morning. At first, their Ops team didn't realize they were under attack and thought it was a traffic spike related to a promotion, but after about an hour of throwing hardware at the surge they realized what it was. And once they tubed the source IPs in the botnet another crop of zombies showed up.
It looks like they are getting a handle on it now as thing
AWS and EC2 (Score:3, Interesting)
A bit strange, the people wondering why this is news. Amazon provides the backend for a number of web services with their EC2 and AWS platforms. This is going to make third parties seriously consider whether or not they want to trust Amazon with their business.
That is yet another reason why this is Real News(tm).
Re:AWS and EC2 (Score:5, Informative)
Re: (Score:3, Insightful)
I know it's been running fine, I happen to use AWS.
But for business purposes, that fact isn't going to matter much to a PHB. What a PHB is going to remember is "Gee, didn't they have a serious outage a little while ago... better use something else!" Even if the best solution is, in fact, AWS + EC2.
Perception is more important than reality in business, unfortunately.
Get better Amazon, we love you! (T_T) (Score:3, Interesting)
Amazon is as good as eBay-Paypal is evil. Both are outstanding products but one is loved and one is hated.
Sooo...in the time that I wrote this post, Amazon lost enough money to sustain me my entire life. That's sad.
Cost of outage (Score:4, Interesting)
Stephan
Re: (Score:2)
Stephan
Re: (Score:2)
It just looks a little simple to divide revenue of a quarter down to the hour and use that as an estimate for the cost of the outage.
For example I wouldn't be surprised if half of the buyers return later on to order the books they couldn't because of an outage.
StephanDown again (Score:3, Informative)
Amazon wishlist widget temporarily down (Score:2)
Amazon declares outage is over. Light on details. (Score:5, Informative)
"But as to the explanation, the company only hinted that its complicated computing infrastructure was, unsurprisingly, a culprit.
'Amazon's systems are very complex and on rare occasions, despite our best efforts, they may experience problems. We work to minimize any disruption and to get the site back as quickly as possible," the company said, declining to comment further.'"
Re:How is this news? (Score:5, Insightful)
Re:How is this news? (Score:5, Insightful)
Re:How is this news? (Score:5, Insightful)
Its less than 31k, but still not cheap (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Funny)
Re:We're sorry... (Score:5, Funny)
Re: (Score:3, Funny)
Re: (Score:3)
Sure! How 'bout:
It's like trying to break a car window [youtube.com] by shooting spitballs [wikipedia.org] at it....
Re: (Score:2)
Re:Analogy (Score:5, Funny)
Out in the back, working on his car.
Re:Analogy (Score:5, Funny)
Re: (Score:3, Informative)
Re: (Score:3, Funny)
There, there little troll. Please continue your nonsensical rant.
Re: (Score:2)
Re:OH NOES (Score:5, Insightful)
You'd have to factor in the ratio of income from the
US site v others (UK, etc.). IMHO, the US site is likely to be more profitable than others. You'd have to plow through an annual report to really know, and factor that in.
The larger flaw, though, is that you're subtracting one minute, when the title states > 1 hour. That implies going on A couple of million US$ in losses, which is significant, as investors don't know the reason, and caution would indicate that it could be recurring, such as the problems SalesForce has had. That hit their stock prices, etc.
The Amazon outage is more complex--TFA indicates that some of their services were unavailable for different amounts of time, etc. What are those service worth? All anyone has is a number--from CNET. Did they do anything like a real analysis, reading quarterly reports, etc? No, by long odds. Amazon does application hosting. What customers were affected, what percentage of the business is involved, and what do CxOs of large clients think?
The odds are actually quite good that many people give a crap. Investors (and CxOs) don't like uncertainty. It wouldn't surprise me to find some Wall Street analyst(s) making calls. Maybe it was an outage on a critical replication server, problem identified, fixed, and will provably never happen again. But maybe not. We'll see.
Re: (Score:2, Informative)
So, because some loser can set up a few records in a completely different domain, this is supposed to somehow have some effect on Amazon's DNS?
Re: (Score:2)
Re: (Score:2)
With Amazon, this isn't really shocking at all. I'm soooo glad I don't work there any more.
Re: (Score:2)
Re: (Score:3, Informative)
It's a giant cube farm, and their code is like some sort of crawling horror of reanimated spaghetti which long ago swallowed up and devoured all documentation. And then there's the deployment system. As I mentioned in another comment on this article, it can't deal with dynamic libraries. When I left, it was a real and immediate issue how we were going to keep a certain product's dependencies small enough that it would be able to *link* in a 32-bit virtual address space. The linker was up to something li
Re: (Score:2)
Re: (Score:2, Informative)
The DNS servers for a domain name are announced in root DNS servers - and there, everything is fine. For example, dig NS amazon.com @a.gtld-servers.net return correct DNS servers: udns1.ultradns.net. and udns2.
However, dnsreport show lots of errors with nameservers:
http://private.dnsstuff.com/tools/dnsreportsmpl.ch?d [dnsstuff.com]
Re: (Score:3, Funny)
Re:do a whois. Looks like DNS got pwn3d. (Score:5, Informative)
However, as has been pointed out, HTTPS works, so it's defininitely not a DNS issue. More likely someone along the chain corrupted a pooling link to the main http server and it propogated. I've done the same thing on apache2 servers in the past and had the same result; https still works fine, but http returns an error on key pages.
Re: (Score:2)
OLD CROTCHETY MAN MODE:
HOST RECORD
Re:do a whois. Looks like DNS got pwn3d. (Score:5, Informative)
A fully-qualified DNS domain name ends with a dot, so you should type 'whois amazon.com.' instead.
Those "hacked" results you are getting are just bogus amazon.com.foo.bar. subdomains.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Funny)
Re: (Score:3)