Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Surveying the World of the Biggest Server Farms

timothy posted more than 5 years ago | from the don't-forget-the-various-tlas dept.

Data Storage 106

1sockchuck writes "Rackspace said this week that it is managing more than 50,000 web servers, raising the question: who else has that many? Of companies that publicly discuss their server counts, there are only a handful that are near or above the 50,000 server mark, including 1&1 Internet, The Planet, and Akamai, as well as Rackspace. The larger totals are found among companies that don't discuss how many servers they're running. The leading suspects: Google, Microsoft, Amazon and eBay."

Sorry! There are no comments related to the filter you selected.

Shouldn't it be easy to figure out? (2, Interesting)

afidel (530433) | more than 5 years ago | (#27960823)

Figure servers per sq ft and add up their total datacenter sq footage. Googles a bit harder due to changing strategy over their current server lines but a good guesstimate shouldn't be too hard.

Re:Shouldn't it be easy to figure out? (2, Informative)

confused one (671304) | more than 5 years ago | (#27960961)

Doesn't always work. A lot of datacenters are not filled out because the power density of racks of modern servers is so high it can result in the datacenter exceeding the available electric service, if they filled out all of their space..

Re:Shouldn't it be easy to figure out? (2, Interesting)

afidel (530433) | more than 5 years ago | (#27961033)

Should still get you in the right ballpark, few companies are going to waste huge amounts of money building empty datacenters =)

Re:Shouldn't it be easy to figure out? (4, Interesting)

Gorobei (127755) | more than 5 years ago | (#27961451)

A lot of datacenters get built in repurposed buildings - the square-footage is often misleading (some are even 60's era compute farm housing - 90% of the space may be unused.)

For low-latency datacenters, you build in the middle of cities. Then you find square-footage really doesn't cover it: the fire-marshall shows up and red-tags you because he doesn't want a six mega-watt dense power sink in the middle of his premium real-estate.

Re:Shouldn't it be easy to figure out? (4, Insightful)

xp (146294) | more than 5 years ago | (#27961939)

Also hopefully they are not counting virtual machines here.
--
Slow Poke [pair.com]

Floor space requirements can change over time. (1)

Richard Steiner (1585) | more than 5 years ago | (#27966715)

Keep in mind that companies which use mainframes have seen a fairly large reduction in machine size over the past 20 years or so. Old water-cooled machines and rows of tape drives are being replaced with CMOS machines and tape silos, each taking of a fraction of the floor space that their predecessors did.

When I started working for Northwest Airlines, for example, there were multiple IBM and Unisys mainframes sitting in the main computer center, all residing in a large temperature-controlled raised-floor room.

These days, most of that floor space is empty. The number of mainframes hasn't been reduced at all ... they're just a lot smaller.

Re:Shouldn't it be easy to figure out? (3, Insightful)

JWSmythe (446288) | more than 5 years ago | (#27961073)

    That's going to come up with some off numbers.

    There are oddities in there. Like, Google has some decent square footage with QTS in Atlanta.

    Is the number, the number of machines in house, or the number of machines managed by the company in house? Places like Equinix have huge facilities, but they don't manage them (except for helping hands support).

    Just because a company takes a bigger footprint, does that make them bigger? My old place had an Alexa rank in the top 300, but the main sites were served from maybe 24 machines set up for well over 10 million users/day, not including hot spares and ancillary equipment (DNS, mail, etc). That was unique users, not requests or page views. :)

    I know Quantcast has a huge footprint, but they're in someone elses facility. I think it was only one of a few DC's that they're in. I didn't know it was their equipment until I talked to one of their techs. It was a very nice setup. The conversation with my coworkers went "I want to set our stuff up like theirs. Too bad we have dissimilar equipment, it'll never look so good."

    Depending on who you're looking at, the footprint isn't the front end either. Places like Google, Quantcast, and many others have a LOT of non-public equipment for doing the fun stuff. Like Google has a slew of spiders crawling, analyzing, etc. It's perfectly likely the could (not necessarily do) get away with just a few dozen front end machines for Google.com, and pass the work off to back end machines.

    It's all in the strategy that they use. If you're a multibillion dollar operation, do you squeeze every bit of power you have out of a machine, or stay real low and spread it across many? For my old place, I set up to squeeze everything I could out of them, and then spread out across machines so we could lose machines (hardware failures usually) without hurting the site. We didn't have the budget (the bosses liked the profit), but if I had really wanted, I could have probably spread it across thousands of machines. It just makes for headaches and larger IT staffs to keep up with it.

Re:Shouldn't it be easy to figure out? (2, Interesting)

0x15 (852429) | more than 5 years ago | (#27961205)

Many of the production data centers I've worked on have been using VMs for web servers for quite some time (e.g. ESX 2.x). That includes one of the companies that is on the list of unknowns in the article. I'm surprised it wasn't addressed and that so many jumped to a physical server conclusion. However, even if the 50k servers are all VMs, that's a major management load. 50k of just about any configuration item takes some work.

Re:Shouldn't it be easy to figure out? (1)

Venik (915777) | more than 5 years ago | (#27961389)

I am running 50,000 virtual servers on my three Dell laptops! Why am I not on the list? Seriously, are we counting physical machines or virtual ones? One system I support is a cluster of 80 HP 585s, each running eight virtual servers. So that's like 640 "servers" in just ten racks. It would be interesting to know who has the most servers in, say, kilograms as opposed to imaginary filesystems.

Re:Shouldn't it be easy to figure out? (1)

littlerubberfeet (453565) | more than 5 years ago | (#27961485)

Depends...do you count raw tonnage of servers, or do you include the ancillaries like cable runs, UPS, cooling etc?

Easy way to win:

My Eniac replica, combined with my replicas of Mayan and Egyptian pyramids (purportedly used as astronomical computers...you know, by the illuminati, etc ;)) means that I win by sheer tonnage!

Re:Shouldn't it be easy to figure out? (-1, Troll)

Anonymous Coward | more than 5 years ago | (#27961651)

Just tell a Jew that if he doesn't tell you the figure you are going to exterminate his entire scheming race.

Then exterminate the Jews anyway. God knows they deserve it.

50,000 web servers, not physical servers (3, Insightful)

Amarok.Org (514102) | more than 5 years ago | (#27960833)

They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

Yes, it's still impressive... but not as impressive as it would first appear.

Re:50,000 web servers, not physical servers (3, Insightful)

linhares (1241614) | more than 5 years ago | (#27960919)

They're using Netcraft to prove their server count - which reports on IP addresses. Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes. The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

Not exactly what netcraft is saying right now [netcraft.com] from TFAL:

Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups.

I suspect there's a power law in here, and that the estimates of google and amazon and others should be way beyond this, perhaps surpassing 500.000. That would be an interesting milestone.

FTFA:

There's a widely circulated estimate of 450,000 servers, but that number is at least three years old. If it was ever accurate, it certainly isn't anymore, given Google's data center building spree. Google's recently revealed container data center holds more than 45,000 servers, and that's a single facility built in 2005.

Re:50,000 web servers, not physical servers (2, Interesting)

ZosX (517789) | more than 5 years ago | (#27961071)

4 years ago. I wonder if all those boxes are still running right now? I wonder what google does when it retires servers.....it would be kind of cool to have a couple of bonafide google racks doing something cool at my house.

Re:50,000 web servers, not physical servers (4, Informative)

m.ducharme (1082683) | more than 5 years ago | (#27961289)

If this [google.com] is to be believed, they salvage and rebuild as much as they can, and send the rest to recycling.

Re:50,000 web servers, not physical servers (1)

wwwillem (253720) | more than 5 years ago | (#27961923)

Plenty of years ago I was in a large datacenter, where Google had a cage full of their racks. These were the old DIY "4 MBs on a tray" systems.

>
>it would be kind of cool to have a couple of bonafide google racks doing something cool at my house
>

Let me tell you something, you don't want to have those servers in your basement, that setup definitely wasn't cool :) the amount of heat that came out of that row of racks was pretty enormous.

OK, this was long time ago, I expect that Google's setup has changed a lot since then. At least they aren't in shared facilities anymore, where normal IT guys like me can have a look at their servers.

Re:50,000 web servers, not physical servers (0)

Anonymous Coward | more than 5 years ago | (#27962833)

Dude, that's eons ago, in tech-time.

Google's strategy, for some years now, has been to treat a datacenter as a "machine". Motherboards are just components in the machine.

The entire datacenter is highly optimized for power use (including lighting and cooling). Think about it: power must be Google's single biggest expense. probably by orders of magnitude. a 1% gain in some index efficiency likely translates into megawatt-hours of power.

Re:50,000 web servers, not physical servers (4, Informative)

nedlohs (1335013) | more than 5 years ago | (#27960953)

If you bother to RTFA you'll see it isn't a netcraft server count, but a mention in their SEC reported earnings something they are unlikely to just make up.

The comparisons are with netcraft numbers. And those netcraft numbers are explicitly not IP address counts and have rackspace as way under 50,000 (which you would expect since many machines wouldn't be web servers (database backends, mail servers, etc) and many would be firewalled to not allow public access.

Re:50,000 web servers, not physical servers (1)

trawg (308495) | more than 5 years ago | (#27961111)

The shared hosting arrangements can easily have dozens and dozens of "servers" operating on the same physical box.

Dozens?! I worked at a web host like 8 years ago and we were running up to 400 servers shared on a single box. I assume you can squeeze on a hell of a lot more these days!

Re:50,000 web servers, not physical servers (0)

Anonymous Coward | more than 5 years ago | (#27964115)

While computing power has gone up massively, the size and complexity of the content has also grown.

Re:50,000 web servers, not physical servers (1)

ronsonk (612948) | more than 5 years ago | (#27961115)

50,000 OS instances (and I'm assuming that each web server is running on in their own instance) is still 50,000 that have to be managed regardless if they are on their own physical server. It is indeed impressive.

Re:50,000 web servers, not physical servers (1)

JWSmythe (446288) | more than 5 years ago | (#27961199)

    You bring up the magic question, how many virtual IP's can be on one box? I don't know the limit on a Linux machine. I brought up an entire private /24 on one, just for the fun of it. So, if you considered that each of 50,000 machines had 254 IP's on them (a wild idea, but still), they could have just 197 machines.

Re:50,000 web servers, not physical servers (1)

0100010001010011 (652467) | more than 5 years ago | (#27961245)

No, they're using Netcraft for the companies that didn't release the information.

TFA:

Here's a look at some of the providers with high server counts, gleaned from public reports and partial data from a recent Netcraft server count report

There's a little note after each company where the data came from, all of the top 5 came from the company itself.

Re:50,000 web servers, not physical servers (0)

Anonymous Coward | more than 5 years ago | (#27961337)

Just because there are 50,000 IP addresses responding to port 80, doesn't mean they have 50,000 boxes.

You're right, but you didn't consider the other direction: it could be much more boxes.

Lots of big sites use load-balancers, so a single IP address has dozens of boxes behind it.

Re:50,000 web servers, not physical servers (1)

moon3 (1530265) | more than 5 years ago | (#27961497)

True, we have have about 3000-5000 webs running on single rack here. A high performance Opteron setup. Most people need only simple web that is hardly used, in real life most web pages are not accessed very often. If they happen to have some traffic (may be lucky 1% of them) they order dedicated servers anyway.

analogy (1)

ELCouz (1338259) | more than 5 years ago | (#27960841)

Are you trying to show who has the biggest dick in the IT world ? :3

Re:analogy (5, Funny)

Anonymous Coward | more than 5 years ago | (#27961003)

Are you trying to show who has the biggest dick in the IT world ? :3

No, didn't you read the summary? We're trying to find out who has the biggest rack.

Re:analogy (1)

Archangel Michael (180766) | more than 5 years ago | (#27961537)

We're trying to find out who has the biggest rack.

Cowboy Neal???

Re:analogy (1)

triso (67491) | more than 5 years ago | (#27971241)

We're trying to find out who has the biggest rack.

Bullwinkle!

Who else? ME. (1)

Narcocide (102829) | more than 5 years ago | (#27960883)

Thats right, me. In fact I just set up #50,001. You'll never guess what I'm doing with them either.

Re:Who else? ME. (2, Funny)

Mad Merlin (837387) | more than 5 years ago | (#27961161)

Thats right, me. In fact I just set up #50,001. You'll never guess what I'm doing with them either.

Do you really need that much porn?

Re:Who else? ME. (4, Funny)

linhares (1241614) | more than 5 years ago | (#27961255)

he's got an 8-screen setup

Re:Who else? ME. (4, Funny)

Joebert (946227) | more than 5 years ago | (#27962047)

There's an octo-mom joke in there somewhere.

Re:Who else? ME. (2, Funny)

tuxicle (996538) | more than 5 years ago | (#27964861)

...maybe if it said "eight-head" setup?

Re:Who else? ME. (1)

bjb (3050) | more than 5 years ago | (#27965091)

he's got an 8-screen setup

So does that make you feel like you're REALLY an evil overlord in a secret lair?

(reference to a dilbert cartoon about getting a second monitor)

Re:Who else? ME. (0)

Anonymous Coward | more than 5 years ago | (#27961571)

Thats right, me. In fact I just set up #50,001. You'll never guess what I'm doing with them either.

Praying they one day can reach out and hug you? Or that maybe they will impress some chick?

Keyword: Evil (2, Insightful)

zerosumgame (1429741) | more than 5 years ago | (#27960895)

50,000 -- that's peanuts compared to the likes of Google or Yahoo etc... Here's a short article on the data center that Google is building (has built??) in Oregon.. http://harpers.org/media/slideshow/annot/2008-03/index.html [harpers.org]

Re:Keyword: Evil (1)

gmuslera (3436) | more than 5 years ago | (#27961515)

Probably big server farms, like the ones of google, yahoo, amazon, etc are orders of magnitude over that. There are estimates of 200k servers in 2005, or 450k in 2006 (according to Wikipedia [wikipedia.org] ). By now could be the order of millons.

Re:Keyword: Evil (2, Insightful)

wwwillem (253720) | more than 5 years ago | (#27962161)

> By now could be the order of millons.

Forget about "could be". According to IDC (feb 2009):

"According to IDC's Worldwide Quarterly Server Tracker, [...] 2008 worldwide unit shipments grew 2.0% to 8.1 million units. [...]" [http://www.idc.com/getdoc.jsp?containerId=prUS21703309]

With 8 million units cranked out in just a single year, there must be 30+ million servers "up and running" in current datacenters.

Which is in sync with my experience with our customers. Ten years ago (which was the time of the dot-com :-) the number of servers decreased heavily caused by consolidation to big servers, etc., resulting in half empty DC rooms. The last few years, because of the pervasiveness of IT in our lives, but likely also because of the adoption of Windows as a server platform, the number of servers has sky-rocketed and everybody is now struggling with space, power and cooling problems in their datacenters.

Re:Keyword: Gross Overestimate (0)

Anonymous Coward | more than 5 years ago | (#27962001)

According to Google [blogspot.com] : in the time it takes to do a Google search, your own personal computer will use more energy than Google uses to answer your query. Another article [google.com] points out that Google datacenters are more efficient than anything anyone else has published for a colo/datacenter.

Typical scare articles forget to divide by the number of users. For example, 50MW used to serve 10M users is 5w/user. Some TVs use that when they are "off".

Yahoo? (2, Insightful)

neoform (551705) | more than 5 years ago | (#27960917)

Seems to me the second largest search engine likely has 50k servers or more..

Re:Yahoo? (1)

centuren (106470) | more than 5 years ago | (#27961521)

I've always been curious about how Blizzard's WoW servers are spread out around the world.

Re:Yahoo? (4, Informative)

eldorel (828471) | more than 5 years ago | (#27962189)

Ask, and ye shall receive...
US_realm_list_by_datacenter [wowwiki.com]

Re:Yahoo? (1)

centuren (106470) | more than 5 years ago | (#27963313)

Interesting! Still, I wonder about server configuration, load, and such. It's obviously a good model of dividing work between client and server, but there must still be some times that are quite taxing.

Re:IBM? (1)

NOLFXceptMe (1013903) | more than 5 years ago | (#27962243)

Hey, why isn't IBM on their list. I doubt why it would need so many servers though. Any ideas?

just look at the cooling equipment (0, Insightful)

Anonymous Coward | more than 5 years ago | (#27960923)

Can't say for sure but looking at cooling equipment from aerial views, you can get a good idea. A datacenter with nearly as real estate in cooling as floor has some serious stacks in place or planned.

Googles been building out datacenter space like money is free.They sell blank space on web pages! I'd peg them at the top of the pile. Given their weird, packed like sardines servers they let the world know about they may be even topping a million if they finish up the new stuff in Europe and worldwide.

Yahoo would have to come in next and probably pretty close to Google. Microsoft would have a total pile but since they can't even do SSL on their update sites they are running cheap and probably have less than 300k even with hotmail. Amazon I can't even guess on. But I would put them behind Microsoft. Ebay? on par with Microsoft.

Re:just look at the cooling equipment (2, Informative)

afidel (530433) | more than 5 years ago | (#27960977)

Uh, MS doesn't do HTTPS on their update site because they do PKI signing on the update package themselves so they don't need to.

Re:just look at the cooling equipment (4, Informative)

Achromatic1978 (916097) | more than 5 years ago | (#27961083)

MS has approximately 160,000. When I was there a year ago, I did a tour of one of their test datacenters... (even regular staff don't get to tour production), and they remarked that they'd recently turned on their 150,000th.

Re:just look at the cooling equipment (4, Informative)

CharredMetal (1463333) | more than 5 years ago | (#27961735)

Microsoft would have a total pile but since they can't even do SSL on their update sites they are running cheap and probably have less than 300k even with hotmail

The update protocol does winhttps (SSL). The actual file downloads are simple winhttp, since they are signed.

What about porn? (3, Insightful)

Mishotaki (957104) | more than 5 years ago | (#27960929)

I'm sure some big porn websites(those that regroup many websites together especially) have quite a lot of servers...

eBay (0)

Anonymous Coward | more than 5 years ago | (#27960939)

has 11k servers or so.

Re:eBay (0)

Anonymous Coward | more than 5 years ago | (#27961011)

has 11k servers or so.

but unlike some other places, they have some big iron too.

That many web server would be good for... (0)

Anonymous Coward | more than 5 years ago | (#27960949)

Porn.

umm.. qualify the result. (0)

Anonymous Coward | more than 5 years ago | (#27960957)

ok, so "managing more than 50,000 web servers"...

pretty sure:
IBM
EDS
CSC
Fujitsu

all trump that. Or did you mean "manage in a single pool" or some such?

From TFA (2, Informative)

actionbastard (1206160) | more than 5 years ago | (#27960963)

Netcraft has developed a technique for identifying the number of computers (rather than IP addresses) acting as web servers on the internet, and attributes these computers to hosting locations through reverse DNS lookups."

Apparently it's not just the number of webservers (just IP addresses), but the number of physical boxes these guys are running. If Netcraft's technique is valid, then it could be helpful in determining the 'true' penetration of FOSS based server installs on the Internet. This could severely impact the ranking of sites that are hosted on certain proprietary OSes.

Re:From TFA (1)

kmike (31752) | more than 5 years ago | (#27961657)

They also do not count non-HTTP servers:

The survey does not attempt to count back-end servers (application or database servers) or servers other than web (HTTP) servers.

One more thing: some hosting companies provide private network only servers, not visible outside of the virtual private network assigned to the customer. Perfect for the backend.
Softlayer does that, for example.

That depends. What is classified as a "server"? (0)

Anonymous Coward | more than 5 years ago | (#27960991)

I have a lot of old 486's and Pentium's buzzing along doing their intended jobs (web server,ftp,IRC) running NT 3.51/4/*BSD/Linux.

If you can count them... (1, Funny)

Anonymous Coward | more than 5 years ago | (#27961057)

Obviously, if you can count them, then you don't have all that many.

"A million here, a million there and pretty soon you have a real server farm."

Server Farms? (0, Offtopic)

tecnico.hitos (1490201) | more than 5 years ago | (#27961089)

I didn't know you could grow them. Have some spare seeds?

Surveying big farms (-1, Offtopic)

macraig (621737) | more than 5 years ago | (#27961135)

I'm frankly more curious about how many farms are operated by Archer Daniel Midlands and Cargill. I'd like to know where my food is coming from and how it's being managed. It's the farms serving my food that interest me more.

Financial Services Industry (5, Interesting)

Chris Snook (872473) | more than 5 years ago | (#27961247)

Most of them won't go into detail, but Wall Street firms have immense server farms. Some of them are limited in size by the amount of electricity the New York City power grid can supply them. They also have huge data centers in less prime real estate, but microseconds are dollars in the financial markets, so they try to keep as many of their systems as close to the action as possible. There are entire floors of NYC skyscrapers full of racks modeling the financial markets in real time, conducting transactions, and crunching numbers for human analysts.

Re:Financial Services Industry (1)

mudshark (19714) | more than 5 years ago | (#27963587)

There are entire floors of NYC skyscrapers full of racks modeling the financial markets in real time, conducting transactions, and crunching numbers for human analysts.

Using flawed algorithms and crap assumptions as input. See where that got us.

Re:Financial Services Industry (2, Insightful)

drinkypoo (153816) | more than 5 years ago | (#27964799)

Using flawed algorithms and crap assumptions as input. See where that got us.

These people don't have "maintaining a stable economy" as a goal. They have "making more money than the other guy". If the market drops more than they do, they consider themselves winners. They can trade into a position that will come up! Who cares what that fluctuation in the market means?

only 50k? (0)

Anonymous Coward | more than 5 years ago | (#27961357)

Coming from the Goog. I would say 50 is a fucking joke... Laughable really. Try 6 times that and climbing every single day.

2 CPU servers are a joke (1, Interesting)

Anonymous Coward | more than 5 years ago | (#27961693)

Companies that have "real" computers - 32-cpus and more and have 10,000 of those are more impressive to me. Having 100,000 servers, all the same, is impressive, but still will use about the same management as 10,000 servers would.

I've deployed (3) 48-way servers with 2 for HA Oracle DBs and the other for DR and testing. Oracle RAC was the best in class at the time - grid didn't exist. I've deployed hundreds of custom servers (diff OS with diff required patches) running specialized applications from many, many vendors. Getting an application from SAIC or Telcordia or Teradata to work inside your normal infrastructure is harder than it sounds. Even for huge customers, they barely bend without huge payments and there usually isn't any competitive alternative.

Some workloads aren't worth hunting/designing ways to split up. Get over it and buy the big servers.

MS runs 160,000 servers? If they converted to Linux, that could easily be reduced to 10,000. ;)

Re:2 CPU servers are a joke (2, Insightful)

lintux (125434) | more than 5 years ago | (#27961815)

And you seriously think that Google (or any other bigger company) is still using 2-CPU servers?

Re:2 CPU servers are a joke (0)

Anonymous Coward | more than 5 years ago | (#27962175)

Considering that 2-CPU motherboards are the only reasonably cheap multi-socket motherboards you can get, I'd say yes. Of course now every CPU has 2 or 4 cores, but still it's 2 CPUs.

dom

Re:2 CPU servers are a joke (1)

PhuFighter (1172899) | more than 5 years ago | (#27966505)

Well, according to this article, it looks like it's still 2-CPU servers: http://news.cnet.com/8301-1001_3-10209580-92.html [cnet.com] Unless, of course, it's an April Fools day joke :) I am sure they are 2 and probably 4 core CPU's though.

If you ask.. (4, Funny)

MobileTatsu-NJG (946591) | more than 5 years ago | (#27961367)

... Google how many servers it uses, does that mean it's self-aware?

Re:If you ask.. (1)

mgblst (80109) | more than 5 years ago | (#27962725)

I am sorry, I can't answer that Dave.

Re:If you ask.. (2, Funny)

drmofe (523606) | more than 5 years ago | (#27962759)

Wait until next week and then ask Wolfram Alpha.

Two hosting providers ago... (3, Funny)

Leomania (137289) | more than 5 years ago | (#27961399)

I'm pretty sure the hosting company I had a few years ago (aka "kiddie hosting") had that many customers on the server that I was on. Does that count?

Google Data Center Tech (5, Informative)

Anonymous Coward | more than 5 years ago | (#27961417)

Hello,
I have worked supporting Google's servers in one of my former employers data center. What I can tell you about there deployments is as follows:
1) 20,000 Servers in our data center; they occupied 8 other sites (~160,000 servers). Our site was one of the largest.
2) Over 30 GigE connections feeding into dual Juniper M20 later upgraded to Juniper T-320
3) Yes they run a custom version of RH

Now for the record; they had approx 160,000 servers in our companies data centers. I have met techs from other data centers which had similar counts. At a minimum I can confirm approx 160,000 and potentially 320,000 and up for other data centers; providing they mirrored their servers. It wouldn't make sense to put all your eggs (servers) in one basket. The time frame for these numbers was back in the early 2002.

Re:Google Data Center Tech (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#27962153)

The time frame for these numbers was back in the early 2002.

so do you think there's more now?

Re:Google Data Center Tech (0)

Anonymous Coward | more than 5 years ago | (#27963265)

> 1 million - easily.

Re:Google Data Center Tech (1, Informative)

Anonymous Coward | more than 5 years ago | (#27969809)

I am the originator of this thread. Yes, I know for a fact that they have expanded. The contract that was signed with Google was "Right of First Refusal".

The "Right of First Refusal" gave Google the power to be the first to occupy new space before any additional customers. At the time they had 1/3 of the data center. My old co-workers said that they had expanded again over the years.

All I know is that they their deployment is even larger. At this point I have no idea of their size. I know it's big; very big. To quote Winston Zeddmore:

That's a Big Twinkie...

Ok...Ghostbusters quote ... I couldn't help it. :-)

Near the NSA? (1)

AHuxley (892839) | more than 5 years ago | (#27961549)

Sean Gorman mapped out the US fiber-optic telco fiefdoms, and mb spotted less redundancy than was claimed.
Parts of his dissertation where "removed".
He showed the choke points and critical links.
This plot of the large server areas would be fun to map and then visit.
Spot the NSA tap points :)

Hey dawg (0, Offtopic)

Korbeau (913903) | more than 5 years ago | (#27961621)

I heard you like to farm servers, so I put a server farm in your farm so you can service your servers while serving your farm.

It's probably NSA or a federal lab (0)

Anonymous Coward | more than 5 years ago | (#27961731)

My single group at a large IC manufacturer had over 1000 compute servers for simulation. My guess is one of the federal labs(LBL, LLNL, Oak Ridge) has the largest.

Work it backwards (1)

drmofe (523606) | more than 5 years ago | (#27961837)

What are the current server specifications necessary to support 50K virtual instances of an O/S? How many boxes does that need to map down to in order to maintain sufficient redundancy and power efficiency? Now, how big a datacentre do you need to support that number of servers and hence what space/power/air conditioning requirements would you actually need?

Google's container-based datacentre is about 1300 servers. If each of those is capable of running multiple instances, that would bump up the total.

gn4a (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#27962239)

AS TO WHICH *BSD 8ot goi8g to play and executes a fate. Let's not be

OVH (1)

m0i (192134) | more than 5 years ago | (#27962613)

they had 40K at the last official count and their new datacenter has a 50K capacity and filling quick (+3 bays/day 7/7). Not surprising given they offer the cheapest dedicated one can find ($15/mnth no contract: Atom 1.6Ghz, 512MB ram/2GB flash for swap, 10GB iSCSI disk, unlimited bandwidth).

Here's your Backup (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#27962733)

http://web.archive.org/web/*/http://www.avsim.com

sort of...

1&1 (1)

hey (83763) | more than 5 years ago | (#27963105)

Never head of this company.

Re:1&1 (1)

rbrausse (1319883) | more than 5 years ago | (#27963921)

a huge hosting company from Germany, most customers are in Europe (but afaik they sell in the US, too).

you sub-6-digits old-timers should try to accept the i18n of /. :P

Re:1&1 (3, Informative)

rbrausse (1319883) | more than 5 years ago | (#27965053)

sorry for the second reply but according to the homepage ( http://www.1und1.com/ [1und1.com] [in German]) the company has over 7.2 million customers in Germany, UK, France, Spain, Austria and the USA running on more than 55k servers.

Re:1&1 (0)

Anonymous Coward | more than 5 years ago | (#27966503)

Never head of this company.

Funny, I've never heard of you!

I'm selling my intelligence... (1)

giuffsalvo (1536695) | more than 5 years ago | (#27963691)

Re:I'm selling my intelligence... (1)

PatrickThomson (712694) | more than 5 years ago | (#27963917)

Wow, that guy must think himself a lot smarter than he actually is, where did you find the link?

Physical, obviously (1)

frozentier (1542099) | more than 5 years ago | (#27963953)

Obviously they are talking physical number of servers, because why in the hell would they list number of virtual machines in a financial report? Who would care about that?

MS Cloud is growing very quickly (1)

Toreo asesino (951231) | more than 5 years ago | (#27964069)

While I can't speak for the others, I know Microsoft are growing their online infrastructure hugely now; at approx rate of "one facebook worth of servers every month" as one guy told me.

Most of it's consumer Live stuff, but there are plans to expand corporate services too so I understand.

50,000 is not that many (0)

Anonymous Coward | more than 5 years ago | (#27964133)

Just in the DC where I work, one of many of our buildings, we have 750-1000 racks of servers. Even if each rack had only 50 discrete systems in it, we'd be past 50,000 at just my location. I think we have several other DC's bigger than this one, so we easily have hundreds of thousands of servers.

Re:50,000 is not that many (1)

Cybershark302 (251658) | more than 5 years ago | (#27964575)

Your average server rack is only 42U in height...50 discrete systems is extremely high density. Even with blades you rarely get past about 64 per rack...Most datacenters have a mix of machine sizes so the average density quickly drops below 40. This also drops again once you put in all the switching and routing necessary to support these networks.

I work for a VAR in our integration facility and the highest densities we ever see are 64 BL460c HP blades in a 42U rack.

softlayer (0)

Anonymous Coward | more than 5 years ago | (#27964347)

And having seen all of softlayer's Dallas areas going to go with large.

Not Web Servers, but ... (2, Funny)

rlp (11898) | more than 5 years ago | (#27964391)

I imagine there is a fairly large server farm somewhere near Ft. Meade, MD.

Blizzard? (1)

maroberts (15852) | more than 5 years ago | (#27964873)

All those WoW and Diablo fanboys have to go somewhere....

new industry dedicated to this (1)

peter303 (12292) | more than 5 years ago | (#27966457)

There are several companies and industry conventions devoted to efficiently building massive server farms. I went to an oil industry convention session on these last year (The energy industry is 6th largest sector of supercomputing for seismic exploration, oilfield simulation and credit transactions.) Server farms consume vast amounts of power in the CPUs and air conditioning.

Modular expansion is fad. Server companies now recycle shipping containers as a row of racks with built in service, power distribution, and air conditioning. You have a certain rating of petabytes and kilowatts per shipping container. You start filling up a warehouse with these on a "as-needed" basis. Google is reputed to be at the top of this game.

"energy refineries" of the 21st century (1)

peter303 (12292) | more than 5 years ago | (#27967291)

Servers farms supply the commodity information for commerce and recreation in the early 21st century. Depending on what source you accept, they, along with client computers and video screens of all sizes, consume 3% to 10% of the US's energy.

I shamelessly stole this idea from Peter Huber of the Manhattan Institutes recent book The Bottomless Well. The trend of human commerce over the past couple centuries was to use more energy in more refined ways: wood, coals, whale-oil, petroleum, electricity, solar, computing, optical ...

I say early 21st century, because maybe some new discovery in computing technology or energy will drastically cut the energy consumption of yottaflops of computing before the century is over.

Google has an order of magnitude more (fact) (1)

whitroth (9367) | more than 5 years ago | (#27969795)

I had a phone interview with google around '05, I think, and was *told* by the interviewer that they had over half a million (physical) servers. I don't remember the exact number, but it was between 500k and 600k servers.

                  mark

GSI Commerce could have a bit. (1)

Aldenissin (976329) | more than 5 years ago | (#27970327)

I know the company I work for GSI Commerce has one of the largest networks in the US, behind (if I recall correctly) Google, Amazon, Wal-Mart and Ebay. Now networks are not servers, but they have well over 100 partners that they represent with many's websites hosted by GSI. But still probably not anywhere near 50,000.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?