Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Amazon Betas 'Elastic' Grid Computing Service

timothy posted about 8 years ago | from the bouncy-bouncy dept.

78

RebornData writes "I receieved an e-mail this morning inviting me to participate in a limited beta of Amazon EC2: the Amazon Elastic Compute Cloud. It's a grid computing service that allows you to create and upload your own Linux-based machine images and run them in Amazon's system, starting at $.10 per "instance hour" (each machine instance being equivalent to a 1.7GHz Xeon with 1.75GB of RAM, and 160GB disk). You can use their tools to create and start new instances dynamically to meet whatever your particular capacity needs are at any given moment. Fedora Core 3 and 4 are explicitly supported, but any distro based on the 2.6 kernel should work. The service documentation provides more technical details. Unfortunately, it appears that the beta is limited to existing Amazon S3 users, and is already full."

cancel ×

78 comments

Sorry! There are no comments related to the filter you selected.

Burstable Servers (5, Insightful)

peterdaly (123554) | about 8 years ago | (#15971060)

At $.10 per hour, that makes a single server instance about $72 per month. If you have minimal storage needs, it can compete with a low end leased server, plus it has other advantages not present in the physical leased box world.

Personally I don't have any need for a scalable system such as this, but it certainly opens the possibility for products or projects that may not otherwise be feasible.

Have a CPU intensive batch job that can broken up and distributed? Use these boxes during the run then eliminate them when it's done. Only pay for the time you use.

At a previous job I had a task that would have been perfect for a burst-able cloud like this. Example:

Every evening we had a large number of scanned tiff images that needed to be manipulated, and a short time window in which to do it. Tiff image manipulation takes a lot of CPU resources and time. We ended up purchasing a bunch of blade servers that sat idle for the 22 hours a day they we not running images. Something like what Amazon is offering may have been a very high performance and cost effective solution to that type of problem. The control via web services could automate the whole process.

Re:Burstable Servers (0)

Anonymous Coward | about 8 years ago | (#15971138)

But what about network traffic costs, eh?

Re:Burstable Servers (1)

cmeans (81143) | about 8 years ago | (#15971282)

I put idle machines to work on SETI@Home [berkeley.edu] (with permission of course).

Re:Burstable Servers (0)

Anonymous Coward | about 8 years ago | (#15971419)

WTH? Somebody mod this offtopic.

Re:Burstable Servers (2, Interesting)

iluvcapra (782887) | about 8 years ago | (#15971283)

It seems like the only serious problem would be getting the 20GB TIFF (or 8 GBs of WAV files) over to the server instance in the first place.

Having to move all your data over to the server and back adds significant set-up overhead, particularly if you only need the monster for 2 hours at a time. When you need the numbers crunched on demand, you don't want to have to wait 6 hours while the data set squeezes its way to the bay area and back over routers.

I'm sure that there are applications for this, but quick-turnaround stuff is hamstrung by the I/O bottleneck known as the Internet.

Use Amazon S3 (2, Informative)

RebornData (25811) | about 8 years ago | (#15971512)

This service is paired with the Amazon S3 storage service, which has a high-bandwidth connection to the servers. Data transfer between EC2 and S3 is free.

-R

Re:Burstable Servers (1)

Hercynium (237328) | about 8 years ago | (#15973444)

There's some interest in doing this same sort of thing at my current employer. Of course, we're a telecom and we'd be providing not just the Utility Computing service but the bandwidth to the customer as well...

I can see this as working *very* well, for companies that can deliver both with reasonable cost.

BTW - I want a datacenter full of these [egenera.com] so bad I can taste it!

You're the grid computing poster child (1)

patio11 (857072) | about 8 years ago | (#15971343)

I used to work on grid computing at a Japanese technology incubator. The project eventually got shelved because there were just not enough applications which honestly benefit from it when you consider the pain involved in programming the application to distribute over the grid (I gather our American competitors are having the same problem), but the idea was taking your company's totally-idle-at-night fleet of desktops and turning them into a virtual Beowulf cluster. I think our biggest success, if you can call it that, was turning about 1,000 computers at 15 sites (hospitals, college labs, and the like) into what for two hours was the largest supercomputer in the prefecture by a factor of 10. (Gifu Prefecture... this is sort of like bragging that you have the biggest computer in Iowa, but hey, you take your wins where you can.)

The downside, aside from the 100% solvable issue of being able to wake up 1,000 computers in the middle of the night without needing a staff of 80 (which was what it took for our "successful" run), is that programming for a grid is hard and tedious, and none of the frameworks that I am familiar with really take it down to the level where it needs to be for "regular" programmers to be able to do it.

In short: a lovely technology, some practical uses, not exactly going to set the world on fire though.

Re:You're the grid computing poster child (4, Interesting)

ispeters (621097) | about 8 years ago | (#15972808)

The downside, aside from the 100% solvable issue of being able to wake up 1,000 computers in the middle of the night without needing a staff of 80 (which was what it took for our "successful" run), is that programming for a grid is hard and tedious, and none of the frameworks that I am familiar with really take it down to the level where it needs to be for "regular" programmers to be able to do it.

You might want to check out Starfish [rufy.com] . It's Google's MapReduce implemented in Ruby, kind of. It makes distributed grid computing possible in six lines of code. Unbelievable, but true.

Ian

PS I've personally got nothing to do with Starfish. I read the author's blog--that's it.

Re:You're the grid computing poster child (0)

Anonymous Coward | about 8 years ago | (#15973643)

You can also look at the Digipede Network which provides a Framework SDK [digipede.net] that uses OOP methodologies. The distributed work is simply defined in a class and the class is distributed on the grid by the Digipede Network. This allows developers to focus on their areas of expertise instead of the grid computing plumbing. It takes about 20 lines of code total to define and create the connection component, the job and tasks, and to convert the distributed class. The Digipede Network is a Windows only grid computing solution which avoids the need to setup a Beowulf cluster to run at night on the Windows desktops. This approach also supports the idea of cycle scavenging during the day.

Re:You're the grid computing poster child (1)

qwix (962550) | about 8 years ago | (#15975179)

Google's Map/Reduce is not an answer to everything (in the same way map() and reduce() do not make a generic programming system/language by themselves). Some problems fall into this category, but things can get trickier if you want to make it generic. Saying that this "makes distributed grid computing possible" is an overstatement.

Re:You're the grid computing poster child (1)

ispeters (621097) | about 8 years ago | (#15977136)

Perhaps I should have used an <em> tag because this is what I meant:

[Starfish] makes distributed grid computing possible in six lines of code.

(Aside: why don't actual <em> tags do anything inside <blockquote> tags?)

Anyway, my point is that the ease with which you can distribute some programs across a grid with Starfish is amazing, at least to me. Also, my naive understanding is that the best problems for solving with a grid are so called "embarassingly parallel [wikipedia.org] " problems and that such problems generally can be expressed in terms of Map and Reduce. Of course, either or both of those two assertions could be false.

Ian

With X11, any process is griddable (0, Troll)

Colin Smith (2679) | about 8 years ago | (#15971426)

Network storage, a few wrapper scripts and you're away. Kick off pretty much any GUI app on "the grid" and have it plonk it's interface back on your display. With something like GridEngine you can designate some of the machines as OpenOffice servers, others as Thunderbird servers to take better advantage of shared libraries, filesystem buffers and CPU caches, or you can just have everything kick off on the least loaded machines. The result is that the user's GUI apps could be running on a dozen different machines completely transparently.

With Windows of course, your extra machines might as well be doorstops.

 

Re:Burstable Servers (0)

Anonymous Coward | about 8 years ago | (#15976557)

$10 / hour -> 240 per day, where did you pull the $72 figure from ???

I can already see it... (2, Interesting)

Lead Butthead (321013) | about 8 years ago | (#15971071)

Compromised Amazon grid being used as a botnet to send spam. Letting people upload their own OS image is really asking for it.

Re:I can already see it... (1)

bcoff12 (584459) | about 8 years ago | (#15971145)

I'm sure monitoring internet traffic, if it is even allowed, is something they are already doing.

Re:I can already see it... (2, Informative)

funfail (970288) | about 8 years ago | (#15973404)

They sell the bandwidth separately and it's not cheap.

Say... (3, Funny)

Doctor Memory (6336) | about 8 years ago | (#15971082)

Amazon EC2: the Amazon Elastic Compute Cloud.
This isn't one of those wearable computers, is it? If so, I hope they're being very choosy about who they "let into the beta"... <shudder/>

Service Documentation? (3, Informative)

neonprimetime (528653) | about 8 years ago | (#15971090)

Ok, so what is just me, or do the first to links on the post point to the exact same spot?
Maybe they meant the Technical Documentation [amazonwebservices.com] ?

Re:Service Documentation? (1)

Bog Standard (743863) | about 8 years ago | (#15977155)

Can anybody provide any pointers to information regarding the underlying infrastructure? What it is based on etc. (I presume it is not VMware). Churz Bogster

I use Gentoo... (4, Funny)

Anonymous Coward | about 8 years ago | (#15971095)

Could I buy some server time to get my initial compile to under a week?

Re:I use Gentoo... (1)

Surt (22457) | about 8 years ago | (#15973136)

Could I buy some server time to get my initial compile to under a week?

I'm sorry, but their grid isn't that big.

UGH (0)

Anonymous Coward | about 8 years ago | (#15974270)

I just made a major boo-boo while migrating my root partition to RAID-5 [gentoo-wiki.com] . Fortunately all my data (home dirs and multi-thousand photo archive) was on another drive (unplugged), so I just have to rebuild the system, not re-scan bajillions of negatives and slides.

The end of the installation section of the Gentoo Handbook [gentoo.org] says:

Congratulations! You now have a working Gentoo system.

Sorry, but my definition of a working system includes the Apps, modules and libraries that I need to do the job: samba, X, NVIDIA-glx, KDE, KMail, Firefox, lcms, Cinepaint, Scribus, Imagemagick, PerlMagick, libusb, VueScan (not in portage), etc in the case of this machine. This Dual Athlon 2600 with 1GB RAM and 4 RAID5'd 7200RPM ATA-133 drives has been compiling over 12 hours, and has a lot more to go.

Incedentaly, that migration guide appears to work, just make sure
  1. Your kernel includes support for ALL your hard-drive controllers, not as modules (Sil680 is modular by genkernel's default)
  2. Device mapper support is enabled (in LVM), it's off by genkernel's default
  3. Your RAID levels are compiled in (they're modular by default)
  4. Once the RAID is up and the old drive is added, DO NOT REBOOT, POWER-OFF, OR POWER CYCLE your system until the RAID is finished rebuilding and the discs are synced. This is where I goofed up. My array went poof & when I went to recover it the filesystem was really mangled. Now I'm posting from an iBook I've just learned may burst into flames at any second while my desktop re-installs everything.

What would these be used for? (1)

Fyre2012 (762907) | about 8 years ago | (#15971096)

Sounds neet, but what practical applications could be made from this service?
I'm not trolling, rather genuinely interested in what /. thinks this may be used for...

Smart Idea (1)

cmdrTacyo (899875) | about 8 years ago | (#15971110)

Building an iron grid while doing a bid
just like fiddy did
I spit tight cuz I'm white
Slashdots delight

Elastic Grid computing? (4, Interesting)

dontbflat (994444) | about 8 years ago | (#15971122)

what? So they are building computers for you and letting you use them for whatever you desire? Hmm. 0.10/hr thats $2.40/day. Thats $876/year. Not a bad deal. Heck its even cheaper than some website services that give you a dedicated server for $200/month.

Re:Elastic Grid computing? (1)

MindStalker (22827) | about 8 years ago | (#15971427)

They are hardly building computers. They are using massive servers that are then virtualized to run thousands of desktop sized computers. I havn't RTFA but can anyone tell me if you can purchase a virtual image of greater stats than the 1.7GHz Xeon .. Or is your only option to purchase several and cluster them?

MPI? (1)

senor_meow (990740) | about 8 years ago | (#15971134)

This sounds better than Sun's grid computing solutions in that there is a lot more control of the machine (at long as its a 2.6 kernel) at a better price. I could use any distributed compiler package/library such as MPI?

Who uses this stuff? (3, Insightful)

fm6 (162816) | about 8 years ago | (#15971136)

This is the umpteenth grid service where anybody can buy huge gobs of computer time. The question is, is there really anybody out there who needs to do this and doesn't have their own hardware? Sun's grid effort has pretty much laid an egg. Perhaps I have the economics wrong, but isn't it more cost effective to build your own cluster out of discarded PCs?

Re:Who uses this stuff? (4, Insightful)

OverlordQ (264228) | about 8 years ago | (#15971196)

Perhaps I have the economics wrong, but isn't it more cost effective to build your own cluster out of discarded PCs?

No, what if you need to do a one-off job. Which is cheaper $.10/hour or paying somebody full time, buying supplies, paying for labor to put it together, paying for power to run it, and then letting it sit there gathering dust.

There's no way you can get parts for the systems and labor for an admin to ~$72/month/server

Re:Who uses this stuff? (1)

fm6 (162816) | about 8 years ago | (#15971355)

But who does one-off jobs? If you have the expertise to program this kind of massive computing project, you're going to employ that expertise continually.

Re:Who uses this stuff? (2, Interesting)

Doctor Memory (6336) | about 8 years ago | (#15971428)

I wonder if there will eventually be a dependent subculture built around grids. Companies who sell pre-written (and customizable) solutions that require a grid to run. Companies might buy/license them and run them quarterly, to (say) update/analyze their data warehouse. Or if somebody develops a "killer app" for grids, then they might set up their own and rent it as a distributed computing solution. Maybe develop some kind of adapter for Mathematica, let people run their huge econ simulations, then truck their results away to slice & dice.

Re:Who uses this stuff? (1)

chris_mahan (256577) | about 8 years ago | (#15971432)

Agreed, and most people who need to do these one-off jobs do them often enough that they have spare hardware to run them on.

Re:Who uses this stuff? (2, Interesting)

joto (134244) | about 8 years ago | (#15971231)

Perhaps I have the economics wrong, but isn't it more cost effective to build your own cluster out of discarded PCs?

You've got the economics wrong. Building your own cluster out of discarded PCs is not economic. Building your own cluster out of brand new PCs might be.

Still, leasing is attractive for many reasons. Such as predictable costs, complying with yearly budgets, etc... If you build your own cluster, and find it doesn't work as expected, or you didn't really need it, or whatever, you are pretty much fucked. Clusterfucked!

Who uses this stuff?-IRTC (0)

Anonymous Coward | about 8 years ago | (#15971348)

"The question is, is there really anybody out there who needs to do this and doesn't have their own hardware? "

http://www.irtc.org/ [irtc.org]

Next question?

Re:Who uses this stuff? (1)

ChrisA90278 (905188) | about 8 years ago | (#15972925)

Perhaps I have the economics wrong, but isn't it more cost effective to build your own cluster out of discarded PCs?

How long would it take you to round up a few PC and build a cluster and get it connected to a high badwidth Internet connection? How much do you want to be paid per hour? What are the payrol taxes on your hourly rate. Add this all up. Assume the old PCs are free. OK now you have the buy-in price. How many hours a month will yo need to keep this system running? At what hourly rate?

Now you have to figure the cost of power is about $0.15 per KWH and each PC will burn about 0.2 KW and every KW the PC burns required at least 1 KW of airconditionaling power. The power osts alone are $0.06 per hour for only four cents more I get a PC and don't have to pay you to build it for me.

You can also... (2, Interesting)

ackthpt (218170) | about 8 years ago | (#15971140)

You can also go fart around with Amazon's Web Services [amazon.com] for fun and profit.

They rolled this out a few months back, when I was one of the brave few to sit through the presentation at a programmers conference in Santa Clara (for a free t-shirt and pen.) It was actually amazingly cool and I'm planning something of my own with it. (but I ain't telling you because I wouldn't want anyone tempted to swipe my neat idea, like thinkgeek did to me once already.)

I am U-BUN-TU -- Fear me Not (-1)

Anonymous Coward | about 8 years ago | (#15971159)

I am U-BUN-TU -- Fear me Not, for I am your Saviour. You are blessed for using UBUNTU. You are damned to eternal damnation for not !!

Great Pricing (3, Insightful)

kognate (322256) | about 8 years ago | (#15971195)

I like their pricing a great deal. It's much, much, much cheaper than many of the alternatives (notable the Sun one) AND you do not have to build your apps to use some proprietary IPC that's no good outside of their cluster.

For example, lets say I had a MPICH (or even a custom) application that I wanted to run. I'm just some joe schoe, so I
can't use the cluster in my (academic) department. I can run my application for one hour using 1000 "computers" for about $100 USD.
That's pretty good. It would cost me $1000 to use the Sun N1 stuff AND I would have to use the N1 grid-engine to develop my app.

Can't wait to see what comes out of the Beta. People give Amazon a bad rap because they're not Google, but make no mistake: they are innovators too.

Re:Great Pricing (1)

crankshot999 (975406) | about 8 years ago | (#15973953)

joe schoe you stole my idedtity [http://home.flash.net/~bob001/identity-theft.htm]

The great sea of resources. (3, Informative)

w33t (978574) | about 8 years ago | (#15971213)

The concept of virtualization is so seductive.

In our server room we have recently begun virtualizing servers and as a result have begun to think not in terms of physical servers and hard disks anymore, but in terms of resource pools of storage and processing.

It's like we have been able to smelt our physical machines and from the molten resources forge anew.

The recoverability and fault-tolerance is amazing as well - if a physical box dies there is basically no interruption in service. If something goes awry with an image we can just pull it and restore from yesterday.

Seeing Amazon offering what seems to be more of an ocean of resource than a pool is very tantilizing.

I'm certainly not the first, but I wonder if indeed local operating systems and cpus will become something of an anacronism, and that most processing will someday occur via the internet: that it will become the world-wide-mainframe.

Obligatory Link? (1, Funny)

Anonymous Coward | about 8 years ago | (#15971251)

Where's the obligatory link to where Barnes & Noble has it for cheaper?

Grid? (0, Troll)

lbmouse (473316) | about 8 years ago | (#15971279)

Anyone else getting tired off all this Web 2.0 (tm) terminology?... "Compute cloud", "Grid computing service", etc. Every time I see this, I think Gay-Related Immune Deficiency computing service. Guess I'm showing my age [brynmawr.edu] . Anyone else remember the diet candy called Ayds?... wow, bet the marketing guys at that company had coronaries after the CDC press release. It's not good when a government agency names a fatal syndrome after your product.

Re:Grid? (1)

Doctor Memory (6336) | about 8 years ago | (#15971492)

Anyone else remember the diet candy called Ayds?
Heh, I was in the meeting the day my boss suggested we change the name of the about-to-be-launched Agricultural Info-Data Service, due to the "unfortunate connotations" the acronym might suggest...

Re:Grid? (1)

cheezit (133765) | about 8 years ago | (#15973825)

"Account Authority Digital Signature"---one letter off, but close enough for me.

Re:Grid? (1)

SameBrian (945591) | about 8 years ago | (#15974047)

Maybe you're just too worried that you're gay and that you've picked up an immune deficiency. Take two Tylenols, go home, watch some Spike, and when you wake up, maybe you'll remember that you're a bigoted a**hole who needs to get some computing done for $.10 an hour.

Re:Grid? (1)

lbmouse (473316) | about 8 years ago | (#15982885)

maybe you'll remember that you're a bigoted a**hole who needs to get some computing done for $.10 an hour.

I remember 3 of my friends in the 80's coming down with GRID before the first heterosexuals were diagnosed and the CDC decided that it was not a 'gay' disease. They all died. That is why the acronym "GRID" is burned into my conscience. Please don't call anyone an a**hole before you know them, that is acting stupid.

Re:Grid? (1)

SameBrian (945591) | about 8 years ago | (#15983222)

Apologies. Didn't know the GRID reference. I thought you were pulling something out of your a**.

Re:Grid? (1)

lbmouse (473316) | about 8 years ago | (#15982993)

I'm sorry for being kurt, but your post hit me in a hard spot (I'm not a biggot [and I hate Spike]). I try to make light of past injuries. I'm assuming that you are young, so here is an FYI about was I was joking about [wikipedia.org] .

Cost sheet (3, Informative)

DAldredge (2353) | about 8 years ago | (#15971294)

Pricing

        * Pay only for what you use.
        * $0.10 per instance-hour consumed (or part of an hour consumed).
        * $0.20 per GB of data transferred outside of Amazon (i.e., Internet traffic).
        * $0.15 per GB-Month of Amazon S3 storage used for your images (charged by Amazon S3).

Data transferred within the Amazon EC2 environment, or between Amazon EC2 and Amazon S3, is free of charge (i.e., $0.00 per GB).

Amazon S3 usage is billed separately from Amazon EC2; charges for each service will be billed at the end of the month.

(Amazon EC2 is sold by Amazon Web Services LLC.)

Re:Cost sheet (0)

Anonymous Coward | about 8 years ago | (#15975318)

Boy I wish I could make up things and post them as 'facts'.

What a klunky name (1)

krell (896769) | about 8 years ago | (#15971295)

Either that, or "Amazon Elastic Compute Cloud" is the name of a new nanobot swarm threat that came out of Brazil.

Virtual? Real? Or does it matter? (1)

vrmlguy (120854) | about 8 years ago | (#15971296)

I'm surprised that Amazon isn't using any of the existing methods of creating virtual machine images, especially since it seems likely that they are using some sort of VM technology to do all of this. VMs are the easiest way to provide identical hardware to all of the instances; VMs also provide the tools to efficently load and unload images on demand. I wonder if we'll see different pricing depending on how CPU intensive an instance is. I can see using an instance as a server for less-popular online games; you'd only pay for those times when someone is playing. Depending on how fast multiple instances could be ramped up, you could also provide slashdot-effect insurance; just bring more instances on-line as more people arrived and shut them down as the peak passes.

Re:Virtual? Real? Or does it matter? (0)

Anonymous Coward | about 8 years ago | (#15974450)

I believe they are piggy-backing Xen, with their tools being wrappers.

What a name! (1)

sammy baby (14909) | about 8 years ago | (#15971321)

Whoa!

"Amazon EC2: the Amazon Elastic Compute Cloud."

Say it with me: Amazon Elastic Compute Cloud.

For God's sake, it's like a Tom Wolfe [wikipedia.org] book!

gah? (0, Offtopic)

tomstdenis (446163) | about 8 years ago | (#15971352)

1.7Ghz Xeon?

tom@bigbox ~ $ cat /proc/cpuinfo  | grep "model name"
model name      : AMD Opteron(tm) Processor 885
model name      : AMD Opteron(tm) Processor 885
model name      : AMD Opteron(tm) Processor 885
model name      : AMD Opteron(tm) Processor 885

1.75GB of ram?
tom@bigbox ~ $ free
             total       used       free     shared    buffers     cached
Mem:       6104840     700028    5404812          0      62092     399356

No thank you.  :-)

Tom

Didnt SUN try this? (1)

mnmn (145599) | about 8 years ago | (#15971454)

I remember when Sun's grid was out. I tried for days to get pricing, and eventually I was quoted something like $1000 per hour, except I couldnt get one hour.

I was trying out different parameter combinations in the uCLinux kernel to check compile sizes and functionality, and wanted horsepower to compile every iteration of the kernel. But The Sun deal was a joke, I could build a few Athlon beige boxes and do it cheaper.

This deal sounds good enough for me to take it out on a spin of a day or two, but I really need to check the bandwidth (both the speed and allowance). Despite this, for bigger tasks, I think its still more feasible to just build beige machines using Core Duo CPUs, some ram and disk and just use those.

When I was at college, I had an idea of putting up a bootp server in the library, getting all the machines to boot the linux image at night, and just using those machines for fun. All 100+ of them. I just didnt have the need for so much horsepower.

Time for Multics (1)

VincenzoRomano (881055) | about 8 years ago | (#15971456)

It seems it is the right moment to revive the Multics [wikipedia.org] project.

Hmm... (1)

creimer (824291) | about 8 years ago | (#15971499)

Maybe that would explain why the last few times I been to the Amazon website that it was unavailable. The grid is sucking up all the available CPU cycles. So much for customer service.

Re:Burstable Servers (2, Insightful)

e4g4 (533831) | about 8 years ago | (#15972743)

Saving sick people, or at least contributing to the solution seems like a legitimate and justifiable use of our (admittedly unrenewable) resources. At least, far more than driving a Ford Expedition to pick up milk and bread...

....What was the article about again? :-P

Re:gah? (1)

tomstdenis (446163) | about 8 years ago | (#15972786)

At the point where you legitmately need 800 processors ... you're probably big enough to borrow the money to build it. Most small million dollar firms I know of run off of simple dual core AMD64s. At the point where you need a 50 million dollar cluster ... you're probably big enough to buy one.

It sounds all nice in practice to buy "cheap time" but it's cheap for a reason

1. You don't own the boxes
2. It's not leased, they could go down at any moment
3. All your data is remote
4. You're subject to your network and their network and all the networks in between
5. You don't own the boxes so you can't add hardware, change key software components, etc...

There are a lot of downsides to it and running a business from it is stupid.

Tom

Re:gah? (1)

Silver Gryphon (928672) | about 8 years ago | (#15975727)

Perhaps, but a one-off or two-off project that requires a million hours of CPU time in 6 months can be outsourced to someone who has the equipment easier and cheaper than setting up your own hardware. Setting up enough servers or even desktop-class workstations could take a big chunk of those 6 months. And it'll probably cost more than the Amazon service, in hardware and manpower.

I say this from experience - my employer specializes in an industry-specific data validation process, and has spent 20 years writing and perfecting a few million lines of code that are repeated over and over within this process. One client came to us and wanted to run this process against an unprecedented amount of data, and it came out to 2000 CPU-years of processing time, with a 1 year deadline. The physical space, network and power requirements for 500 quad-processor servers would overwhelm our facilities, not to mention our network staff, so I'd at least consider outsourcing the hardware at this scale. Alas, we're decidedly a Windows shop and this isn't an option for us yet... Mono may change that.

Most businesses would not "run from" this cluster on a daily basis, but use it for a specific task-oriented purpose like research, math, graphical rendering, etc. Something they do once or twice, or for that client that wants, "What you do best, just 10,000 times bigger."

Re:sysdamin costs (1)

42forty-two42 (532340) | about 8 years ago | (#15972810)

Keep in mind that keeping around old images isn't really much of a burden - 10G on S3 is $1.50/mo and $2.00 to upload (unless you make it from a running EC2 instance!), and as it's compressed, the prices may be even lower. As for network security, what makes you think amazon hasn't properly firewalled these VMs to keep them from forging their IPs/MACs, and to prevent sniffing? They're already configured with a default-deny firewall that the user needs to lift with an API call.

Re:sysdamin costs (1)

steve_l (109732) | about 8 years ago | (#15978068)

I havent looked at the networking details; configuring virtual lans for isolation is possible, albeit tricky.

One thing about image size is it isnt mandatory to have guis on these things. I would go for something like DSL for a lightweight system image, they produce very small binaries. That only leaves the data. S3 stores bulk data nicely, but you also need a back end RDBMS, or you host mysql in a cluster all of your own.

I suspect that the next offering of amazon will be pay-to-use RDBMS with an availability guarantee; MySQL-7x24 or something, so you can use your own front end images against the back end.

I also think this service shows us how they are moving to host their own systems; fully virtualized, on-demand cloning. nice,

Re:Fantastic for indie MMOG hosting (1)

edmudama (155475) | about 8 years ago | (#15973062)

A very good idea IMO. Say your game burns 30KB/s of bandwidth per player... it'll take (by my weak math) about 9 hours to use 1GB of bandwidth, for which you paid $0.20... on average, that's gotta be at most an average daily usage. So effectively, your cost per user is in the $0.20/day range, or about $6/month for bandwidth.

Wonder what sort of latency these clusters have, if they'd be suitable to host a BF2 server or something.

The Death of Personal Storage, right here... (0)

Anonymous Coward | about 8 years ago | (#15973156)

Ah yes, I'v ebeen perdicting this for 5 years now, and I see that it's finally "coming of age".

The beginning of the end of Personal Storage.

All your data belongs to us.

Stock up on hard drives while you can.

Privacy is dead, long live privacy.

Re:gah? (1)

jb.hl.com (782137) | about 8 years ago | (#15973172)

God there's more penis waving going on in this thread than in a gay porno.

Re:Gates' Taxes (1)

Anomalyst (742352) | about 8 years ago | (#15973488)

No grid needed its $0.00, all thise bribe^Wcampaign contributions to his pet congress critters are write-offs.

Re:Burstable Servers (1)

cheater512 (783349) | about 8 years ago | (#15973594)

I'm sure you turning your computer off at night will prevent global warming.

It would be more productive going after the Oil companies instead of making people turn their computer off.

Re:Burstable Servers (1)

gbjbaanb (229885) | about 8 years ago | (#15977065)

I seem to have touched a nerve with some people, and yet I was just trying to be funny :)

(serious point though: with the modern CPUs that have frequency throttling power management, running 'spare cycle' apps do waste power whereas once they wouldn't make much difference)

Wow! (1)

mikehunt (225807) | about 8 years ago | (#15973987)

This is cool. The ultimate silent machine.

And think of the possibilities...run a virtual
machine inside your Amazon VM, totally encrypted
with ssh tunnels in and out...

Sorry, I appear to be drooling!

Re:The great sea of resources. (1)

w33t (978574) | about 8 years ago | (#15974193)

Wow, thanks! :)

Sounds like something from the 5th element? That's one of the best compliments I've ever gotten! ¦D

Re:Render farm (1)

shess (31691) | about 8 years ago | (#15974316)

This service could be extremely useful as a render farm for 3D graphics. It would be wonderful to be able to call up 1000 CPUs for a couple days or weeks at a time, without paying for them when not in use.

The problem is that movies output at something like 10M per frame (4000 pixels wide by 2000 pixels high by 48 bits per pixel div 8 bits per byte div 4 lossless compression factor), so you'd be talking about nearly a gigabyte for 4 seconds of footage (and my compression factor may be way off). So you can spin up 1000 CPUs for a half hour to generate five minutes of footage, and wait for weeks for your data to download over your T4 link. You could decide you just want to use this for proofs and cut the resolution in half on each dimension, cut to 24 bits per pixel, allow lossy compression, which cuts down the space by a factor of 16, but ... now you only need 50 or 60 CPUs, so it starts to look more reasonable to just have them on-site.

Yeah, I've wondered about this exact issue in the past. I work at a place which specializes in building amazing giant clusters, and I have friends at Pixar and Dreamworks. But this one is going to be hard to carry. Movie houses keep their render farms hot for 6 months at a time, or if they're like Pixar, pretty much continuously. When ILM moved from Santa Rosa to Presidio, they basically installed a 10gbit fiber link (*) between to help with the transition period (when the render farm was in a different location from the workstations). Amazon isn't going to install a 10gbit fiber link for you :-).

-scott

(*) http://www.computerworld.com/hardwaretopics/storag e/story/0,10801,105052,00.html [computerworld.com]

You're not kidding? Well, I am. (1)

alienmole (15522) | about 8 years ago | (#15974766)

Um, you do realize you have to pay for the CPU time you use, right? I estimate that every cent of RSA factoring prize money will cost you about $50 spent on Amazon Compute Cloud services. But with the capacity of EC2, I bet you could make up for it in volume!

Re:Use Amazon S3 (2, Informative)

cduffy (652) | about 8 years ago | (#15974810)

Upload it as it's generated, so you aren't waiting until just before you run your batch to do the transfer all at once.

Re:What would these be used for? (1)

tomhudson (43916) | about 8 years ago | (#15975466)

Why? Just your data transfer charges and disk storage space (those are billed extra by Amazon) are going to cost more than the electricity to run your box overnight (and you already have the box, so its not like you're spending extra for it). Plus it takes time to upload all that video data - time which your machine is running anyway, so you might as well just use it to encode it yourself.

cpu cycles have become so commoditized that there's no point in going to the hassle of "leasing" a small virtual cluster. Most people have several boxes already - cluster them and be done with it.

Re:Burstable Servers (0)

Anonymous Coward | about 8 years ago | (#15975599)

Maybe someone should create a Folding@Home-style project to work on a cure for arrogant, pig-ignorant hippies.

Why don't you turn off YOUR computer?

Re:Burstable Servers (0)

Anonymous Coward | about 8 years ago | (#15976482)

Yeah, turn off your computer and do your work on paper like normal people, and we can save our planet! ............... >.>
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>