Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Collaborative Map-Reduce In the Browser

kdawson posted more than 5 years ago | from the suercomputer-on-the-very-cheap dept.

Supercomputing 188

igrigorik writes "The generality and simplicity of Google's Map-Reduce is what makes it such a powerful tool. However, what if instead of using proprietary protocols we could crowd-source the CPU power of millions of users online every day? Javascript is the most widely deployed language — every browser can run it — and we could use it to push the job to the client. Then, all we would need is a browser and an HTTP server to power our self-assembling supercomputer (proof of concept + code). Imagine if all it took to join a compute job was to open a URL."

cancel ×

188 comments

Sorry! There are no comments related to the filter you selected.

Random Thoughts (5, Interesting)

AKAImBatman (238306) | more than 5 years ago | (#27056153)

Two comments:

1. He places the map/emit/reduce functions in the page itself. This is unnecessary. Since Javascript can easily be passed around in text form, the packet that initializes the job can pass a map/emit/reduce function to run. e.g.:

var myfunc = eval("(function() { /*do stuff*/ })");

In fact, the entire architecture would work more smoothly using AJAX with either JSON or XML rather than passing the data around as HTML content. As a bonus, new types of jobs can be injected into the compute cluster at any time.

2. Both Gears and HTML5 have background threads for this sort of thing. Since abusing the primary thread tends to lock the browser, it's much better to make use of one of these facilities whenever possible. Especially since multithreading appears to be well supported by the next batch of browser releases [owensperformance.com] .

(As an aside, I realize this is just a proof of concept. I'm merely adding my 2 cents worth on a realistic implementation. ;-))

Re:Random Thoughts (-1, Redundant)

IamGarageGuy 2 (687655) | more than 5 years ago | (#27056235)

The idea of the java hijacking the threads and using all of the cpu power is what is the main problem here. Who wants to open up a page that suddenly uses all of their cycles and makes their computer useless for anything else while this is running. I think that I would be willing to donate some of my cycles but Java has a bad habit of locking up the browser till the process is completed.

Re:Random Thoughts (1, Informative)

AKAImBatman (238306) | more than 5 years ago | (#27056283)

What does Java have to do with anything?

I think that I would be willing to donate some of my cycles but Java has a bad habit of locking up the browser till the process is completed.

Maybe you should try reading the post you're responding to?

Re:Random Thoughts (0, Flamebait)

cripkd (709136) | more than 5 years ago | (#27056853)

What? JavaScript is NOT java ? Next you'll tell me that there is no connection too!

Re:Random Thoughts (1)

Man On Pink Corner (1089867) | more than 5 years ago | (#27056285)

Translation: Who wants a patent suit from Digg.com?

Re:Random Thoughts (1)

corsec67 (627446) | more than 5 years ago | (#27056473)

I think that I would be willing to donate some of my cycles but Java has a bad habit of locking up the browser till the process is completed.

And then Firefox (or Epiphany) pops up a dialog after JS maxes out the CPU for 30 seconds asking if you want to permit the execution to continue.

So you have to limit yourself to 20 seconds per page load, and have the overhead of using Javascript as opposed to a better language like Java. Actually, a Java applet seems like what you would really want to do if you were really going to do this.

This idea is an interesting "hijack visitors to do work", but has some major issues.

Re:Random Thoughts (0)

Anonymous Coward | more than 5 years ago | (#27056643)

Actually, a Java applet seems like what you would really want to do if you were really going to do this.

Or both of you could try reading the post that started this thread. You know, multithreaded Javascript?

Re:Random Thoughts (1)

Jurily (900488) | more than 5 years ago | (#27056713)

Who wants to open up a page that suddenly uses all of their cycles and makes their computer useless for anything else while this is running.

Can you actually do that on a modern OS and a modern CPU? I regularly have my CPUs on 101% load and it stays snappy as ever. Only heavy I/O on the system drive makes it unresponsive. This applies for both Gentoo and Vista. (The default install of Linux Mint, however, sucks horribly.)

You're a faggot (-1, Troll)

Anonymous Coward | more than 5 years ago | (#27056641)

cocksucker...

A bunch of problems (3, Insightful)

Briden (1003105) | more than 5 years ago | (#27056911)

best comment on TFA:

I think this approach to MapReduce is a pretty creative angle to take on it. However, there are a number of distributed systems-type problems with doing it this way, that would need to be solved to actually make this realistically possible:

1) The dataset size is currently limited by the web server's disk size.
Possible solution: push the data to S3 or some other large store.

2) There is a single bottleneck/point-of-failure in the web server. In theory 10,000 clients could try to emit their map keys all at once to the web server. IIRC, Google's mapreduce elects nodes in the cluster to act as receivers for map keys during the map/sort phase.
Possible solution: Again, if you were using S3, you could assign them temporary tokens to push their data to S3 -- but that would be a large number of S3 PUT requests (one per key).

3) Fault-tolerance -- what happens when a node in the browser compute cluster fails for any of N reasons? How does the web server re-assign that map task? You'd especially want to ensure that computation finishes on a job in an unknown environment such as 1,000,000 random machines on the internet.
Possible solution: If you haven't heard from a node in N seconds, you could reassign their map task to someone else. This is a similar idea to the MapReduce paper's description of sending multiple machines on a single map task, and racing them to the finish.

4) Security -- there is no way to deterministically know whether the data emit()ed from a user's browser session is real or not. How do you trust the output of 1,000,000 users' Javascript browser executions (I think the answer is, you don't).

Re:A bunch of problems (3, Insightful)

AKAImBatman (238306) | more than 5 years ago | (#27057139)

Further down in the Slashdot comments, a poster also pointed out that Javascript is a poor platform for computationally intensive work. Which I agree with on a general level. The Javascript number system is designed for genericity, not performance.

In the end this is just a cute idea that has any number of practical problems. Many of them reflect the fact that distributed computing is hard, but many of them also reflect the fact that the suggested platform is less than ideal for this function. Especially if you're going to be pushing workloads that take more time and resources to transmit back and forth than to simply compute them.

Doesn't stop me from humoring him, though. We all have to dream. ;-)

And besides, this may just inspire the next fellow down the line to use the technology for a more practical purpose.

Re:New concept for worms... (1)

nobodylocalhost (1343981) | more than 5 years ago | (#27057785)

AJAX + self updating js saved in cookies

Botnet (3, Insightful)

ultrabot (200914) | more than 5 years ago | (#27056159)

Imagine how much *spam* you could send using this approach.

No, wait...

Re:Botnet (5, Insightful)

MonoSynth (323007) | more than 5 years ago | (#27056251)

With ever-increasing JavaScript performance, there's a lot of cpu power available for cracking passwords and captcha's... Just include the code in an ad and you're done. No tricky installs needed, just the idletime of the user's web browser.

Rather have a cold PC (2, Insightful)

Wee (17189) | more than 5 years ago | (#27056731)

My CPU time isn't idle. It's keeping my laptop from being too hot to touch and too noisy to work on. And there's no reason to pay more for electricity than I already do.

-B

Re:Rather have a cold PC (1)

Chabo (880571) | more than 5 years ago | (#27056817)

I really don't think laptops were designed to run at 100% all the time anyway, so yeah, I'd avoid any distributed computing projects on your computer.

I run it on my two desktops at home though, and there's barely any difference in my electric bill. Idle vs load for me is about 40W difference -- I could save more by turning off a fairly dim bedside lamp.

Re:Botnet (1)

euxneks (516538) | more than 5 years ago | (#27057187)

This makes me hate ads even more. I am so glad I use adblock plus right now.

Re:Botnet (0)

Anonymous Coward | more than 5 years ago | (#27056259)

Heh, no shit.

You don't see many Russian malware authors writing CureForCancer@home or StemCellSim@home, do ya?

Re:Botnet (1)

HTH NE1 (675604) | more than 5 years ago | (#27057111)

You don't see many Russian malware authors writing CureForCancer@home or StemCellSim@home, do ya?

No, but you might meet some of their representatives in person (for the first and last time IYKWIM) if you start implementing this and undercutting their networks.

Another article published by.... (0, Troll)

Anthony_Cargile (1336739) | more than 5 years ago | (#27056365)

KDawson! No, its not the typical KD FUD, but still - we all could continue to thrive without seeing some "proof of concept" bullshit as useless as this. Remind me again how he is still an editor? I mean, this is unbelievable! I stared at the headline and summary for a second, and then it dawned on me - I knew who posted this before even looking under the title, and that is just pitiful.

Join compute cloud (4, Insightful)

Imagix (695350) | more than 5 years ago | (#27056161)

We already have that. See botnets.

RE: Botnets (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#27056215)

I am not a botnet, you ignorant clod!

In Soviet Russia, botnet (Map)Reduces YOU!

BOINC (4, Insightful)

Chabo (880571) | more than 5 years ago | (#27056205)

If you were really interested enough to donate your CPU cycles, is it really that much harder to install BOINC, and get a job running?

Plus then you can run native code instead of having to run in [shudder]Javascript[/shudder].

Re:BOINC (0)

Anonymous Coward | more than 5 years ago | (#27056353)

BOINC is this whole huge system, and people hate installing things.

This idea captures the CPU cycles of lazy people much more effectively.

And with modern Javascript engines, Javascript can be dynamically recompiled to run at almost 1% the speed of native code! Isn't that great?

Re:BOINC (1)

Tyler Eaves (344284) | more than 5 years ago | (#27056483)

BOINC is quite possibly the single worst bit of software I've ever seen. It's kind of like the team did a detailed study of the best practices for software usability and then did the exact opposite.

Re:BOINC (1)

Chabo (880571) | more than 5 years ago | (#27056513)

If you're talking about the UI, then I'll agree it needs a bit of work, but then it is still a "nerd project" at this point. With any nerd project, the interface is at the bottom of the TODO list.

If you're talking about the code, care to explain? I've never looked at it.

Re:BOINC (1)

D Ninja (825055) | more than 5 years ago | (#27057043)

With any nerd project, the interface is at the bottom of the TODO list.

Which is exactly why many people won't use many of these tools even if the tool is "better." UI is extremely important for most people to consider using a tool.

Re:BOINC (1)

kae_verens (523642) | more than 5 years ago | (#27056845)

I don't get what's the big problem people have with "[shudder]JavaScript[/shudder]".

It's a Turing-complete language, which means it can be used to do anything from simple form validation to ray tracing and neural net simulations.
    With AJAX to handle file interactions, I don't understand the problem that people have with it. What is it that you think JavaScript can't do that 'x' language can?

I wish people would get over this childish bias and accept that JavaScript is a /real/ language, and not a toy.

The FA demonstrates that...

Re:BOINC (2, Insightful)

Chabo (880571) | more than 5 years ago | (#27057087)

A big thing is the same thing people have against VB: there may not be anything technically wrong with it, but bad programmers are drawn to it because it's easy, so you hardly ever see a good VB program. There's especially nothing wrong with VB now, when writing a program in VB.NET gets you the same result as if you'd written it in C#: you still get CIL code when it's compiled.

However, Javascript gets used for way too much, and historically it's been a huge browser security issue. Even if you use it responsibly, that doesn't mean everyone does.

Re:BOINC (1)

Anonymous Coward | more than 5 years ago | (#27057639)

have you actually tried to write something in javascript?

what works in one browser doesnt work in another...

the number of simple functionalities that are
missing sleep() for instance?

and its only just recently got threading support

every time i try to do something in javascript i come across some kind of limitation which means that i have to use some kind of strange hackish unreliable workaround!

Noscript (4, Informative)

sakdoctor (1087155) | more than 5 years ago | (#27056221)

Progress is running less JavaScript, not more.

Re:Noscript (5, Funny)

OzPeter (195038) | more than 5 years ago | (#27056325)

Sir, I have the '80s on hold on the phone at the moment. They want to know if you want to by some stuff called .. umm .. hang on .. yes here it is .. "static HTML pages" ..

Re:Noscript (4, Insightful)

wirelessbuzzers (552513) | more than 5 years ago | (#27056403)

Actually it was the '90s, but whatever. The thing is, non-DHTML web pages are actually pretty good for most things... what made those early '90s web pages so awful was no CSS, slow connections, and the fact that people really didn't know how to design for this new medium.

Probably 99% of the web still shouldn't need Javascript or flash, though pages usually do need to be dynamic on the server side.

Re:Noscript (1)

OzPeter (195038) | more than 5 years ago | (#27056677)

I know it was the '90s, I was just trying for the double joke of it being an advanced concept/vapourware for the '80s. Won't try that one again.

But the argument against javascript is one that is countered by your own comment: "the fact that people really didn't know how to design for this new medium".

Javascript is a tool just like another on the Internet. It can be used for good or evil depending on who writes the program. And as you mentioned, retreating from javascript means going back to a purely (dumb) client / (smart) server technology with every state change within the same page ina browser meaning a round trip to the server and back again.

Re:Noscript (1)

rjolley (1118681) | more than 5 years ago | (#27056693)

99% of the web COULD work without javascript while just having everything run on the server, but I happen to like web pages that benefit from XHR's. You keep your noscript plugin, and I'll keep intuitive webUI.

Re:Noscript (0)

Anonymous Coward | more than 5 years ago | (#27056741)

Enjoy your zombie pc.

Re:Noscript (1)

Jester998 (156179) | more than 5 years ago | (#27056939)

You keep your noscript plugin, and I'll keep intuitive webUI.

'Intuitive' web UI doesn't need JavaScript -- it can be done purely with CSS.

Example [meyerweb.com]

Re:Noscript (1)

Darkness404 (1287218) | more than 5 years ago | (#27057173)

Yes, until someone using an old browser attempts to use the site...

Re:Noscript (1)

rjolley (1118681) | more than 5 years ago | (#27057325)

IE6 came out almost 10 years ago. It sucks at css, but with workarounds does an ok job at javascript. If I lose .1% of my users because there browser is older than IE6 I am not too concerned.

Re:Noscript (0)

Anonymous Coward | more than 5 years ago | (#27057201)

You are forgetting that XHR's are what make the web so easy to use. All the (excuse me for saying this) web 2.0 stuff that lets you submit forms behind the scenes and update small portions of the page you are viewing without having to refresh are what make web applications usable for me. Sure I got by fine 5+ years ago, but I don't realy want to go back.

Furthermore, the current lousy adoption of css standards by MS make writing CSS just as annoying as writing javascript. With javascript at least there are convenient libraries to use that help mitigate some of this.

Re:Noscript (1)

rjolley (1118681) | more than 5 years ago | (#27057237)

Hate to reply to AC, except in this case I should point out AC is me.

Re:Noscript (1)

memco (721915) | more than 5 years ago | (#27057489)

Ok that's great, but what about keyboard users? They don't see all the sub links. And nav menus are hardly a proof that the net doesn't need javascript. I suppose in 5 years we won't need to do the same things with JS (form validation, rich text editing, zebra striping, animation, etc.), but since we're not in the future, I think it's safe to say that there are some legitimate uses for JS.

Re:Noscript (0)

Anonymous Coward | more than 5 years ago | (#27057647)

There is a little bit more to an intuitive user interface than popup menus, dickwad.

Re:Noscript (1)

wirelessbuzzers (552513) | more than 5 years ago | (#27057163)

I don't use noscript. However, I dislike most flash- and javascript-based UIs. They're often heavy and unintuitive, and they usually break the back button, deep linking and "open in a new tab".

What's more, javascript-intensive UIs are terrible for the disabled, for scripts, for search engines, and often for mobile devices as well.

That's not to say that Javascript is useless, or even Flash. They have their place in web design, but they do give designers an awful lot of rope.

Re:Noscript (1)

girlintraining (1395911) | more than 5 years ago | (#27056695)

Actually it was the '90s, but whatever. The thing is, non-DHTML web pages are actually pretty good for most things... what made those early '90s web pages so awful was no CSS, slow connections, and the fact that people really didn't know how to design for this new medium.

Sure it's fine when you've got a 2GHz processor and a smack of RAM to compile and run an interpretive language -- with the sole purpose of relatively simple data manipulation, validation, and perhaps some light processing to kick a chunk of data back. But when you are talking about serious data crunching, you want code running natively, not in a locked down little box, like SETI@Home, and optimized for that architecture and platform.

People think because you can put it on the web, you should. That is, at best naive, and at worst professionally negligent to suggest.

Re:Noscript (1)

Pollardito (781263) | more than 5 years ago | (#27057281)

what made the 90s awful was black page backgrounds, blink tags, banner tags, HR tags that dripped blood, and the GeoCities navigation that was no doubt accompanying all the above "features"

Re:Noscript (1)

AKAImBatman (238306) | more than 5 years ago | (#27056335)

Which is to say that Javascript is the future of all computing progress.

"First they ignore you, then they ridicule you, then they fight you, then you win." --Mahatma Gandhi

Mwuhahaha! MWHahaha! MUHAHAHAAAAAAaaaaaa--

*cough* *cough* *wheeze* *sputter* *wheeze*

*clears throat* 'scuze me!

MWUHAHAHAAHAHAAAA!!!! :-P

Re:Noscript (1)

Camann (1486759) | more than 5 years ago | (#27056443)

Thank you for jumping straight to ridicule. I was afraid of having to spend a bunch of time in "ignore", but this is much faster.

Re:Noscript (1)

AKAImBatman (238306) | more than 5 years ago | (#27056753)

What is wrong, Sid? You no like-ink my evil laugh? Here, I do it more: Mwhahahah! :-P

Re:Noscript (2, Informative)

Anonymous Coward | more than 5 years ago | (#27056683)

"First they march you through hundereds of miles of jungle without food or water, then they shoot you, then they disembowel you, then you lose." --Mahatma Gandhi, had the Japs won WW2.

Re:Noscript (1)

Reality Master 101 (179095) | more than 5 years ago | (#27056363)

Yeah, yeah, and Usenet was the ultimate discussion group and everything's been downhill from there, right? And 25x80 column monitors were plenty (who needs proportional fonts?) and color is way overrated, and...

Why is it that we always need the previous generation twho remembers "what it was like before all this newfangled nonsense" to die off before we can make progress?

Just because you're looking for the web to look like a static newspaper doesn't mean the rest of the world wants the same thing.

Re:Noscript (1)

pjt33 (739471) | more than 5 years ago | (#27056439)

Just because you're looking for the web to look like a static newspaper doesn't mean the rest of the world wants the same thing.

There are situations where JavaScript is good, but it simply breaks things like the ability to bookmark your page and then restore it as it was from the bookmark. Then you have the sites which really abuse it: for example, you can't book a flight with Ryanair if you have JS disabled (or a browser which doesn't support it: they don't seem to have come across the concept of degrading gracefully).

Re:Noscript (0)

Anonymous Coward | more than 5 years ago | (#27056775)

seriously, bookmark breaking and lack of graceful degradation is because of javascript.. sounds like shitty developers to me.

Re:Noscript (1)

MrEricSir (398214) | more than 5 years ago | (#27056469)

If the static newspaper means nobody can make my browser window dance around the screen while Rick Astley plays, please, PLEASE count me in.

Re:Noscript (0)

Anonymous Coward | more than 5 years ago | (#27056545)

Think his point is that static HTML doesn't infect the client with mal-ware.

Re:Noscript (1)

localman (111171) | more than 5 years ago | (#27056997)

How come the new kids who come in can't tell the difference between progress and two-steps-forward-two-steps-back? You make a valid point that some people resist change for poor reasons, but I would say an equal or greater problem is people embracing change for poor reasons.

DHTML is fine when it works, and it's just starting to get there. But I'd say that web usability was at an all-time low between 2000 and 2006 when the new kids thought everything should be dynamic without the slightest understanding of usability or browser compatibility. The number of high-profile busted pages I encountered was staggering. It wouldn't be so annoying to go through such phases if there was an honest recognition of the shortcomings, but the new kids are always religious zealots who think anyone criticizing their "progress" is a stupid old fart.

It's like how young web developers can't fathom why anyone would still want to use tables for layout when it's fairly easy to explain [www.olav.dk] (i.e. lack of grid layouts/element size dependencies). Fine, I'm happy about CSS too, but because you're having so much fun masturbating over it and won't admit that there are some things lacking, we can't make _real_ progress.

Ah well, it all works out in the end after 10 rewrites to get the same functionality you had in the first place.

Would this work for music? (4, Funny)

Anonymous Coward | more than 5 years ago | (#27056241)

You could also use this to index the MP3 files on everybody's hard drives, then share the music just by visiting a URL!! ... oh wait...

Why? Why? WHYWHYWHYWHY??? (5, Insightful)

wirelessbuzzers (552513) | more than 5 years ago | (#27056333)

Javascript really isn't suited for this kind of thing, even with worker threads, for two reasons I can think of. First, web clients are transient... they'd have to report back often in case the user clicks away.

But more importantly, Javascript just isn't a good language for massive computation. It only supports one kind of number (double), has no vectorization or multicore capabilities, has no unboxed arrays, and even for basically scalar code is some 40x slower than C, let alone optimized ASM compute kernels. (This is for crypto on Google Chrome. Other browsers are considerably slower on this benchmark. YMMV.)

Re:Why? Why? WHYWHYWHYWHY??? (1)

godrik (1287354) | more than 5 years ago | (#27056541)

You cannot say that guy. This is web 2.0. This is progress. If it does not work in your browser, you can still an web 2.0 internet browser application to make it compatible.

Re:Why? Why? WHYWHYWHYWHY??? (1)

doti (966971) | more than 5 years ago | (#27057235)

I think you accidentally the last sentence of your post.

Re:Why? Why? WHYWHYWHYWHY??? (4, Insightful)

Instine (963303) | more than 5 years ago | (#27056785)

and you don't think you could get 100 times more users to visit your web app than you could convince to download and install an exe?

Re:Why? Why? WHYWHYWHYWHY??? (1)

psetzer (714543) | more than 5 years ago | (#27057093)

Or they could use an Applet or JWS and get several times the performance for only a mild reduction in install base. JWS would even be able to run offline or when the browser window's closed and cache some output to a JVM-managed scratchpad file on disk.

Re:Why? Why? WHYWHYWHYWHY??? (2, Interesting)

wirelessbuzzers (552513) | more than 5 years ago | (#27057431)

It would need to be 10000x at the very minimum.

If a user downloads, say, folding@home, it's running all day, every day, on all cores of the machine, whenever the computer is on and idle, which is most of the time. The user doesn't have to remember to run it, doesn't have to devote screen real estate, attention and so on, and the program is less annoying because of its low priority and relatively low memory footprint (less boxing).

Additionally, the 40x I cited was in the fastest available browser (Chrome), compared to a relatively slow implementation (OpenSSL), for code that doesn't benefit from vectorization (at least, not on x86-based processors). I expect that the difference between a scientific compute kernel in JS and in assembly would be at least 100x, maybe 200x or more.

Let's suppose that everyone in your rosy world uses FF 3.1 with JIT. That's 3-5x slower than Chrome in my benchmarks; say 4x. Let's suppose that Chrome is 25x slower than unvectorized C, which is 4x slower than optimized assembly. Let's say people run the site 5 hours a day on one core for a week, but have their dual-core computers on for 10 hours a day, 90% idle and would keep folding@home installed for a year.

Then the EXE is 4 * 25 * 4 * 2 * 2 * 50 * 0.9 = 72000x more productive.

Use the right tool for the job.

Re:Why? Why? WHYWHYWHYWHY??? (3, Insightful)

Nebu (566313) | more than 5 years ago | (#27057379)

Javascript really isn't suited for this kind of thing, even with worker threads, for two reasons I can think of. First, web clients are transient... they'd have to report back often in case the user clicks away.

I don't see why web clients being transient is a problem. The whole point of the MapReduce algorithm is that each worker (the web clients in this case) don't need to know anything about what the other worker is doing, what the system as a whole is doing, nor what it had done with any past job.

I want in (1)

illegibledotorg (1123239) | more than 5 years ago | (#27056343)

So, where do I sign up for grid time?
Time to work on a Javascript MD5 bruteforce implementation. SHA, too.

Link (5, Informative)

Jamamala (983884) | more than 5 years ago | (#27056357)

for those like myself that had no idea what MapReduce was:
http://en.wikipedia.org/wiki/MapReduce [wikipedia.org]

Re:Link (5, Insightful)

MarkGriz (520778) | more than 5 years ago | (#27056569)

Thank you. Nice that we have "volunteer" editors, since slashdot doesn't seem to employ them any longer.

Re:Link (3, Funny)

Logic and Reason (952833) | more than 5 years ago | (#27056857)

Thank you! I had completely forgotten how to use Google!

Re:Link (4, Funny)

Anonymous Coward | more than 5 years ago | (#27057169)

So had I; here's a link: http://www.google.com [google.com]

Been there, done that... (0)

Anonymous Coward | more than 5 years ago | (#27056397)

The virus/worm writers, that is...

[Personal Opinion]

I think i recognize this one.... (1)

Ommadawn (5636) | more than 5 years ago | (#27056421)

joining a supercomputer by just clicking a URL.. isn't that the definition of a botnet?

MapReduce fanboyism (5, Insightful)

Estanislao Martnez (203477) | more than 5 years ago | (#27056429)

Oh, please, make the MapReduce fanboyism stop.

Yes, it's a neat technique. It's also very old and obvious. Google's implementation is also good, but this stuff is just not rocket surgery. It's just a simple pattern of how to massively parallelize some types of computational tasks.

But somehow, just because some dudes at Google wrote a paper about it, it's become the second coming of Alan Turing or something among some silly folks. Hell, a couple of weeks ago somebody was saying on the comments here that MapReduce was a good alternative to relational databases. Now that is silly.

Re:MapReduce fanboyism (3, Funny)

Anonymous Coward | more than 5 years ago | (#27056537)

listen to him everyone, he must know what he's talking about since I don't know what rocket surgery is.

rocket surgery (1)

doti (966971) | more than 5 years ago | (#27057255)

It's probably related to brain science.

Another way to hijack a browser? (1)

Crashspeeder (1468723) | more than 5 years ago | (#27056457)

If clicking a link is all it would take to donate my CPU cycles I'm getting off the internet now. I'm sure we've all been conned into clicking on tubgirl, goatse, lemon party, etc. what's to stop somebody from conning you into clicking a mass computing (see botnet above) cloud masked by a fun but pointless flash game people could enjoy for hours on end?

One-click CPU cycle donation is a bad idea. Make it deliberate, make it a program that has 30 checkboxes making sure the user understands they're donating cycles and on their terms.

Re:Another way to hijack a browser? (2, Insightful)

Darkness404 (1287218) | more than 5 years ago | (#27057231)

But, you can just close the browser or type "killall firefox" and the program dies and you have to go to the URL again to get to it. So, though this is bad for other reasons, yours just isn't one of them.

Re:Another way to hijack a browser? (1)

Crashspeeder (1468723) | more than 5 years ago | (#27057781)

But, you can just close the browser or type "killall firefox" and the program dies and you have to go to the URL again to get to it. So, though this is bad for other reasons, yours just isn't one of them.

please reread the following section:

masked by a fun but pointless flash game people could enjoy for hours on end

As in, people would keep it open playing the flash game. I didn't say you couldn't close the browser, I implied one could be tricked into keeping it open. I would again contend that my reason is one of them.

I stopped at... (2, Insightful)

greymond (539980) | more than 5 years ago | (#27056481)

"Javascript...â" every browser can run it..."

There is a huge difference between being able to run javascript apps and run javascript apps well - not to forget that a lot of the javascript I see out there really only works on PC's with IE or Firefox, Opera and Safari, especially on OS X seem to have trouble with some sites that aren't coded for compatibility, but instead pushed out quickly with little regard for anything other than IE on Windows.

Bandwidth and Exercise (3, Insightful)

clinko (232501) | more than 5 years ago | (#27056505)

A common mistake in multi-server builds is that bandwidth is free.

Bandwidth Costs Money and Time. Both are reduced by having the network closer to the processing. This is one of the reasons google bought all that "dark fiber" left around after the .com bust.

Another flaw is that computation of data is difficult to provide "good results" in blocks unless they're doing relativity matrices (Think PageRank).

Something to think about:
If I'm sending names to your pc, what can I derive from that list without having the entire list?

Re:Bandwidth and Exercise (2, Interesting)

Nebu (566313) | more than 5 years ago | (#27057463)

Something to think about: If I'm sending names to your pc, what can I derive from that list without having the entire list?

Frequency of each name? Frequency of characters in names? Bayesian probability of one character following another in names? Number of names of a particular length?

Each worker would compute the stats for their chunk of work (the "Map" part of MapReduce), and then send the results back to the server to be aggregated (the "Reduce" part of MapReduce).

Some of these may seem interesting, but then again, what interesting data can you derive at all from a list of names, even if you had the whole list?

Pay Me (4, Interesting)

Doc Ruby (173196) | more than 5 years ago | (#27056529)

If there were a couple-few or more orgs competing to use my extra cycles, outbidding each other with money in my account buying my cycles, I might trust them to control those extra cycles. If they sold time on their distributed supercomputer, they'd have money to pay me.

As a variation, I wouldn't be surprised to see Google distribute its own computing load onto the browsers creating that load.

Though both models raise the question of how to protect that distributed computing from being attacked by someone hostile, poisoning the results to damage the central controller's value from it (or its core business).

Re:Pay Me (1)

dencarl (138314) | more than 5 years ago | (#27056909)

The comments to the linked article are quite interesting. http://www.pluraprocessing.com/ [pluraprocessing.com] seems to have a viable business model for doing this.

Re:Pay Me (1, Insightful)

Anonymous Coward | more than 5 years ago | (#27057827)

Start paying attention to the ads. Look for "Click here to help X and get paid ONE MILLION DOLLARS!"

Scripts taking too long (4, Funny)

Anonymous Coward | more than 5 years ago | (#27056553)

Is this why my browser keeps telling me scripts on the slashdot main page are taking too long and do I want to stop them for the last few months?

God, I fucking hate Slashdot. (0)

Anonymous Coward | more than 5 years ago | (#27057361)

And why clicking on a tag pops up a login box, NOT THE ARTICLES TAGGED WITH THE TAG?

Change != progress, assmonkeys.

Self-defeating idea (2, Insightful)

whizbang77045 (1342005) | more than 5 years ago | (#27056595)

This seems to me a self-defeating idea. The obvious goal is to get more processing power. Yet using a scripted language is inefficient, and a waste of processing power. If you want more processing power, you need to group computers of the same general instruction set, and which can run compiled (or, dare I say it?) assembled machine code.

Re:Self-defeating idea (1)

godrik (1287354) | more than 5 years ago | (#27056811)

you can do this at runtime.
a daemon runs on the machine and analyze the machine, processor, operating system... and download the right binary to do the computation.
This is classical. Desktop grid guys do this all the time.

Yes, this solution is stupid.
You gain:
-people do not have to install a software on their machine.
you lose:
-you have to write your application in JS or use a langage interpreted in js
-browser incompatibilities
-no hardware information, which means no low level optimization, which are the basis of HPC.

Oh, but I forgot, this is web 2.0, so I must be wrong, this is obviously fucking cool

Map Reduce is an astonishing concept. (0)

Anonymous Coward | more than 5 years ago | (#27056639)

This revolutionary technology can be used to create a superior networked colonial defence system.

-- Gaius Baltar

FORTRAN FTW (0)

Anonymous Coward | more than 5 years ago | (#27056673)

First off, I'm impressed by the geekiness and the creativity of the idea. It's awesome.

But, scientists had better performance 10 years ago. On a single machine. Because they wrote things in FORTRAN.

"crowdsource" (0)

Anonymous Coward | more than 5 years ago | (#27056689)

"Crowdsource" is not a word. It's not hip or cool or whatever, it's an annoying buzzword. Stop using it. kthxbye.

Fully distributed P2P (-1, Offtopic)

CookieOfFortune (955407) | more than 5 years ago | (#27056691)

Why use a torrent program at all when you can just run it over HTTP and process it all in JS?

ep... (-1, Troll)

Anonymous Coward | more than 5 years ago | (#27056705)

imagine a Beowulf cluster of trojans and drive-bys (1)

swschrad (312009) | more than 5 years ago | (#27056709)

hijack-weasels who need to be shot have pretty much ruined the idea of distributed donated computing resources, thanks.

Waste of electricity - Javascript harms the Earth (0)

Anonymous Coward | more than 5 years ago | (#27056763)

install powertop and watch what those "spare" CPU cycles cost you in terms of Watts. Every computation costs energy. Even worse is when you waste energy on inefficient computation. Don't promote wasteful computing, use a real language that is actually capable of providing fast array access (not possible in JS due to prototypes and parent array deletions).

More Concrete Objectives Please (1)

deanston (1252868) | more than 5 years ago | (#27056869)

Think Google already does that with their Chrome, Desktop, and Updater apps on your machine. Problem I have is how the author dismiss MOST of the issues and purpose off hand with an over-simplified sentence: "Aside from storing and distributing the data the most expensive part of any job is the CPU time..." (my emphasis). The LHC project would've given us a chance to test the capability of one of the world's biggest Grid infrastructure - if it ever run in production. P2P does not a Super-Computer/Search Engine make.

Most popular mature application has a scripting interface. Like it or not, if MSFT cannot kill JavaScript, it is here to stay. But to condamn JS straight away is like saying no language - C/C++/C#/Java/etc. - can and ever should evolve. Might as well belittle Ruby/Perl too since they cannot even render a web page without HTML/CSS. So is proprietary plug-in like Adobe Flash/Flex your preference? Just get rid of browsers altogether and implemnet custom HTTP interface in each app?

Not an original idea... (1)

ody5iu5 (1235270) | more than 5 years ago | (#27056979)

http://www.pluraprocessing.com/ [pluraprocessing.com] has been doing this for a while now using Java.

What about the data? (1)

leenks (906881) | more than 5 years ago | (#27057005)

MapReduce is interesting because at Google and Hadoop it has a distributed filesystem underneath it too. The clever part is how data is distributed, and processing is moved to the data rather than moving data to the processing. I don't really see how this really helps matters, unless you are going to have data involved too - which brings in the privacy concerns yadda yadda yadda.

Sure, some things would work that require huge amounts of processing power on limited data, but why would you use map-reduce for that - why not just use conventional divide and conquer techniques.

Skynet? Really? (1)

Milamber_Cubed (949767) | more than 5 years ago | (#27057015)

Does anyone else find it worrying that one of the projects linked to in TFA is called (out of choice) Skynet? Unlikely, I know, but really, why tempt fate?

Yes but.. (0)

Anonymous Coward | more than 5 years ago | (#27057081)

is it beta?

Sweet! I'm HTTP GETting a bunch of doubles (1)

kkrajewski (1459331) | more than 5 years ago | (#27057607)

And I'll perform floating point operations on them with my Pentium and send them back.

(ca. '94 rimshot)

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>