Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Facebook's Prism, Soon To Be Open Sourced, Gives Hadoop Delay Tolerance

timothy posted about 2 years ago | from the and-eventually-smoke-signals dept.

Facebook 17

snydeq writes "Facebook has said that it will soon open source Prism, an internal project that supports geographically distributed Hadoop data stores, thereby removing the limits on Hadoop's capacity to crunch data. 'The problem is that Hadoop must confine data to one physical data center location. Although Hadoop is a batch processing system, it's tightly coupled, and it will not tolerate more than a few milliseconds delay among servers in a Hadoop cluster. With Prism, a logical abstraction layer is added so that a Hadoop cluster can run across multiple data centers, effectively removing limits on capacity.'"

Sorry! There are no comments related to the filter you selected.

This changes everything (1)

Anonymous Coward | about 2 years ago | (#41868115)

...but when?

Re:This changes everything (4, Insightful)

interval1066 (668936) | about 2 years ago | (#41868149)

Actually, its pretty cool. Its a solution to a problem that needed a solution, for once. Quite frankly, even though I'm not an army of PhD C-Sci scientists, I'm sorry I couldn't have come up with it. Its weird little problems like this with their solutions that win the "cool" race. Or the "king of geeks" race, or whatever you want to call the brainaic metric.

Re:This changes everything (4, Funny)

Anonymous Coward | about 2 years ago | (#41868269)

Sounds like it shouldn't be hard to...Hadooplicate?

Re:This changes everything (0)

Anonymous Coward | about 2 years ago | (#41868385)

So cheesy, but so funny.

Re:This changes everything (1)

nurb432 (527695) | about 2 years ago | (#41868183)


You FAIL it... (-1)

Anonymous Coward | about 2 years ago | (#41868603)

you ne3d to suuceed

Imagine that (-1)

Anonymous Coward | about 2 years ago | (#41869539)

Slashtards can't say anything about this. This is about real computing, not just fanboi chest pounding. The local faggots are at a loss of what to say!

I think it's great (1)

koan (80826) | about 2 years ago | (#41869575)

Love to see useful stuff open sourced, but part of me is annoyed it is Facebook doing it.

Re:I think it's great (0)

Anonymous Coward | about 2 years ago | (#41869735)

Oh don't worry, knowing Facebook, using their software will probably Hadooplicate your data and make it all public.

And then monetize it.

And then corrupt it.

And then lose it.

Re:I think it's great (0)

Anonymous Coward | about 2 years ago | (#41870275)

Glad your understanding of how open source works is so lacking. You and the grandparent poster shouldn't stop your nerd rage though. Keep up the virginity.

Re:I think it's great (0)

Anonymous Coward | about 2 years ago | (#41873725)

Glad your understanding of how satire works is so lacking.

Re:I think it's great (0)

Anonymous Coward | about 2 years ago | (#41873415)

As much as I hate Facebook, they really do have some great engineers and scientists working behind the scenes.

Facebook Prison (1)

SirAdelaide (1432553) | about 2 years ago | (#41877021)

I misread the title, was disappointed until I saw the word Hadoop. It's such a a silly name.

What? (1)

fa2k (881632) | about 2 years ago | (#41871893)

You typically have O(ms) seek latency for hard drives, does this mean that Facebook had all data in RAM before they made Prism?

Re:What? (1)

garaged (579941) | about 2 years ago | (#41876673)

Google does that, I wouldn't be surprised that facebook does it too

Re:What? (1)

haruchai (17472) | about 2 years ago | (#41877233)

Altavista was doing this way back. When the typical Windows desktop was 16 - 32 MB RAM, they have a RAM cache of up to 64GB.

(Relatively) lay explanation of bottleneck? (1)

DriedClexler (814907) | about 2 years ago | (#41877329)

What is the sub-problem when running a Hadoop job that has this bottleneck and requires such low latency? Is it something that could have been avoided for a start?

And how does (or if, predictably, the media reports don't explain it, *would*) a logical abstraction layer solve this problem such that Hadoop's programmers couldn't have more easily done it within the application's own code?

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?