Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Facebook's Prism, Soon To Be Open Sourced, Gives Hadoop Delay Tolerance

timothy posted about a year and a half ago | from the and-eventually-smoke-signals dept.

Facebook 17

snydeq writes "Facebook has said that it will soon open source Prism, an internal project that supports geographically distributed Hadoop data stores, thereby removing the limits on Hadoop's capacity to crunch data. 'The problem is that Hadoop must confine data to one physical data center location. Although Hadoop is a batch processing system, it's tightly coupled, and it will not tolerate more than a few milliseconds delay among servers in a Hadoop cluster. With Prism, a logical abstraction layer is added so that a Hadoop cluster can run across multiple data centers, effectively removing limits on capacity.'"

cancel ×

17 comments

This changes everything (1)

Anonymous Coward | about a year and a half ago | (#41868115)

...but when?

Re:This changes everything (4, Insightful)

interval1066 (668936) | about a year and a half ago | (#41868149)

Actually, its pretty cool. Its a solution to a problem that needed a solution, for once. Quite frankly, even though I'm not an army of PhD C-Sci scientists, I'm sorry I couldn't have come up with it. Its weird little problems like this with their solutions that win the "cool" race. Or the "king of geeks" race, or whatever you want to call the brainaic metric.

Re:This changes everything (4, Funny)

Anonymous Coward | about a year and a half ago | (#41868269)

Sounds like it shouldn't be hard to...Hadooplicate?

Re:This changes everything (0)

Anonymous Coward | about a year and a half ago | (#41868385)

So cheesy, but so funny.

Re:This changes everything (1)

nurb432 (527695) | about a year and a half ago | (#41868183)

soon

You FAIL it... (-1)

Anonymous Coward | about a year and a half ago | (#41868603)

you ne3d to suuceed

Imagine that (-1)

Anonymous Coward | about a year and a half ago | (#41869539)

Slashtards can't say anything about this. This is about real computing, not just fanboi chest pounding. The local faggots are at a loss of what to say!

I think it's great (1)

koan (80826) | about a year and a half ago | (#41869575)

Love to see useful stuff open sourced, but part of me is annoyed it is Facebook doing it.

Re:I think it's great (0)

Anonymous Coward | about a year and a half ago | (#41869735)

Oh don't worry, knowing Facebook, using their software will probably Hadooplicate your data and make it all public.

And then monetize it.

And then corrupt it.

And then lose it.

Re:I think it's great (0)

Anonymous Coward | about a year and a half ago | (#41870275)

Glad your understanding of how open source works is so lacking. You and the grandparent poster shouldn't stop your nerd rage though. Keep up the virginity.

Re:I think it's great (0)

Anonymous Coward | about a year and a half ago | (#41873725)

Glad your understanding of how satire works is so lacking.

Re:I think it's great (0)

Anonymous Coward | about a year and a half ago | (#41873415)

As much as I hate Facebook, they really do have some great engineers and scientists working behind the scenes.

Facebook Prison (1)

SirAdelaide (1432553) | about a year and a half ago | (#41877021)

I misread the title, was disappointed until I saw the word Hadoop. It's such a a silly name.

What? (1)

fa2k (881632) | about a year and a half ago | (#41871893)

You typically have O(ms) seek latency for hard drives, does this mean that Facebook had all data in RAM before they made Prism?

Re:What? (1)

garaged (579941) | about a year and a half ago | (#41876673)

Google does that, I wouldn't be surprised that facebook does it too

Re:What? (1)

haruchai (17472) | about a year and a half ago | (#41877233)

Altavista was doing this way back. When the typical Windows desktop was 16 - 32 MB RAM, they have a RAM cache of up to 64GB.

(Relatively) lay explanation of bottleneck? (1)

DriedClexler (814907) | about a year and a half ago | (#41877329)

What is the sub-problem when running a Hadoop job that has this bottleneck and requires such low latency? Is it something that could have been avoided for a start?

And how does (or if, predictably, the media reports don't explain it, *would*) a logical abstraction layer solve this problem such that Hadoop's programmers couldn't have more easily done it within the application's own code?

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...