Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Facebook open sources its Hadoop job scheduling enhancement, Corona

stormspirit (2770655) writes | about a year and a half ago

Facebook 0

stormspirit (2770655) writes "Today, Facebook open sources its scalable Map-Reduce job scheduling framework, Corona, as an enhancement for Hadoop. Facebook has been using Hadoop as MapReduce framework for years. According to the engineering notes, by early 2011, it started to hit the scalability wall of Hadoop. In particular, the job scheduling module became the bottleneck for large workload as there's only a single job tracker in Hadoop to both manage cluster resources and task trackers. Corona mainly divide the two responsibilities between a central resource manager and per task job tracker, allowing the module to scale. Other improvement includes: a pushed-based task assignment, dynamic resource management and better fairness guarantee. Corona is now deployed in all production systems with its design goals achieved."

cancel ×

0 comments

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...