Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Twitter On Scala

kdawson posted more than 5 years ago | from the going-off-the-rails dept.

Programming 324

machaut writes "Twitter, one of the highest profile Ruby on Rails-backed websites on the Internet, has in the past year started replacing some of their Ruby infrastructure with an emerging language called Scala, developed by Martin Odersky at Switzerland's École Polytechnique Fédérale de Lausanne. Although they still prefer Ruby on Rails for user-facing web applications, Twitter's developers have started replacing Ruby daemon servers with Scala alternatives, and plan eventually to serve API requests, which comprise the majority of their traffic, with Scala instead of Ruby. This week several articles have appeared that discuss this shift at Twitter. A technical interview with three Twitter developers was published on Artima. One of those developers, Alex Payne, Twitter's API lead, gave a talk on this subject at the Web 2.0 Expo this week, which was covered by Technology Review and The Register."

cancel ×

324 comments

Sorry! There are no comments related to the filter you selected.

Should have used PHP. (4, Interesting)

0100010001010011 (652467) | more than 5 years ago | (#27495261)

Kidding aside, is this a 'nail' in the coffin of scalable Ruby? 5 years ago people were saying the same thing about PHP scaling but Facebook has done a rather nice job of making it scale. Twitter was supposed to be the poster child of how awesome Ruby and RoR was.

Difference is, Facebook is still using php, Twitter is going toScala.

Re:Should have used PHP. (1)

stoolpigeon (454276) | more than 5 years ago | (#27495285)

It's funny you mention that on a day that facebook has been up and down like a yo-yo.

Re:Should have used PHP. (2, Insightful)

Foofoobar (318279) | more than 5 years ago | (#27495365)

Never was a nail except for the Ruby community that was in denial. Ever developer in the world except Ruby fanboys knew Ruby's inherent problem was scalability. Benchmarks showed it but they would always slant their own benchmarks to show the opposite. But facts are facts and in the end you can't deny the truth. So this is where we are at. The question is will Ruby fanboys still choose to deny the issues with Ruby or accept that it does have inherent limitations?

Re:Should have used PHP. (4, Insightful)

K. S. Kyosuke (729550) | more than 5 years ago | (#27495517)

Ruby is a language. Languages usually don't have problems with scalability.

Re:Should have used PHP. (-1, Troll)

Anonymous Coward | more than 5 years ago | (#27495579)

Ruby fanboy found. A language is only as good as its compiler/interpreter.

Re:Should have used PHP. (2, Insightful)

nicolas.kassis (875270) | more than 5 years ago | (#27495759)

Except most languages have more than one interpreter/compiler whit varying performance in varying environments. Jython/IronPython/Python all perform a little different. Ruby 1.9 and MacRuby don't perform the same.

Re:Should have used PHP. (1)

grumbel (592662) | more than 5 years ago | (#27496169)

The problem is that most of those compiler/interpreters suck enormously. Or is there any high level 'scripting' language out there that can actually measure up to a decent Lisp implementation?

Re:Should have used PHP. (1)

K. S. Kyosuke (729550) | more than 5 years ago | (#27496353)

Sorry, this is not the fanboy you are looking for. This one is a Scheme fanboy. ;-)

Re:Should have used PHP. (2, Informative)

digitig (1056110) | more than 5 years ago | (#27495675)

Really? Try comparing the scalability of, say, FORTRAN or COBOL against any modern language with decent encapsulation.

Re:Should have used PHP. (1)

K. S. Kyosuke (729550) | more than 5 years ago | (#27496139)

"Decent encapsulation?" Don't know what that is supposed to mean here, but Common Lisp, e.g., fares quite favourably against Fortran.

Re:Should have used PHP. (5, Insightful)

tcopeland (32225) | more than 5 years ago | (#27495761)

> Ruby is a language. Languages usually don't have problems with scalability.

Quite right. An application with 8 million users will have scalability challenges regardless of what type of language opcodes are being executed. At some point it's all about architecture.

Re:Should have used PHP. (1)

Foofoobar (318279) | more than 5 years ago | (#27495885)

Ruby is a language. Languages usually don't have problems with scalability.

So you are saying Visual Basic and C are going to have the same scaling challenges and that they scale equally well? Well all I can say it pass that magical crack pipe cause I want a toke too.

Re:Should have used PHP. (2, Insightful)

K. S. Kyosuke (729550) | more than 5 years ago | (#27496341)

Visual Basic was so stupid that Microsoft could have easily made it almost as fast as C - if only they had wanted. I guess they had other priorities.

I must ask one thing: Have you ever heard of Self 93? They made it so good fifteen years ago that it topped at half the speed of C...a language where everything (!) was a message send and *anything* could have been redefined at *any* moment.

If someone threw the necessary money at Ruby, it could have easily the performance of a modern Smalltalk implementation, as Ruby is essentially Smalltalk in diguise. The only major difference I spotted was the fact that Ruby objects behave as hashtables, whereas Smalltalk objects are essentially arrays of references with methods fetching the references ("instance variables") using fixed indices that change only during class redefinitions. I guess you could fake the former with the latter, enjoying a speedup, as even in Ruby most objects of a single class have the same attributes.

But still, I guess that the perf. problems of Twitter are primarily due to Rails and not to Ruby.

Re:Should have used PHP. (1)

mini me (132455) | more than 5 years ago | (#27495851)

Everyone knows that Ruby has performance issues. The fact is that the performance problems are not problems for short-lived operations like serving web pages. The article even backs up the claim.

Ruby scales just as well as every other language. The article never once claimed that Ruby does not scale. It claimed Ruby had performance problems; a topic mostly unrelated to scaling.

If Ruby users were in denial about the problems with Ruby, there would not be several projects working on solving the problems. However, like I mentioned above, just because a problem exists in one domain does not mean it exists in another. Ruby will scale and perform just fine under tasks the Ruby fanboys you refer to are using it for (like Rails applications).

Re:Should have used PHP. (1)

Foofoobar (318279) | more than 5 years ago | (#27496019)

And where Facebook and PHP can accomplish this goal for the frontend and the backend (and many other corporate and enterprise PHP sites), Ruby can merely do frontend work. That doesn't seem like scalability to me. That seems very limited. That to me says 'as long as you don't TAX the engine, it can perform'. That's not scaling. That's getting by.

Re:Should have used PHP. (1, Informative)

Anonymous Coward | more than 5 years ago | (#27496203)

PHP on the backend is as bad if not worse than Ruby (probably more in the worse category). It does not even have threading built in. Try building a php messaging queue and let me know how that goes. Facebook does not use PHP on the backend anymore than twitter was using Ruby for it. They have a ton of different stuff in many languages. I don't think you get what they are trying to do with Scala.

There you go again! (5, Informative)

Jane Q. Public (1010737) | more than 5 years ago | (#27495909)

Ruby does not have a problem scaling. Neither, for that matter, does even Rails. (As the companies that run Basecamp, Campfire, LinkedIn, Lighthouse, and many others will tell you.)

The fact is that the Twitter folks tried to write their own message queue in Ruby [unlimitednovelty.com] , when there was absolutely no reason to do so: there were plenty of pre-made message queues already available for Ruby, and already optimized. Not only did they choose to write their own, unnecessarily, they did it badly [obiefernandez.com] .

And not only that, but Alex Payne has a hidden agenda: he is trying to push Scala to boost interest in the book about Scala he just wrote! [obiefernandez.com]

Please get some facts before digging up this long-dead and well-buried "Ruby or Rails doesn't scale" bullshit again.

Re:There you go again! (5, Interesting)

Radhruin (875377) | more than 5 years ago | (#27496199)

Anyone who thinks Ruby [hulu.com] on [amazon.com] Rails [zvents.com] can't [scribd.com] scale [yellowpages.com] is as dogmatic in their anti-hype as the original hypers were. The right tool for the right job and all that.

Re:There you go again! (1)

mckinnsb (984522) | more than 5 years ago | (#27496267)

Mod parent up. He is dead on.

Re:There you go again! (1)

chromatic (9471) | more than 5 years ago | (#27496287)

Alex Payne has a hidden agenda: he is trying to push Scala to boost interest in the book about Scala he just wrote!

If that's damning criticism, it shows the stupidity of someone for believing that a technical book will sell enough copies that the author will earn more than minimum wage from writing it (a $39.95 book from O'Reilly with two authors gives Payne about $1 per copy -- he might earn $12k over three years).

Re:Should have used PHP. (1)

vlm (69642) | more than 5 years ago | (#27495981)

Never was a nail except for the Ruby community that was in denial.

Well, yeah, no kidding. Thats because all I see is an endless circle of people repeating what they heard other people saying:

1) All the cool kids (us) know it won't scale. If you disagree, then you're not cool. Ha ha, thats our proof it doesn't scale.

2) I don't use it, never used it, never will use it, don't know anyone using it, but I am certain it won't work.

3) If it won't work on the worlds biggest websites, I can't use it on the departmental intranet, because I'm as cool as the worlds biggest website, just the world doesn't know it yet.

I can't find anything on google explaining why it "won't scale". Sometimes you can find ridiculous marketing droid babble about "tiers" but nothing technical or even logical.

No one can explain, why it matters if it can't handle the load of one of the top 10 or top 100 websites in the entire world, it therefore is utterly unacceptable on the other 99.999999999% of internet and intranet sites.

Personally, I think it's because the RoR user/devs are too busy making profitable apps to bother discussing foolishness?

Re:Should have used PHP. (0)

Anonymous Coward | more than 5 years ago | (#27496165)

They started mega-hyping RoR before any of the application servers got out of alpha stage, that's the main reason it "won't scale".

Also Ruby is just slow.

Re:Should have used PHP. (2, Insightful)

nicolas.kassis (875270) | more than 5 years ago | (#27496297)

Also Ruby is just slow.

Not anymore. Go install 1.9 and start using it TODAY.

The ruby community needs to leave 1.8.* to die as quickly as possible.

Re:Should have used PHP. (5, Insightful)

julesh (229690) | more than 5 years ago | (#27495481)

Difference is, Facebook is still using php, Twitter is going toScala.

PHP was a mature environment when facebook was launched. RoR was (and still is to a certain extent) a fad environment, popular primarily because of its differentness. People who build sites on a platform because it's the latest thing are less likely to stick with that platform than people who choose a platform that has a solid reputation but is boring. Scala, at a guess, is going to be the next fad platform. Like Ruby, it has some interesting ideas behind it, but it needs a lot of development before we can consider a stable platform for serious applications, I think.

Re:Should have used PHP. (2, Interesting)

Radhruin (875377) | more than 5 years ago | (#27495967)

I find your assertion of differentness being the main reason to use Ruby on Rails to be somewhat offensive let alone uninformed, as it suggests that the multitudes of developers using it are doing so not because of technical merits but because they're buying in to some image of differentness. A cursory examination of the typical Rails project and developer should indicate otherwise. Because you don't find it helpful in your work doesn't mean others don't find it helpful or see real benefits from using the system. Perhaps you should be less dismissive -- you'd find yourself with a lot more interesting stuff to discover!

Re:Should have used PHP. (0)

Anonymous Coward | more than 5 years ago | (#27496185)

Ruby and PHP launched the same year.

Re:Should have used PHP. (4, Insightful)

iluvcapra (782887) | more than 5 years ago | (#27496247)

RoR was (and still is to a certain extent) a fad environment, popular primarily because of its differentness.

Huh, I generally use it because it has really good ORM and migrations, and I really like the syntax (coming from Objective-C it's pretty slick. I also used the PHP language when I was starting out, but one day it tried to insist that $myArr[0] and $myArr["0"] actually pointed to the same object, and I have refused to deal with it ever since; I also got tired of typing str_sub_case_insensitive_for_real_safe(haystack, needle) -- or is it needle, haystack? And is this one of those prank functions that fails to substitute the value but still returns a value that evals true? Or if I leave out one of those underscores, am I in fact calling a function that behaves almost exactly the same way but fails under difficult-to-reproduce circumstances? Maybe they've fixed this and the other sundry atrocities? Maybe they've stopped trying to make it into Perl, as compiled by a C++ compiler, and tried fashioning it into an actual dynamic language? I know, I know, some people like PHP, but I think arguments for the superiority of PHP over Ruby (or Python or Scala or Lisp or WebObjects or Perl6 or really anything else) are going to rest completely on the skills of the Zend interpreter writers, and almost never on the quality/readability/maintainability of the code, or the ease of the development process. You can write good safe code in PHP, that is true, but it isn't very ergonomic.

You know, RoR is really good at replacing those old Paradox and FMP database systems. I can see how Facebook might prefer PHP, but people trying to replace little inventory/business processes systems generally only need to support a few dozen users, and don't have an army of developers to keep it running. The Universe is big enough to accommodate the utility of Ruby on Rails and the Twitter developer's stupidity.

Re:Should have used PHP. (4, Insightful)

mini me (132455) | more than 5 years ago | (#27495491)

While Facebook uses PHP where Twitter uses Rails, Facebook uses a plethora of languages to make the whole system work. So Twitter really isn't going to Scala any more than Facebook is going to Erlang. Which is the say that they use the best tool for the job, not one tool for every job.

Re:Should have used PHP. (1)

Spaham (634471) | more than 5 years ago | (#27495725)

yeah, reading the article I kept on thinking "but why do they keep on tryin unknown, unused, under-development languages ??".
Then they explain that they only used research oriented languages... Seems like they weren't programmers to begin with...

Re:Should have used PHP. (2, Interesting)

Jane Q. Public (1010737) | more than 5 years ago | (#27495977)

Scala may be unknown, unused, and under-development, but Ruby is over 12 years old and the community that uses it is huge. So it is neither unknown or unused. (It is, however, under development, as most all modern languages are.)

Re:Should have used PHP. (1)

Radhruin (875377) | more than 5 years ago | (#27495863)

This isn't a nail in any coffin. The fact of the matter is that Twitter's implementation sucked, and sucked hard. It was not designed to scale in the least, and its authors had no idea how to build scalable websites, but instead learned as they went along.

Starling, for example, was Twitter's initial solution to their scaling problems. It was a way for them to offload tasks into a queue to be completed by workers. Pretty common use case, and indeed there are plenty of industry standard tools to do the job, and do it quickly. Twitter, however, decided to implement their own queue for reasons unknown, and guess what, it sucked. Starling was slow as molasses until it was almost entirely rewritten (still in Ruby, mind) somewhat recently.

Not to mention that listening to a Twitter developer blame Ruby for Twitters problems and pimping Scala is unsurprising. Not only is it typical for an engineer to blame his tools for his own failure, but Payne also has a vested financial interest in pimping Scala as the next big thing due to his book deal.

Basically, Twitter blaming Ruby is absurd. The problem lies with the developers and their lack of experience building scalable web infrastructure. You put the same developers in front of PHP and I guarantee you they'd have failed just as hard.

Should have used Erlang! (1)

K. S. Kyosuke (729550) | more than 5 years ago | (#27496089)

'Nuff said. Those twitter posts were once described as "internet SMS messages", didn't they? Short messages...phone systems and heavy networking...Ericsson...Erlang...

Stupid (2, Funny)

AKAImBatman (238306) | more than 5 years ago | (#27495287)

They should have just used Java. Wait--

Who gives a shit about twitter? (5, Insightful)

snowraver1 (1052510) | more than 5 years ago | (#27495323)

Seriously.

Re:Who gives a shit about twitter? (2, Interesting)

ajs (35943) | more than 5 years ago | (#27495563)

Well... just to name a few:

  • Anyone watching which companies are growing during the recession
  • Anyone looking at how the latest high-volume services are building infrastructure on the Web
  • Neil Gaiman [twitter.com]
  • Demi Moore [twitter.com]
  • The President of the United States [twitter.com] (though to be fair, his status isn't updated recently
  • Al Gore [twitter.com]
  • John McCain [twitter.com] (currently updating from Hong Kong)
  • John Battelle [battellemedia.com] , whose insights about the search industry are often enlightening

This, of course, does not make Twitter a panacea, but it certainly makes it interesting enough to warrant the occasional Slashdot article.

Re:Who gives a shit about twitter? (2, Insightful)

nothing2seehere (1496253) | more than 5 years ago | (#27495739)

Wow, Demi Moore? Really? Now that's an influential user!

Re:Who gives a shit about twitter? (1)

ajs (35943) | more than 5 years ago | (#27495783)

Wow, Demi Moore? Really? Now that's an influential user!

Not really, but given that I was trying to give a sense of the breadth of the people interested as users, commentators and misc. interested parties, I think she fits in the list just fine.

Re:Who gives a shit about twitter? (1)

Dynedain (141758) | more than 5 years ago | (#27495985)

Having a marketing team that makes tweets for you is completely different than being a twitter user yourself.

Of the people you listed, how many subscribe to twitter feeds?

Re:Who gives a shit about twitter? (1)

Jane Q. Public (1010737) | more than 5 years ago | (#27496023)

The only one in that whole list I am interested in is Neil Gaiman.

Re:Who gives a shit about twitter? (1)

snowraver1 (1052510) | more than 5 years ago | (#27496053)

Anyone watching which companies are growing during the recession
... Should get their info from a more complete source.
Anyone looking at how the latest high-volume services are building infrastructure on the Web
.. I'll give you that
Neil Gaiman [twitter.com]
Who? (Turns out to be some author) **Update! I just wrote another page!
Demi Moore [twitter.com]
Who cares?
The President of the United States [twitter.com] (though to be fair, his status isn't updated recently
I would prefer something with more substance...
Al Gore [twitter.com]
Who cares?
John McCain [twitter.com] (currently updating from Hong Kong)
Who cares?
John Battelle [battellemedia.com], whose insights about the search industry are often enlightening
You linked to his site (which actually has content) not to twitter.

Twitter is garbage. Our elected officials are dicking around on their smartphones posting gossip to twitter instead of listening. I live in Canada, and our Members of Parliment have been told repeatedly to turn off their phones while in Parliment, but some refuse to do that and waste time and money posting -- quite literally -- gossip to twitter. Fuck Twitter.

Re:Who gives a shit about twitter? (1)

Hijacked Public (999535) | more than 5 years ago | (#27496335)

Demi Moore [twitter.com]
Who cares?

An absolute fuckton shitload of people?

Re:Who gives a shit about twitter? (0, Flamebait)

FredFredrickson (1177871) | more than 5 years ago | (#27495609)

Seriously. They've got heavy traffic on a single line comments website! It's pretty much the "hello world" app of web applications (compared to facebook, or, heck, anything google does). Why are we discussing it? It's amateur crap at best, and nobody cares about its performance with its fad language, because it doesn't actually accomplish anything. In fact, it accomplishes in its main goal - what is mostly a side feature of any other site or blog - short comments - rather unspectacularly.

Re:Who gives a shit about twitter? (2, Funny)

FredFredrickson (1177871) | more than 5 years ago | (#27495757)

Wow I just realized what an angry old man I must sound like there. I didn't mean to be that harsh. I'm just don't like twitter, and spending time speaking about its technical *wonders* seems to be a waste of time, since I assume it's about 3 lines of code (Or should be), and mimics most highschool coding class students' first projects.

Wow, there I go again. So angry. I just quit caffeine, you must understand!

Re:Who gives a shit about twitter? (1)

Jane Q. Public (1010737) | more than 5 years ago | (#27496077)

Nothing personal intended here, but not only do you sound like an angry old man, but calling Ruby a "fad language" makes you sound like an ignorant, angry old man. Ruby has been around for over 12 years and is still one of the fastest growing languages around.

Re:Who gives a shit about twitter? (1)

adwarf (1002867) | more than 5 years ago | (#27496087)

You forgot the obligatory, "get off my lawn."

Re:Who gives a shit about twitter? (1)

Serious Callers Only (1022605) | more than 5 years ago | (#27496417)

since I assume it's about 3 lines of code (Or should be), and mimics most highschool coding class students' first projects.

That'll work for 100 users, let us know how it works out for 10,000, or 8 million.

I can't comment on the technical wonders (or otherwise) of the twitter code-base, but your comment really just illustrates your ignorance on the topic of programming languages and messaging. The problem of dealing with that many users sharing messages is not trivial, and has very little to do with the choice of programming language.

Re:Who gives a shit about twitter? (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#27496213)

you are a dumb cunt so stfu
oh and ur mom licked my asshole and it felt great

Scala seems to be Java+/- (1)

ajs (35943) | more than 5 years ago | (#27495325)

Scala looks and feels like Java with a tiny bit of Python thrown in for good measure. I'm really not certain why anyone would use it over, say, groovy or just plain Java.

The really odd part is trying to imagine Scala as a reasonable replacement for Ruby or any other higher level language.

Re:Scala seems to be Java+/- (3, Insightful)

geekoid (135745) | more than 5 years ago | (#27495421)

It's new! It's hot! it's what all the kids are doing! It's what crappy programmers can pretend to adapt to instead of writing solid code!

weeee!!!

Re:Scala seems to be Java+/- (2, Insightful)

hedleyroos (817147) | more than 5 years ago | (#27495547)

I'm confused - how does a language "scale"? Can you suddenly have 1 billion items in your array instead of 100 million? If that is indeed the case then like the parent said: it is crappy coding and/or design.

I'm fairly confident you can do twitter in whatever language you want and if designed properly it will scale in the proper sense of the word.

Re:Scala seems to be Java+/- (1)

A.K.A_Magnet (860822) | more than 5 years ago | (#27495795)

A language scales if the language itself (not the "runtime" or whatever) can grow with user constructs to suit the ever-changing needs. For instance, in Python, if you write
l = [a]

you will end up with a Python built-in list. Squared-brackets are part of the language syntax and will always refer to Python builtin lists.

In Scala, which is meant to be a scalable language (as the name implies: SCAlable LAnguage), if you write
val l = a :: Nil

you will end-up with a Scala list only because the ::' "keyword" (which in fact isn't) is in the scope, but you could specify another class if you'd like. Use another implementation, yours if you'd like.

All of Scala is made this way and that's why the *language itself* is scalable.

Now, the article talks about scalable architectures, not scalable languages -- they use Scala because they like it :). Yet since Scala compiles to Java bytecode, it offers all the JVM's scability features (clustering with terracotta, ...) and it's a great language to write scalable software in :).

Re:Scala seems to be Java+/- (2, Informative)

Bruce Perens (3872) | more than 5 years ago | (#27496279)

This is the same for Ruby. You can create a list, or you can create some object you wrote in hand-optimized C which just responds to the same operations as a list.

Re:Scala seems to be Java+/- (1)

AKAImBatman (238306) | more than 5 years ago | (#27495933)

I'm confused - how does a language "scale"?

"Scalability" is a multi-faceted term. Most people think of "vertical scaling of the servers" when they hear the term "scalable". Which is to say that the code can handle a higher transaction load on a beefier server.

But there is quite a bit more to scalability than that. There's horizontal scalability of the servers. i.e. Does the software support plugging more boxes in to handle a greater load? Then there's the development scalability. i.e. Are the concerns of the code separated well enough to where multiple development teams can work in parallel without stepping on each other's toes or creating conflicts over code modules? Finally there's organizational scalability. i.e. Is this system designed to allow the organization to effectively split back-end responsibilities as the application grows. (An example of this would be scaling to meet the demands of a large sales force or distributed fulfillment process.)

The language you chose can have an impact on these larger concerns. Various features of the language can support aspects of these needs more naturally than other languages. But at the end of the day, it's still up to the architect to design a system that meets the needs of organization. The language is merely a small cog in the much larger machination of solving the challenges that face the business.

Re:Scala seems to be Java+/- (4, Insightful)

A.K.A_Magnet (860822) | more than 5 years ago | (#27495615)

I read between the lines that you call C or C++ solid-code, and if I'm not mistaken, you will find that the kids are doing Scala because the code is more solid. Scala benefits from a typing system close to OCaml's which makes Scala code very, very solid -- especially if you keep away Java specifics (such as nullable objects) in your code and take special care when interacting with Java libs that may do so.

If I'm mistaken and you're not talking about C/C++, I hope you are not talking about dynamic languages which offer no guarantee whatsoever; you know as a developer I enjoy actually spending my time on working on the business side of my application -- and how to make it scalable, rather than working on low-level specifics and on testing if every pointer is null before dereferencing them. A type system that does this for me (which Scala or ML's parametrized type Option allows) is a bliss.

Now, I'm not enumerate every language under the sun to see what code you call solid, I guess your answer would be that the code is solid whatever the language it's written in. In the end, it all comes down to binary instructions, right? The question is: how many guarantees do the tools give you? In the case of Scala's compiler, it gives you a lot AND offers you a very enjoyable, lightweight yet powerful syntax.

Re:Scala seems to be Java+/- (0)

Anonymous Coward | more than 5 years ago | (#27496145)

It's new! It's hot! it's what all the kids are doing! It's what crappy programmers can pretend to adapt to instead of writing solid code!

And it's what will get those crappy programmers hired into higher-paying, better-connected, better-located, more trend-setting jobs than we programmers who actually know what we're doing could ever hope to find... *grumble*

Re:Scala seems to be Java+/- (2, Insightful)

Anonymous Coward | more than 5 years ago | (#27495425)

Scala compiles to Java Bytecode and runs on an unmodified Java Virtual Machine. It can call any Java Library and you can write code in Scala that is callable from Java.

Scala offers much more than simple syntactic sugar. Take a look at their documentation. Especially the nice support for embedded DSLs is VERY interesting.

Re:Scala seems to be Java+/- (0)

Anonymous Coward | more than 5 years ago | (#27495477)

So does Jython. So... yeah, whats the point?

Re:Scala seems to be Java+/- (1)

Jane Q. Public (1010737) | more than 5 years ago | (#27496109)

Other languages like Ruby and Python also support DSLs just fine. That isn't the reason Twitter wanted Scala.

Re:Scala seems to be Java+/- (1)

nicolas.kassis (875270) | more than 5 years ago | (#27495427)

Or just jruby which has proven itself to be quite able to handle under load. From what I understand they had issues with their Message Queue which could have benefited from the native threads in jruby. But we are all speculating on what really is going on with Twitter.

And what does scala have over say erlang for concurrency and performance?

Re:Scala seems to be Java+/- (1)

A.K.A_Magnet (860822) | more than 5 years ago | (#27495445)

What? Scala is Java mixed with OCaml -- you get an extremly powerful typing system, but it feels like a "dynamic language" such as Python or Ruby. With the performance of Java. Under the hood, it's a brand-new language, very different from all those: it merges functional and object-oriented programming. Yet, for the regular programmer, it feels like Ruby... until he gets used to more powerful features and learns how to designs more complex libraries as embedded DSLs. All that while running on the JVM and thus giving free access to all Java/JVM libs out there with no overhead. The really odd part is that not more people are using Scala as a replacement as most languages -- except C for device level and Erlang for distributed stuff.

Re:Scala seems to be Java+/- (0)

Anonymous Coward | more than 5 years ago | (#27495729)

Yeah "more complex", thats what we need.

Re:Scala seems to be Java+/- (1)

halll7 (1009535) | more than 5 years ago | (#27495965)

Something that gets attention from Java developers is Scala's implementation of the Actor concurrency model [wikipedia.org] , which is a fundamental alternative to the shared-state locking that Java provides. Erlang has offered this for many years, but Scala offers it with a more familiar syntax, and is likely to get a more enthusiastic uptake for that reason alone.

As well as this Scala is functional (first-class functions, currying etc) as well as OO, and Scala programs can use the standard Java libraries if they wish, which again makes the transition easier for Java developers.

Re:Scala seems to be Java+/- (1)

nuttycom (1016165) | more than 5 years ago | (#27495993)

The combination of static typing, type inference and concise syntax for higher-order functions give you a lot that Groovy and Java just fall short on.

In Java's case, it's the simple fact that you can't reasonably implement HOF's like filter, map, etc. without using anonymous inner classes. If you use anonymous inner classes, you end up with a ratio of like 8:1 of boilerplate:relvant expressions. Believe me, I've tried it. If you want to see just how baroque it gets, take a look at functionaljava.org [functionaljava.org] .

Groovy has HOF, but it's interpreted and it doesn't have real static types. So it's slower and you don't have nearly as much ability to do compile-time type checking.

Scala rocks. I've been using it for several months now building a $50M/year transaction processing system and I never want to go back to writing regular old Java again.

Re:Scala seems to be Java+/- (1)

Mike Buddha (10734) | more than 5 years ago | (#27496395)

The really odd part is trying to imagine Scala as a reasonable replacement for Ruby or any other higher level language.

Particularly with options like Stackless Python [http] available. Python is well supported. Why throw out the baby with the bathwater?

Proving that.. (2, Insightful)

hexghost (444585) | more than 5 years ago | (#27495327)

Twitter's developers care more about being cool and hip and using the latest tool so that they remain popular, than they do about having a site that stays up 7 days a week.

Re:Proving that.. (3, Insightful)

AKAImBatman (238306) | more than 5 years ago | (#27495367)

Twitter's developers care more about being cool and hip

Not to be too pedantic, but doesn't that sum up Twitter as a whole?

Re:Proving that.. (1)

FredFredrickson (1177871) | more than 5 years ago | (#27495623)

Yeah, I mean, don't let them get too caught up with features or anything. Maybe some day, they'll add some features, but right now, they're just selling cool.

Re:Proving that.. (3, Insightful)

joe_bruin (266648) | more than 5 years ago | (#27495545)

Twitter's developers care more about being cool and hip and using the latest tool so that they remain popular, than they do about having a site that stays up 7 days a week.

Exactly. Scalability problems arise from poor implementation, not from language choices. Scalable platforms have been implemented in the past with PHP, ASP, Perl, C, Java, and I'm sure with Ruby, Python, or your favorite new language. Twitter is a massive-scale site, they should be looking at deep engineering, not a buzzword platform that promises easy scalability for dummies.

Scala may help them alleviate problems they've hit in the Rails framework. What will help them with the problems they hit in Scala?

Mod Up (0, Redundant)

Jane Q. Public (1010737) | more than 5 years ago | (#27496171)

The problems they had were not due to Rails. The problems arose from poor implementation of their message queue... some bad engineering decisions.

http://unlimitednovelty.com/2009/04/twitter-blaming-ruby-for-their-mistakes.html [unlimitednovelty.com]

And some of their recent Scala noise might be due to the fact that Alex Payne just wrote a book about Scala that was just released, or is about to be.

Re:Proving that.. (1)

Algan (20532) | more than 5 years ago | (#27496329)

What will help them with the problems they hit in Scala?

Erlang :)

Seriously speaking, when you cannot have long lived processes due to memory leaks, you're having a language/platform problem. When you only have green threads, you're having a language/platform problem. Can you architect around it? Probably, but it may be sub-optimal, and bottom line is, why bother when there are better tools available?

Re:Proving that.. (1)

go_epsilon_go (802226) | more than 5 years ago | (#27495875)

Amen to that. Instead on focusing on the latest and greatest fad, new developers should try to do the following: Learn the craft, not only the tools. A little of algorithmic complexity and data structures won't kill you. Throw a little math for good measure. Abstraction and modularization is the key for good architectures, it does not matter if you prefer procedural, functional or object oriented techniques. Just my 2 cents.

Useful! (5, Insightful)

castorvx (1424163) | more than 5 years ago | (#27495331)

Twitter using new-and-fancy programming languages has a way of load testing them for all of us.

I'm not sure I'd have the balls to take a 5 year old development platform/framework and drop it into something that sees so much traffic. Hopefully they share their experiences in some form.

Re:Useful! (1)

Captain Spam (66120) | more than 5 years ago | (#27496381)

Hopefully they share their experiences in some form.

If downtime and lag each count as "some form", I think we'll have us plenty of data to observe!

Good thinking, (3, Funny)

geekoid (135745) | more than 5 years ago | (#27495403)

replace one language that wasn't tested on that scale and replace it with another one that wasn't tested on that scale.

Good thinking~

Oh look, twitter is down..again.

Re:Good thinking, (2, Insightful)

AKAImBatman (238306) | more than 5 years ago | (#27495475)

As the joke I posted above insinuates, Scala runs on top of the Java platform. And unlike Ruby, it focuses on the use of the platform's features. So the platform is more than tested enough. Why they feel the need to use Scala rather than straight-up Java is one of life's great mysteries. But for now, their platform should be fine.

Whether the code they write is scalable and holds up under loads or not is an entirely different topic.

Re:Good thinking, (3, Interesting)

The Slashdolt (518657) | more than 5 years ago | (#27496129)

Read this and all will become clear:
Event-Based Programming without Inversion of Control [lamp.epfl.ch]

Re:Good thinking, (0)

Anonymous Coward | more than 5 years ago | (#27496189)

Summary: Blah, blah, blah, nothing to do with the subject at hand, blah, blah, blah.

Re:Good thinking, (1)

The Slashdolt (518657) | more than 5 years ago | (#27496325)

From the Article:
Scala is different from other concurrent languages in that it contains no language support for concurrency beyond the standard thread model offered by the host environment. Instead of specialized language constructs we rely on Scala's general abstraction capabilities to define higher-level concurrency models. In such a way, we were able to dene all essential operations of Erlang's actor-based process model in the Scala library.

However, since Scala is implemented on the Java VM, we inherited some of the deciencies of the host environment when it comes to concurrency, namely low maximum number of threads and high context-switch overhead. In this paper we have shown how to turn this weakness into a strength. By dening a new event-based model for actors, we could increase dramatically their efficiency and scalability. At the same time, we kept to a large extent the programming model of thread-based actors, which would not have been possible if we had switched to a traditional event-based architecture, because the latter causes an inversion of control.

Re:Good thinking, (4, Interesting)

burris (122191) | more than 5 years ago | (#27496293)

Maybe they use Scala because writing Java code is painful by comparison. Tons of boilerplate, every exception has to be caught in every scope, no pattern matching, no named arguments, and on and on. For people like me, without Scala the JVM wouldn't even be under consideration, though I admit that Java has been more usable since it got generics.

Re:Good thinking, (0)

Anonymous Coward | more than 5 years ago | (#27496367)

Whether the code they write is scalable and holds up under loads or not is an entirely different topic.

Duh. Of course it's Scalable. It says so, right there in the title!

Thank you, I'll be here all week, posting on Slashdot because I don't have a job!

Re:Good thinking, (0)

Anonymous Coward | more than 5 years ago | (#27496419)

I've seem some Scala benchmarks that showed it to be faster than even java in some occasions.

One things is sure, It does not have the overhead of Jruby or groovy.

Obligatory... (1)

dex.pdx (923011) | more than 5 years ago | (#27495523)

Perl would be better.

p.s. I don't want to here any of this Perl is ugly crap either - it's not my fault I don't need my language to -make- me write readable code. A good Perl programmers code is readable anyways.

Re:Obligatory... (1)

ttfkam (37064) | more than 5 years ago | (#27496103)

p.s. I don't want to here any of this Perl is ugly crap either

How can we trust you with Perl when we can't even trust you with English?

Re:Obligatory... (0)

Anonymous Coward | more than 5 years ago | (#27496219)

Too bad there are no good perl programmers~

Scala? Not for me! (2, Funny)

bogaboga (793279) | more than 5 years ago | (#27495553)

Scala is not for me I can confidently say. I am too old to learn a new [programming] language. The languages I know will suffice for now.

There is a saying too: "You cannot teach old dogs new tricks."

Re:Scala? Not for me! (4, Funny)

Anonymous Coward | more than 5 years ago | (#27495763)

You know, at first I thought this post was off-topic, because it's an incredibly pointless life update that none of us could possibly care about.

Then I realized that the story is about Twitter, and suddenly I think it's the most relevant post so far.

Re:Scala? Not for me! (1)

nothing2seehere (1496253) | more than 5 years ago | (#27495823)

I am too old to learn a new [programming] language. The languages I know will suffice for now.

Understood - and incidentally, I would like fries with that, Grandpa.

Re:Scala? Not for me! (1)

proton (56759) | more than 5 years ago | (#27496373)

There is a saying too: "You cannot teach old dogs new tricks."

And in the Mythbuster way, thats totally busted! (BOOM)

As for the language, use whatever floats your boat. It could be total crap and still work (like PHP for me).

Aw, Java and Python had a baby! (5, Funny)

gpig (244284) | more than 5 years ago | (#27495651)

Isn't she cute :)

Re:Aw, Java and Python had a baby! (0)

Anonymous Coward | more than 5 years ago | (#27496205)

Hopefully she isn't as slow as one of her parents. /cough Java.

Scala is great (5, Interesting)

burris (122191) | more than 5 years ago | (#27495681)

If I want to use any Java software then I'll use Scala. I see people bashing Scala, saying the languages they know are good enough or they can just use jython/jruby/groovy, but they clearly know little about Scala.

One thing that's nice about Scala that Java, Jython, JRuby, and Groovy all lack is it's powerful type system and pattern matching. Once you get used to good pattern matching like in Scala, SML, OCaml, or Haskell you won't want to go back. Plus you get all the benefits of running on the JVM at high speed (unlike all the aforementioned JVM languages, except Java itself.)

Honestly, you should check out Scala before you bash it. It's a very good choice wherever you might choose Java, which is a good choice for the back end. Twitter's developers are smart and experienced. They didn't choose Scala just to be cool. It is a powerful tool that can get the job done in an elegant way.

Re:Scala is great (1)

AbbyNormal (216235) | more than 5 years ago | (#27495881)

Why wouldn't Groovy be a suitable replacement? (Curious about Scala).

Re:Scala is great (1)

burris (122191) | more than 5 years ago | (#27496179)

Groovy is very slow [debian.org] . Maybe that will get better in Java 7. There are probably other meaningful differences that I would be able to point out if I were better versed in Groovy.

Re:Scala is great (0)

Anonymous Coward | more than 5 years ago | (#27496295)

Why wouldn't Groovy be a suitable replacement? (Curious about Scala).

Groovy is dynamically typed and slow as python or ruby.

Mod down (5, Funny)

Anonymous Coward | more than 5 years ago | (#27495721)

OP is just a twitter sock puppet.

more like (1)

toby (759) | more than 5 years ago | (#27495737)

Java and Erlang did. Python isn't particularly 'functional' despite some recent syntactic grafts.

Here is an interesting discussion on alternatives (5, Informative)

tieTYT (989034) | more than 5 years ago | (#27495915)

http://unlimitednovelty.com/2009/04/twitter-blaming-ruby-for-their-mistakes.html [unlimitednovelty.com]

This blog post takes the attitude that Twitter didn't move to Scala because ROR had a problem, but because the in-house messaging system Twitter created performed poorly. The author does not work at Twitter but many of the Twitter developers (including Alex Payne) respond in the comments. I found the article to be very interesting and the comments even more so. They give a sense of how much research Twitter did before this change.

Twitter on Scalia! (1)

Steve1952 (651150) | more than 5 years ago | (#27496321)

At first I read the title as "Twitter on Scalia". Justice Scalia is one of the most conservative US Supreme Court justices, and I'm not sure that this would have been a happy combination.

I hope it's at least fast. (1)

rduke15 (721841) | more than 5 years ago | (#27496359)

Well, at a first glance, it looks just as rigid and boring as Java.

I guess that can be considered Falmebait?

Anyone remember this quote?

[..] but it's certainly the case that over the last twenty years or so, many Computer Scientists have come out in opposition to the Art of Programming. In trying to make programming predictable, they've mostly succeeded in making it boring. And in so doing, they've lost sight of the idea that programming is a human pursuit. They've designed languages intended more to keep the computer happy than the programmer. Was any SQL programmer ever happy about having to declare a value to be varchar(255)? Oops, now it's a key, and can't be longer than 60. Who comes up with these numbers?

Cost of developers vs. cost of hardware (1)

janwedekind (778872) | more than 5 years ago | (#27496403)

From The Register [theregister.co.uk] article:

"Investors now want to know that you're not going to be paying tons and tons of money for servers because you decided to build in a stack that isn't as efficient as possible," he (Alex Payne) said.

I think in the beginning it matters a lot what language you use. If the scale of your Web business is small, the salaries of your staff will be the biggest item on the list of expenses. I guess now they'll have quite a number of computers and having to buy 20% more servers will cost them quite a bit. If they are saying now that Scala is good for them, it does not mean anything. At this point it probably would be profitable for them to port a large part of their software to COBOL. A more forward-looking approach would be to improve the Ruby VM. But maybe they need somebody else to do it for them.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?