Comparison of Nine Ruby Implementations

An anonymous reader writes "Zen and the Art of Programming published a new version of The Great Ruby Shootout, which was aimed at testing the performances of multiple Ruby implementations. On the benchmark table this time around are Ruby 1.8 (on GNU/Linux and Windows), Ruby 1.9 (aka Yarv), Ruby Enterprise Edition (aka REE), JRuby 1.1.6RC1, Rubinius, MagLev, MacRuby 0.3 and IronRuby. The results of this comprehensive comparison show that for this set of benchmarks, Ruby 1.9.1 is almost 5 times faster than the notoriously slow Ruby 1.8. Is Ruby finally going to be acceptably fast?"

Re:

[sveard]

That's the worst cockney accent I've ever heard in my life.

Re:

[Anonymous Coward]

You should buy a screen reader with cockney support then, mate. Personally, I use the CockneyCobblers3000 Deluxe, it even has support for rhyming slang:

I'll get the jam jar round and we'll get daaaann the rub-a-dub, for a few Uri Gellas.

Comes out, read like this [youtube.com], as:

I will drive the car around to the front entrance. You shall get in, and we will go to the pub. Once there we will purchase and drink at least two pints of Stella Artois lager.

Ruby 1.9.1 and JRuby

[mgkimsal2]

Ruby 1.9.1 and JRuby seem to be the 'winners' in most tests. Have a look at the PNG here: http://antoniocangiano.com/images/shootout3/main_time.png

I've heard a lot of good things about the JRuby project, and this test seems to demonstrate that it's probably the closest in terms of speed to the standard Ruby (canonical Ruby?)

Re:Ruby 1.9.1 and JRuby

[Foofoobar]

Well JRuby is an implementation of Ruby in Java which explains the speed but it is still quite slow as it has to 'interpret' everything into java. If you are a fan of Java and like Ruby, I'd suggest Groovy. It's blazingly fast and even puts these to shame. Plus works well Rails, Struts and most MVC frameworks.

Re:

[mgkimsal2]

See my signature - I'm involved in the Groovy world (via GroovyMag [groovymag.com]) already. :) I do try to keep tabs on what's going on in other JVM languages, especially JRuby.

Re:Ruby 1.9.1 and JRuby

[Foofoobar]

Um... I suffer from a rare disease known as sig blindness. :)

Re:

[goombah99]

groovy seems like the ideal language for me. Or at least that's my superficial impression. But it seems like it can seemlessly gain speed by implementing the slow bits into JAVA with very little pain. Or am I mistaken.

I can't actually make any use of groovy however till it catches up with scipy and mayavi.mlab in python. When is that going to happen?

Ultimately what I want is an interpreted language that can be compiled. So for example, in python one rarely actually uses the introspective ability to modi

Re:

[Foofoobar]

Well I just played with it a little myself too and it is GREAT for frontend development. You could use it for ALL development if you want but it gives you ('the developer') the option of mixing 'pure java' on the backend with Groovy if you wish (since Groovy compiles to Java anyway from what I remember).

Very flexible, and very powerful. None of that worry of not scaling that there is with Ruby.

Re:Ruby 1.9.1 and JRuby

[K. S. Kyosuke]

"Ultimately what I want is an interpreted language that can be compiled. So for example, in python one rarely actually uses the introspective ability to modify ones self, or even takes advantage of duck typing. instead one usually calls functions with the same type arguments and so forth. So if one just had the ability to switch off the dynamic typing and self-modifying capabilities so that one could compile it it sure would be one sweet language."

I felt a great disturbance in the Force, as if millions of Common Lisp hackers suddenly cried out in terror and were suddenly silenced...

Re:

[johanatan]

Haskell too!

Re:

[Sciryl Llort]

# Oh ruby ruby ruby ruby rubeeeey
    I think you're groovy groovy groovy groovy grooveeeey /#

Re:

[chthon]

Use Common Lisp, CLISP and SBCL can both compile to native executables, while having the best methods available for interactive testing.

Re:

[DragonWriter]

Well JRuby is an implementation of Ruby in Java which explains the speed but it is still quite slow as it has to 'interpret' everything into java.

JRuby can do either JIT or ahead-of-time compilation to JVM bytecode, it does not 'interpret' (or even 'compile') everything to Java.

If you are a fan of Java and like Ruby, I'd suggest Groovy. It's blazingly fast and even puts these to shame. Plus works well Rails, Struts and most MVC frameworks.

Well, Rails doesn't run in Groovy, though Groovy has a Rails-inspired

Re:

[Foofoobar]

Looks to me like Groovy outperformed JRuby in many ways. I don't see what you are talking about. And it's still only in BETA!

Re:

[rogerdpack]

Seems that grails isn't "tons" faster than rails--see http://docs.codehaus.org/display/GRAILS/Grails+vs+Rails+Benchmark [codehaus.org] etc. -=R

JRuby is the clear winner

[0xABADC0DA]

Look at how many 0.002 and similar instantaneous results there are. Tests were run 5 times and the best result used, so there probably wasn't even a single garbage collect that happened in many of the tests. You can't really say from such short benchmarks, but those tests probably did not include Java's strength of hotspot optimization over time.

There were a couple tests that were much slower in JRuby, but this is countered by the test that failed on Ruby 1.9.1; fast means nothing when you don't finish.

Re:

[JesseMcDonald]

There were a couple tests that were much slower in JRuby, but this is countered by the test that failed on Ruby 1.9.1; fast means nothing when you don't finish.

Ruby 1.9.1 failed two tests; JRuby failed three.

None of the VMs managed to complete all the tests. Other than Rubinius, which timed out frequently, Ruby 1.9.1 had the fewest errors.

Also, the input sizes for many of the tests numbered in the millions. If that's not enough repetition to take advantage of hotspot optimization, then I'd say they were right to exclude it.

Re:

[0xABADC0DA]

Ruby 1.9.1 failed two tests; JRuby failed three.

I stand corrected. I blame it on the results being a png so no "highlight all" search results ;-P

Also, the input sizes for many of the tests numbered in the millions. If that's not enough repetition to take advantage of hotspot optimization, then I'd say they were right to exclude it.

Hotspot tracks how much time was spent in the function among other things. With millions of iterations it may be compiled once, but not recompiled using deeper inlining or frequent-type information. In any case, a benchmark that fast is pretty meaningless for any number of reasons.

Re:

[ESqVIP]

There were a couple tests that were much slower in JRuby, but this is countered by the test that failed on Ruby 1.9.1; fast means nothing when you don't finish.

You should note, though, that one of the failures of YARV was because of code incompatibility (assuming String#[] returns an Integer, which changed in Ruby 1.9). This is an actual worry, because you have to port your code to 1.9 (unlike JRuby, which should be very close to a drop-in solution for 1.8). The other was excessive recursion, something pretty unlikely to find in a "real-world" app.

The JRuby errors were all implementation failures. Pretty minor ones which might be already fixed by now, but still

Re:

[0xABADC0DA]

Let me follow up to answer several replies. Yes, official Ruby 1.9.1 was overall moderately faster than JRuby. JRuby still is the clear winner:

1) JVM has a garbage collector known to work on multi gb sized heap and JVM has a parallel collector that can use multiple processors. So for large programs (programs that actually matter) one would expect JVM to perform better than these microbenchmarks in respect to Ruby 1.9.1.

2) JVM adapts over time to increase the performance of the most-used parts. The bench

No Parrot?

[Ed Avis]

Is there a reason why they did not test Cardinal, the Ruby implementation for Parrot? I know it is not production-ready yet but it would be interesting to see how performance compares.

Re:

[Adam Jorgensen]

That seems to be the case with a lot of Parrot projects...

Re:

[facebook12]

We can use Ruby in Facebook application [facebookster.com].

Platform-based Ruby

[tcopeland]

I heard Rich Kilmer [blogs.com] talk about the various Ruby implementations once; his take on them were that each would be used to leverage the underlying platform. In other words, if you want to use Java libraries, you'll use JRuby. If you want to use the Mac libraries (e.g., via Hot Cocoa [macruby.org]), you'll use MacRuby. And if you want to use C extensions like RMagick and libxml2, you'll use MRI.

I thought that was an interesting way of looking at the various implementations... each one would be appropriate for a different scenario.

Re:

[tcopeland]

> a different version of a language for whatever platform I want.

Hm, the language would be the same, but the libraries that you would have available or leveraging would change.

Re:

[Surt]

You would write code the same in any version (using the Ruby language). But you would run in a different container depending on your platform. This is no different from using a different compiler for C, or a different runtime for Perl. The availability of different libraries on different platforms is universal for basically all languages.

Re:

[moderatorrater]

The availability of different libraries on different platforms is universal for basically all languages.

Of course there will be, because the platform has different underlying features. The problem this presents is that you're adding another layer on top of it. That statement wasn't like saying that a certain perl program will only run on windows (which would still be considered bad practice btw), but on top of that you'd have to have the right version of the three available interpreters for windows to run it. What happens when someone distributes one program that requires jruby and someone else distributes a

Re:

[Surt]

But seriously, it's the same issue no matter where you go. You can write ruby that will run anywhere, and you can write ruby that will run only on a specific interpreter. You can write c that will link anywhere and you can write c that will link only within a specific compiler.

Encouraging people not to fragment is hopeless, an unfragmentable language would be useless ... there are just too many useful libraries out there.

Is Ruby finally going to be acceptably fast?

[bill_kress]

> Is Ruby finally going to be acceptably fast?

Ruby has always been acceptably fast for most values of acceptably.

In cases where it has not been acceptable, it probably will continue not to be since those are cases where better performance is always welcomed (large volume web services).

Re:

[nettdata]

I guess it depends on the web services.

A lot of LinkedIn stuff is in Ruby... over a billion page views a month and growing.

Their Bumper Sticker app alone has huge numbers.

Interesting read here: http://blog.linkedin.com/2008/06/23/web-scalability-practices-bumper-sticker-on-rails/ [linkedin.com]

Re:

[bill_kress]

If it's always been in ruby, then it's always been fast enough, and remains so-just like I said. Not that a little bump wouldn't help, but sometimes I wonder why people sound like they are disagreeing when they are actually supporting.

or did I read your post wrong?

PHP is still faster

[unity100]

sorry, but ruby to php is not what c to assembler was. and never will.

php isnt that hard to work with or time consuming. its not that low level either. its high level enough to work acceptably fast.

i see ruby as another 'hey, that's 2.0 !' fad.

you can try to persuade me to the opposite. noone has been able to yet.

Re:

[unity100]

for large web apps, using higher tools is BAD.

you will get innumerable scaling/modification/adaptation issues in future.

faster as in

[unity100]

faster to build.

Re:

[resonantblue]

"high level" enough is a relative term. For me, PHP is not enough unless you put it together with one of the MVC frameworks like CakePHP or Symfony. In any case, there are a ton of people (including myself) who like Ruby better for a variety of reasons and we don't have problems with it being too slow (though faster speed is always a welcome improvement). Point is: if you want to use PHP, do it. I don't really care to "persuade" you otherwise. But it's always annoying to see such comparison on a thread

its not about 'like'

[unity100]

its about which technology is better.

because, even if we like it or not, we are being pushed stuff that are a pain to work with or problematic for the future, just because of the hype the '2.0' and similar crowds generate. and they are being herded by sources that want to sell certificates, books and make money.

it affects our lives. sooner or later some supervisor or client of yours is gonna shove some hyped tech up your butt, demanding you do stuff with it.

and no, its not about 'bettering yourself

Re:

[resonantblue]

Dude, comon, it's not about books or making money. I am a former PHP programmer who has switched to Rails; and as far as I'm concerned, it *is* the better platform (that said, if you like PHP, I think it's a fine platform too). I not only like the elegance of the Rails framework, but I like many things about Ruby including the easy use of anonymous functions. Now you may discount anonymous functions as syntactic sugar, but it comes in very useful in so many situations to the point where I don't understan

Re:

[unity100]

http://www.killerphp.com/articles/will-ruby-kill-php/ [killerphp.com] http://phplens.com/phpeverywhere/?q=node/view/222 [phplens.com] well. in my eyes, ruby is just another fad pushed forward to make new money. its not only about the language itself, its the widespread usage.

there has been many languages which were touted to be 'better', rightly or wrongly, that faded away.

Re:

[resonantblue]

I'm not sure what you're driving at here because:
a.) I never said Ruby will kill PHP
b.) I didn't say Ruby will appeal to everyone

As for "fads," Ruby and PHP have both been around for about 13 years now. Ruby is not some new kid on the block. Rails came along later and really popularized Ruby in the web world, but Ruby as a language is hardly a fad.

Re:

[Bert64]

Yeah i agree there...
Lots of people seem to be converting existing projects to ruby because "it's the in thing", without bringing any practical benefit to the project, but at the same time replacing perfectly good code with new untested code which runs much slower and consumes more resources...

Compare the performance of metasploit 2 to metasploit 3 for a good example.

acceptably fast?

[jbeyer05]

What does acceptably fast really mean? If I can write code 5x as quickly relative to Java or C, Ruby will always be fast enough for me. Humans are expensive, CPU cycles are cheap.

Re:

[Bert64]

CPU cycles, power, memory, all adds up... And once the load exceeds a single box, you have to get into clustering and load balancing.

How many times is your code going to run? Slower code wastes time each and every time it is executed, so the time you saved writing it could end up being lost a thousand times over as you wait for the code to finish it's task.

In this case, acceptably fast is "as fast or faster, using the same or less memory" than other competing options.

My biggest gripe is when people take a p

Ruby gets faster while Python gets slower?

[Aloisius]

It is a bit annoying that Python 3.0 is about 10% slower than 2.6.

I've heard some sensationalist claims that Ruby 1.9.1 is now significantly faster than Python, but of course they always turn out to be some simple test like tail recursion. At the same time, I've seen some awful performance numbers from things like Rails compared to Django.

Anyone have a rough idea of the performance comparison between the two and what bottlenecks people are likely to hit on either side? I know the GIL in Python is sometimes

Re:

[DragonWriter]

Anyone have a rough idea of the performance comparison between the two and what bottlenecks people are likely to hit on either side? I know the GIL in Python is sometimes blamed for a lot of problems...

Ruby 1.9.1 trades green threads for native threads with a GIL, so probably has similar issues to Python (the plan is to eliminate the GIL eventually.) JRuby, I think, uses Java threads (which, on most platforms, IIRC, are native threads) without a GIL.

Re:

[setagllib]

It's true that JRuby has no overall GIL, but it definitely has plenty of internal fine-grained locks to maintain consistency and determinism. However thanks to Java 6 optimisations, almost all of those locks are completely elided while running a single-threaded program. That's just another way JRuby benefits from JVM technology, and yet another thing a C implementation cannot reasonably implement.

YARV to LLVM

[chrysalis]

I have great hopes in the YARV to LLVM compiler (yarv2llvm), even if you can't read japanese, have a look at the numbers: http://d.hatena.ne.jp/miura1729/20081012/1223785541 [hatena.ne.jp]

What is it with ruby?

[Bert64]

I don't get all this fascination with ruby, and why it currently seems to be so fashionable that people are willing to take perfectly good projects and rewrite them in this new fashionable language...

Take for example metasploit (see metasploit.com), version 2.x was written in perl and was reasonably quick, version 3 is a complete rewrite in ruby and it's now orders of magnitude slower and more memory hungry than the perl version ever was... Trying to script the CLI based version is now pretty much useless b

Re:

[Lord Ender]

It's data-oriented--everything is an object. Your code is just objects and flow control.

Compare that to perl, which is an ugly mess of unix idioms nailed together in a hurry (but polished for years). Even Python requires you to memorize a list of functions to discover the properties of datatypes, rather than just having accessor methods to the data itself (foobar.length, not len(foobar)).

Data-oriented, the way the real world is.

Re:

[k8to]

because len(foo) and foo.len() are soooooooooooo different.

They both make use of a common protocol. Only the purist need care.

Re:

[Lord Ender]

Wrong. No matter what object I'm working with, I can find its length with the .length property if such property is meaningfull. "len()" on the other hand, only works for certain types of objects. You need to memorize different tricks to find lengths of different data types in non-fully-OO languages.

Having a consistent interface to your data really is a wonderful thing. It means no hacks, no lists of utility functions; things just work as you would expect.

Re:

[RAMMS+EIN]

``Having a consistent interface to your data really is a wonderful thing. It means no hacks, no lists of utility functions; things just work as you would expect.''

This is really one of Ruby's greatest strenghts, and why I like it so much. It really feels like it was _designed_, rather than cobbled together.

On the other hand, the object.method notation isn't great for consistency. Invariably, you are going to get methods that don't rightly belong with one of the objects they affect. And then you have to wond

Re:

[poopdeville]

Wrong. No matter what object I'm working with, I can find its length with the .length property if such property is meaningfull. "len()" on the other hand, only works for certain types of objects.

(I don't do Python, but I know my type theory) What exactly do you think a type is in a language like Ruby or Python?

A "duck type" for an object is the type defined by the methods (and their signatures) it responds to. So tell me, if "length" isn't meaningful for an object, why would "len()" be?

What I'm trying to

Re:

[Lord Ender]

The len() function may work for one datatype. You need different utility functions for different data types. But objects with length can all have .length properties.

Everything-Is-An-Object means working with objects, not poking at things with utility functions. It just makes more sense unless you have been indoctrinated in more primitive language types.

Re:

[SoupIsGoodFood_42]

It's not fashion, it's more the work culture of the people who use it. Look at a few Ruby projects and you're more likely to see a project where interface design is taken a bit more seriously as well as following web standards etc. And please, don't give examples to the contrary, as when it come to this sort of thing, it's hardly a certainty and more of a noticeable correlation.

Re:

[nschubach]

I've been reading up on it and use as a quick solution finder. I have already used it for things like file manipulation/creation using regex and simple I/O. Sure, I could code up something in another language, but it's so simple and quick in Ruby.

Re:

[RAMMS+EIN]

I love Ruby, and I completely agree with you. Just because a newly discovered programming language is good, doesn't mean it's a good idea to go and rewrite your perfectly good code in it. It doesn't mean it's a Bad Thing, either, but, if the results are as you state (which I find easy to believe - ruby 1.8 _is_ dog slow), it's hard to argue that it is an improvement.

There is such a thing as the right tool for the job, and Ruby is not (with current implementations) the right tool for things that are CPU inte

Re:

[account_deleted]

Comment removed based on user account deletion

Re:

[bill_mcgonigle]

I don't get all this fascination with ruby, and why it currently seems to be so fashionable that people are willing to take perfectly good projects and rewrite them in this new fashionable language...

It's an excellent language, taking most of the good stuff from Perl and giving it proper object orientation. This lets you develop faster and potentially do better things. When time to market is more important than runtime performance then Merb might make sense.

I'm still writing Perl code until the VM is fast

Who cares?

[Ash Vince]

Ruby is not really a serious programming language to be used in mission critical projects. It is an excellent resource for teaching development best practice due to the patterns it enforces but this comes at a cost. That cost is inflexibility. It frequently restricts thing that are on the borderlines of acceptable practice but in certain very specific circumstances are the best way of achieving a given end.

The only immediate example I can think of in my lunch break is it rigorous enforcement of best pract