Comments on: Ruby vs JRuby Fractal Benchmark

By: Anonymous

Anonymous — Thu, 05 Jul 2007 21:34:00 +0000

Might it be a good idea to give a lot of credit for the performance to the Java virtual machine.
I’m seeing “Ruby Virtual Machine”, but I’d like to see the actual JVM be given some cudos. Nothing wrong with taking some pride in all this hard work…

By: igouy

igouy — Sun, 10 Jun 2007 16:47:00 +0000

Julian Doherty wrote What is really interesting is that JRuby is faster in this case when...

and when we ignore jvm startup by measuring elapsed time within the program...

What is really interesting is that performance measurements come bundled with implicit usage scenarios which may or may not be valid.

(Those JRuby guys are doing great work!)

By: Charles Oliver Nutter

Charles Oliver Nutter — Thu, 07 Jun 2007 17:03:00 +0000

Java benefits from using actual primitive values, so that’s not quite a fair comparison to Ruby’s everything-is-an-object. But yes, we have a lot more work to do. There may even be a chance we can use primitives in certain situations.

By: A.A.A

A.A.A — Thu, 07 Jun 2007 13:17:00 +0000

Just for reference, in Java (1.5 u9) it takes 0.07 seconds in my machine after the second iteration. which is around 100 times faster i think.. well, sure it is a micro benchmark .

By: Julian Doherty

Julian Doherty — Wed, 06 Jun 2007 21:51:00 +0000

Excellent info Charles. I’ve included your 2nd comment in the main post.

By: Anonymous

Anonymous — Wed, 06 Jun 2007 19:44:00 +0000

um, holy cow? JRuby has practically zero performance penalty?

By: Charles Oliver Nutter

Charles Oliver Nutter — Wed, 06 Jun 2007 15:24:00 +0000

FYI, I grabbed the code in question to try it out myself. These are my numbers using JRuby trunk (not really different from RC3), Java 6 server VM on OS X, and the following command line:

jruby -J-server -J-Djruby.jit.threshold=0 -O fractal.rb

JRuby:
Ruby Elapsed 6.454000

Ruby 1.8.6:
Ruby Elapsed 6.971203

The command line above adds three options:

-J-server turns on the server VM

-J-Djruby.jit.threshold=0 sets the JIT threshold to zero, so it will try to compile methods immediately (critically important for this benchmark since the methods are only called once), and -O turns off ObjectSpace, which is pure overhead for us.

Additionally, if I let the benchmark run ten times, allowing the JVM to optimize it further, I get the following result for JRuby on the tenth iteration:

Ruby Elapsed 5.629000

There's a good page in the JRuby Wiki for performance tuning JRuby. The reason this wasn't compiling is twofold: the methods weren't called enough times to trigger the JIT, and the script itself had a "class" definition in it, which the compiler doesn't yet handle.

By: Charles Oliver Nutter

Charles Oliver Nutter — Wed, 06 Jun 2007 15:05:00 +0000

I’d guess the code in question simply isn’t compiling in JRuby. JRuby only compiles about 50% of methods out there, and a substantially smaller percentage of actual code (since small methods usually compile and large ones usually don’t). We haven’t focused on completing the compiler for 1.0, but after the release we’ll hit it a lot harder. We’ve got a lot of planned optimizations coming.

You might get a boost if you make sure you’re running under Java 6 with the server VM…but if it’s this slow for you right now there’s probably some bottleneck in JRuby artificially slowing it down.