Raw Block

After reading about benchmarking various languages generating fractal patterns, I tried throwing the Ruby code at both the standard MRI Ruby and JRuby interpreters to compare relative performance. This is all totally unscientific, but is interesting all the same.

MRI Ruby (Matz Ruby Interpreter)


ruby 1.8.5 (2006-12-25 patchlevel 12) [i686-darwin8.8.1]

Rendering

Ruby Elapsed 6.732136


JRuby


ruby 1.8.5 (2007-06-04 rev 3812) [i386-jruby1.0.0RC3]

Rendering

Ruby Elapsed 68.757000


C Ruby is the clear winner, but that’s not so surprising. What is surprising is how far JRuby was behind - by at least 10 times. I’ve heard good things about JRuby’s ability to compile Ruby down to bytecode, but I didn’t have too much luck with that. Admittedly I didn’t play around with it for long.

I was curious where the bottlenecks are in both versions. After running the code through both VMs with the -r profile flag, really slow floating point operations turned out to be the culprit. The profile for each was pretty much identical. Lots of float operations (+ / < * etc), with everything multiplied by 10 for the JRuby VM.

So what does this tell us? Pretty much nothing. Most apps won’t be doing this kind of CPU heavy crunching (and if you’re using an interpreted language you’re asking for trouble). Especially the kind of apps you’d use either MRI Ruby or JRuby for, which spend most of their time waiting for the database. I might have another play and see if I can get JRuby to run using bytecode - the hype is that it will be as fast if not faster than MRI.

Disclaimer - I’m actually a huge fan of both MRI Ruby and JRuby and just did this out of curiosity. Put AWAY the flame torches now…

Update - JRuby can actually be as fast or faster than MRI

Charles Oliver Nutter (of the JRuby team), has added some suggestions for tuning the JRuby VM. The following is also in the comments, but I’ve included it in the main post as it is useful information that might get missed. What is really interesting is that JRuby is faster in this case when the VM is tuned and the code gets a chance to run enough for the JIT compiler to kick in.

FYI, I grabbed the code in question to try it out myself. These are my numbers using JRuby trunk (not really different from RC3), Java 6 server VM on OS X, and the following command line:

jruby -J-server -J-Djruby.jit.threshold=0 -O fractal.rb

JRuby:

Ruby Elapsed 6.454000

Ruby 1.8.6:

Ruby Elapsed 6.971203

The command line above adds three options:

-J-server turns on the server VM

-J-Djruby.jit.threshold=0 sets the JIT threshold to zero, so it will try to compile methods immediately (critically important for this benchmark since the methods are only called once), and -O turns off ObjectSpace, which is pure overhead for us.

Additionally, if I let the benchmark run ten times, allowing the JVM to optimize it further, I get the following result for JRuby on the tenth iteration:

Ruby Elapsed 5.629000

There’s a good page in the JRuby Wiki for performance tuning JRuby. The reason this wasn’t compiling is twofold: the methods weren’t called enough times to trigger the JIT, and the script itself had a “class” definition in it, which the compiler doesn’t yet handle.

Cheers Charles, extremely useful.