Hacker News Reader

Comment by ninjin | Hacker News Reader

I really want to get into Julia. As I have been using an increasing amount of numerical code NumPy, SciPy and even my own Cython code has started to fail me performance-wise. The Julia community is really friendly and I tried rolled out a few implementations of the inner-most loop of a Recursive Neural Net in a variety of languages.

https://github.com/ninjin/ppod/tree/master/hck/rnn_perf

Disappointingly enough I was unable to push the performance of Julia beyond that of pure (and ugly) Matlab. I tried asking over at #julia, but it seems that my implementation should be reasonably canonical. This is really a pity for me since I would love to remain highly productive in terms of code and use something scripting-esque rather than have to re-consider going back to C. I have also had a look at Nimrod, but the scientific library support appears somewhat lacking at the moment.

View on HN

I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.

Thanks a ton Iain, it pushed it down by 60% on my end as well. About the C code, I used -O2, the specific compiler flags are in the rnn_perf.sh file.

https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e9861...

Julia is really tempting for my next project, the benefits of scripting without all the OOP cruft and a community that just shines. Also, did I mention the awesome meta programming?

Having people like Iain – with this kind of knowledge of numerical programming and optimization – help you out with your code is one of the major benefits of Julia programming :-)

This is some quality advertising on your and Iain's part. It makes me want to use Julia even more when I get the chance.

Wow. If this is what the community in Julia is like, count me in.

I've been focusing on mastering the scientific libraries of Python and waiting until I hit performance walls before I took Julia on, but perhaps I'll start now.

I also starting working on a neural network implementation for my first foray into Julia:

https://github.com/bachase/nnadl-julia.

The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh.

As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window.

On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.

I was curious to know how modern C++ would compare. Here is a version (not 100% sure about the result because I was not 100% sure of the meaning of some of the numpy constructs when applied to matrixes).

http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc

It seems pretty fast!

> my own Cython code has started to fail me performance-wise.

I think this is an odd statement, seeing as how Cython allows you to write pure C code without making any use of the Python C API, so by definition Cython can be as efficient as C minus a single function call from Python to C.

I think your benchmark is way too small to be interesting. You're only testing raw numerical performance, and of course C wins there. Any of these languages allow you to wrap a C library with more or less convenience. I think the more interesting questions are how well these languages allow you to build larger applications with performance-sensitive components, how extensive the ecosystem of libraries is, and the quality of debugging and profiling facilities.

Just curious, is it possible to vectorize your MATLAB implementation? In your test code you don't use the loop index in the calcs, so that makes it hard for me to tell if it is or is not possible to vectorize it for your actual application.

Reply on news.ycombinator.com