Julia Impressions

I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.

> my own Cython code has started to fail me performance-wise.

I think this is an odd statement, seeing as how Cython allows you to write pure C code without making any use of the Python C API, so by definition Cython can be as efficient as C minus a single function call from Python to C.

I think your benchmark is way too small to be interesting. You're only testing raw numerical performance, and of course C wins there. Any of these languages allow you to wrap a C library with more or less convenience. I think the more interesting questions are how well these languages allow you to build larger applications with performance-sensitive components, how extensive the ecosystem of libraries is, and the quality of debugging and profiling facilities.

I also starting working on a neural network implementation for my first foray into Julia:

https://github.com/bachase/nnadl-julia.

The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh.

As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window.

On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.

Just curious, is it possible to vectorize your MATLAB implementation? In your test code you don't use the loop index in the calcs, so that makes it hard for me to tell if it is or is not possible to vectorize it for your actual application.

I was curious to know how modern C++ would compare. Here is a version (not 100% sure about the result because I was not 100% sure of the meaning of some of the numpy constructs when applied to matrixes).

http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc

It seems pretty fast!

It depends on what you need. DataFrames [1] is a pretty good replacement for R's data frames. Distributions [2] is pretty amazing. Gadfly.jl [3] is a great ggplot-style plotting library that integrates nicely with IJulia [4]. If you have questions about specific things you need to do and want pointers, post on the julia-users list [5] and you should get some good answers.

[1] http://harlanh.github.io/DataFrames.jl/

[2] http://distributionsjl.readthedocs.org/en/latest/

[3] https://github.com/dcjones/Gadfly.jl

[4] https://github.com/JuliaLang/IJulia.jl

[5] https://groups.google.com/forum/#!forum/julia-users

Out of curiosity, why are you so desperate to get away from R? Is it for the speed of a high performance language?

As you mention, the vast selection of libraries (and the continual development of these libraries) make even the python module universe look small (in reference to statistical methods, statistical algorithms/machine learning, plotting).

Further, not only are there a lot of libraries.. but many of them are bleeding edge written by the same people who originally invented the given algorithm. Most similar libraries in other languages (like Python) have incomplete ports of these libraries to them often not written by the same people. Not to mention, the documentation behind these algorithms is often excellent - besides the usual man files, there's "vignettes" (detailed instruction manual) as well as an entire journal that now has become a place to publish papers detailing the usage and implementation of many of these algorithms.

So I'm curious, why leave?

This is one of the misconceptions about Python, that it's "slow". Python has several type annotation systems with JITing capability, and can easily achieve C-speeds (as well as interface easily to C libraries). We've given two tutorials at Supercomputing in the last two years, and the trend for high performance computing in Python is on the rise. Don't count it out!

Assuming that this was fixed by caching the machine code, cant the same approach be used for packages ? Or to turn my question around: what is different about packages is it that they are not necessarily written in Julia ?

I have still not taken a serious plunge with Julia, testing waters.

Any plans for doing the same to individual scripts ? It would be handy to start off from the last known set of JIT'ed functions, may be retiring paths that are no longer frequented.

OP here. I'm currently using 0.2, I'll have to upgrade and see if that fixes the issue. Thanks!

Is the error reporting improved for 0.3? That was my main problem with 0.2 (on linux). I ran into situations where all I knew was that something went wrong somewhere in my program. I've seen other people make similar comments.

(Aside from that I really love the language, and I'm looking forward to doing more with it.)

May I ask what you're doing with all those instances?

And to be extremely bold: what are you paying for all those licenses?

Dispatch is the process of selecting and executing the matching overload. Multiple-dispatch refers to the ability to select the best overload based on more than just the first argument. C++ can do arity and type-based multiple-dispatch, but only statically. Julia supports dynamic multiple dispatch, where as C++'s virtual methods are only single-dispatch dynamically.

From playing a bit with the Julia REPL (http://forio.com/julia/repl/), it seems that the dispatch of the method is dynamic, as opposed to the dispatch of overloaded functions in C++:

  julia> function foo(s::String) "$(s)!!!" end
  foo (generic function with 1 method)
  julia> function foo(n::Int) n + 1 end
  foo (generic function with 2 methods)
  julia> foo(rand() < 0.5 ? "Hello" : 41)
  42

I'm not used to this kind of dispatch for bare functions, but it seems to me that it could offer similar capabilities to what subtype polymorphism provides in OO†, without having to define new types for this methods to live in or polluting the existing types.

† Probably more, as the dispatch can be resolved based on many arguments.

Stefan Karpinski wrote a very nice overview of multiple dispatch with many examples and some discussion of design implications:

http://nbviewer.ipython.org/gist/StefanKarpinski/b8fe9dbb36c...

(he gave this as a talk at Strange Loop 2013, but the video seems to still be embargoed)

Julia's comprehensions are multidimensional array comprehensions so allowing filtering doesn't make much sense because the result needs to be rectangular and filtering can create arbitrary holes.

For a one-dimensional array comprehension it does make sense to allow filtering and this can be easily expressed in Julia by making a coroutine that produces the values and collecting its output in an array.

Multidimensional (here multi=2) array result:

    julia> [i+j for i=1:3, j=1:3]
    3x3 Array{Int32,2}:
     2  3  4
     3  4  5
     4  5  6

One dimensional array result, with filtering:

    julia> collect(@task for i=1:6; if i%3>0; produce(i); end; end) 
    4-element Array{Any,1}:
     1
     2
     4
     5

Notice that this last technique is more powerful than typical comprehensions, since it doesn't just add the possibility of filtering but arbitrary code to generate the elements.

Single inheritance is possible and used extensively in Julia. Multiple inheritance is not (currently) supported, and there are a few places where it would be useful, but it is surprising that this would be a show stopper.

There is this, which is similar but not quite the same:

    julia> "hello" |> reverse |> uppercase
    "OLLEH"

That's an important part of Julia, but it's also important to talk about the language as a language. One of nice things about Julia is that it has things to talk about other than its numerical computing capabilities :) Those are fine and dandy, but as a non-numerical-computer, I find lots to like about Julia that has nothing to do with its ability to multiply matrices.

They are: http://docs.julialang.org/en/latest/manual/modules/

Or is that not what you were referring to?

Huh, what upside does Java have?

Julia is specifically designed to be a great high-performance high-level language for scientific computing. See http://julialang.org/blog/2012/02/why-we-created-julia/

public class Answer {

public static void main(String[] args) {

   System.out.println("That's why");

 }

}

A lot of people doing scientific programming are not, in many senses of the word, 'programmers', and java doesn't really fit in the kinds of workflow they want. (That's not a condemnation, it's just the way it is, see also the ghosts of fortran 77)