I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.