Hacker News Reader

Comment by enriquto | Hacker News Reader

> it's faster than other scripting languages

That certainly depends on your use case. For instance, the launch time is ridiculously slow, so that you cannot realistically run a small matrix computation in julia from within a shell loop. It is better to use octave for that, where the startup time is almost negligible (just a bit slower than starting a subshell).

View on HN

I agree. I meta-programmed (enormous) expressions from analytical expressions exported from Mathematica to Julia, because I found Julia to be ~3000 times faster than Mathematica when it comes to calculating eigenvalues. Using BigFloat for higher precision, my matrix function in Julia took ~20 minutes to compile on the first run and ~20 GB of RAM. Smooth once compiled, but I was the only one of my collaborators that had the capacity to run it.

Note, that I still do like using Julia. I'm a physicist and need to do a lot of computations in a hassle-free way, then jupyter+Julia (+ SymPy) is(/are) the best available tool(s).

The above may only be an issue of BigFloat, to be fair, since Float64 compiled in an instant (never measured, and the time never bothered me).

So Julia has solved a lot of problems for me, and I see great potential for it in the future.

I didn't find it slow, but:

    $ time julia -e 'print(1)'
    1
    real    0m0.438s
    user    0m0.300s
    sys     0m0.118s
    $ time python -c 'print(1)'
    1

    real    0m0.040s
    user    0m0.036s
    sys     0m0.003s

it is slower..

That said, instanciating julia every step of a bash loop.. I think it requires a jvm mindset, warmup once and iterate inside rather than outside.

> I think it requires a jvm mindset, warmup once and iterate inside rather than outside.

I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful!

For example, imagine I have a collection of a few hundred images with their projection matrices (in text files). I want to crop them and apply a simple imagemagick operation (which is not available from inside julia). The elementary solution is to run a shell loop to apply the crop, and call julia to perform a simple adaptation of each projection matrix. This is impossible today: most of the running time of such loop is spent on julia initialization. Half a second to do nothing is simply unacceptable in a serious scripting language.

>I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful!

Yet you picked a specialized tool "imagemagick" (made for working with images), and want to drive it by another specialized tool (the shell, made for file and process management).

Since you want "mindset oblivious" tools why not write everything from scratch, in C?

> Since you want "mindset oblivious" tools why not write everything from scratch, in C?

Because imagemagick, julia and other specialized programs already provide the advanced algorithms that I need. Unfortunately, julia is not a good team player.

The Julia tooling is not currently designed for repeatedly creating new REPL/interpreter instances to quickly solve one small problem.

What it excels at is rapidly solving the same problem many times within one REPL/interpreter (since after the first time the code will be compiled), or for longer computations where the overhead of the initial compilation does not matter.

It would be great if it could also get fast at the use case you are interested in, but this isn’t currently what it excels at.

>Unfortunately, julia is not a good team player.

Or, you know, Julia is not specialized for your use case.

I guessed that it wasn't your habits but if you step outside that box a bit you'll realize that lifting up or down things inside loops is the most natural thing to do. And for you example (which may not be your real use or workflow) you could have two loops, independent or coupled through a queue so that the julia process is only started once.

Of course! Once I realize that after a few seconds only a dozen images have been processed I cut the loop, remember that the julia repl is dog slow, and then, rewrite the task in a different way. But I would prefer not to have to do that. Moreover, for more complicated examples, there may be data dependencies that make the loop commutation non trivial.

The slow startup time may be a minor inconvenience, I agree. But nonetheless it seems to be a case of sloppy engineering, as other scripting languages do not have this egregious problem. It sets a bad tone, and makes me wonder if there are other hidden monsters inside the interpreter, that may be solved by "you are holding it wrong" like this one.

Yeah I get it, it's inconvenient. But I taking the jvm thing again, I think julia's value is outside of this way of doing things.

Out of curiosity I timed ocaml

    $ time echo 'let x = "1" in print_endline(x)' |  ocaml -stdin - 
    1
    
    real    0m0.045s
    user    0m0.039s
    sys     0m0.007s

I thought julia init time was due to type checking phase but ocaml seems to have no issue, even with non zero code:

     $ time echo 'let l = [1;2;3;4] in let m = List.map (fun x -> x*x) l in print_int(List.length(m))' |  ocaml -stdin - 
     4
     real    0m0.050s
     user    0m0.040s
     sys     0m0.011s

Sad, I liked a lot of Julia features..

ps: this hints at the init time being constant recompilation of its core https://www.reddit.com/r/Julia/comments/4c09m1/julia_045_sta... and they were thinking (2 years ago) of caching..

What were you planning to use Julia for where its init time matters?

Nothing, i'm just curious [1] about Julia. I never used it.

[1]It's just because i find the science python stack a bit absurd.

Right now, using is as a “quick” command line utility for solving small problems is probably not a good Julia work flow (unless the overhead you mention doesn’t matter to your application). Where it shines is for longer-running or repeated computations where the initial compilation cost of starting a new REPL doesn’t matter.

Preaching the convinced

You have to use it inside a REPL, like R or Matlab. Julia using LLVM and compiling on-the-fly is not sloppy design, it's a design tradeoff (and a great one, if you really need speed).

Julia is not a scripting language, it's a language for mathematical analysis.

Also going back to your example the solution would be to use ImageMagick.jl from Julia, and extend the library (4 extra lines with ccall) if it's not available, and send a pull request to help the community.

> Julia is not a scripting language, it's a language for mathematical analysis.

Ok, this clarifies the matter a lot. So julia is not intended to be a general-purpose programming language. Are you involved in julia development?

(Besides, I strongly dislike the design tradeoff of not being a good unix citizen.)

Julia is a general-purpose programming language. The language itself is actually really good as a scripting language, including integrating well with shell, C, Fortran, R and Python, for example:

https://docs.julialang.org/en/latest/manual/running-external...

The only trade-off here is the restricted resources the Julia devs can afford. Focusing on the scientific computation niche, the Julia compiler was built to compile code just ahead of time, since this way it can create code as fast as compiled languages while still being dynamic like any interpreted language (such as Matlab and Python). But there is nothing in the language that prevents it from being fully interpreted (and therefore having minimal compile time overhead, but generating poorly optimized code for long running processes), or alternatively pre-compiling and caching to have both (which is being worked on: https://github.com/JuliaLang/PackageCompiler.jl )

Not Scripting != Not general purpose

I'm not a Julia developer, but the great thing with Julia is that the libraries are exceptionally easy to read and extend, because the core developers make a strong effort to write clean code. Just look at ImageMagick.jl and you'll see, but I have the same experience with other libraries as well.

Also the language itself supports extending code (multiple dispatch really helps it to stay clean).

Why can you not call you imagemagick operation from julia? Can't you just `run(``)` it?

This is a part of a quite large system that does a lot of things. I do not want to wrap everything inside julia, that would force me to rewrite everything in julia. I just need to invert a few 4x4 linear systems from inside a shell script. I agree that this is not a very representative use case, and maybe it is irrelevant for most people. Yet, if I try to evangelize julia in my lab this problem blocks it, and it is better to stick to octave.

To demonstrate how slow the startup time is, here’s the respective times for executing ‘print(1)’ from Bash and from REPL on my machine:

REPL:

    julia> @time print(1)
    1  0.000031 seconds (7 allocations: 272 bytes)

Bash:

    $ time ./julia -e 'print(1)'
    1
    real    0m0.184s
    user    0m0.156s
    sys     0m0.168s

So yes, that’s a slow launch time. If you use Julia in a shell script and it’s starting up Julia on each calculation it will be brutally slow. Five orders of magnitude slower, in this case.

honestly, parent had a use case in mind but it's ultra orthogonal with julia's goals. They're aiming at large numerical problems.. as I said above, if you need julia to make a quick one shot computation in a bash script loop, by all means use something else. But my bet is that anybody reading about julia is planning to do the heavy work inside of it, in which init time will probably be insignificant.

It's a bit annoying it's true, a well featured language with dynamic/repl appeal that is not in the ms launch time. But alas..

Totally agreed, I'm a fan of Julia. I just wanted to comment with the delta between startup time and actual execution for passerby.

First of all that is ridiculously slow. Secondly, unlike Matlab you don't have everything you need available after starting the REPL. For instance if you want to plot something you might run `using Gadfly`. How long does that take?

16 seconds. Sixteen seconds. For real. This is not usable.

Hold on how long is MATLAB boot ?

maybe he was thiking about Octave, that is an interpreter for the same language as Matlab and it starts much faster.

Yeah but heaven help you if you need nested for loops in octave.

Much less than 16 seconds. And that gives you a whole IDE.

Agreed Octave works excellently as a linear algebra/numerical subshell for Shell scripting, better than Python IMO.

Reply on news.ycombinator.com