Hacker News Reader

Comment by enriquto | Hacker News Reader

> I think it requires a jvm mindset, warmup once and iterate inside rather than outside.

I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful!

For example, imagine I have a collection of a few hundred images with their projection matrices (in text files). I want to crop them and apply a simple imagemagick operation (which is not available from inside julia). The elementary solution is to run a shell loop to apply the crop, and call julia to perform a simple adaptation of each projection matrix. This is impossible today: most of the running time of such loop is spent on julia initialization. Half a second to do nothing is simply unacceptable in a serious scripting language.

View on HN

>I do not have this mindset then. I prefer tools who are mindset oblivious. They are really useful!

Yet you picked a specialized tool "imagemagick" (made for working with images), and want to drive it by another specialized tool (the shell, made for file and process management).

Since you want "mindset oblivious" tools why not write everything from scratch, in C?

> Since you want "mindset oblivious" tools why not write everything from scratch, in C?

Because imagemagick, julia and other specialized programs already provide the advanced algorithms that I need. Unfortunately, julia is not a good team player.

The Julia tooling is not currently designed for repeatedly creating new REPL/interpreter instances to quickly solve one small problem.

What it excels at is rapidly solving the same problem many times within one REPL/interpreter (since after the first time the code will be compiled), or for longer computations where the overhead of the initial compilation does not matter.

It would be great if it could also get fast at the use case you are interested in, but this isn’t currently what it excels at.

>Unfortunately, julia is not a good team player.

Or, you know, Julia is not specialized for your use case.

I guessed that it wasn't your habits but if you step outside that box a bit you'll realize that lifting up or down things inside loops is the most natural thing to do. And for you example (which may not be your real use or workflow) you could have two loops, independent or coupled through a queue so that the julia process is only started once.

Of course! Once I realize that after a few seconds only a dozen images have been processed I cut the loop, remember that the julia repl is dog slow, and then, rewrite the task in a different way. But I would prefer not to have to do that. Moreover, for more complicated examples, there may be data dependencies that make the loop commutation non trivial.

The slow startup time may be a minor inconvenience, I agree. But nonetheless it seems to be a case of sloppy engineering, as other scripting languages do not have this egregious problem. It sets a bad tone, and makes me wonder if there are other hidden monsters inside the interpreter, that may be solved by "you are holding it wrong" like this one.

Yeah I get it, it's inconvenient. But I taking the jvm thing again, I think julia's value is outside of this way of doing things.

Out of curiosity I timed ocaml

    $ time echo 'let x = "1" in print_endline(x)' |  ocaml -stdin - 
    1
    
    real    0m0.045s
    user    0m0.039s
    sys     0m0.007s

I thought julia init time was due to type checking phase but ocaml seems to have no issue, even with non zero code:

     $ time echo 'let l = [1;2;3;4] in let m = List.map (fun x -> x*x) l in print_int(List.length(m))' |  ocaml -stdin - 
     4
     real    0m0.050s
     user    0m0.040s
     sys     0m0.011s

Sad, I liked a lot of Julia features..

ps: this hints at the init time being constant recompilation of its core https://www.reddit.com/r/Julia/comments/4c09m1/julia_045_sta... and they were thinking (2 years ago) of caching..

What were you planning to use Julia for where its init time matters?

Nothing, i'm just curious [1] about Julia. I never used it.

[1]It's just because i find the science python stack a bit absurd.

Right now, using is as a “quick” command line utility for solving small problems is probably not a good Julia work flow (unless the overhead you mention doesn’t matter to your application). Where it shines is for longer-running or repeated computations where the initial compilation cost of starting a new REPL doesn’t matter.

Preaching the convinced

You have to use it inside a REPL, like R or Matlab. Julia using LLVM and compiling on-the-fly is not sloppy design, it's a design tradeoff (and a great one, if you really need speed).

Julia is not a scripting language, it's a language for mathematical analysis.

Also going back to your example the solution would be to use ImageMagick.jl from Julia, and extend the library (4 extra lines with ccall) if it's not available, and send a pull request to help the community.

> Julia is not a scripting language, it's a language for mathematical analysis.

Ok, this clarifies the matter a lot. So julia is not intended to be a general-purpose programming language. Are you involved in julia development?

(Besides, I strongly dislike the design tradeoff of not being a good unix citizen.)

Not Scripting != Not general purpose

Julia is a general-purpose programming language. The language itself is actually really good as a scripting language, including integrating well with shell, C, Fortran, R and Python, for example:

https://docs.julialang.org/en/latest/manual/running-external...

The only trade-off here is the restricted resources the Julia devs can afford. Focusing on the scientific computation niche, the Julia compiler was built to compile code just ahead of time, since this way it can create code as fast as compiled languages while still being dynamic like any interpreted language (such as Matlab and Python). But there is nothing in the language that prevents it from being fully interpreted (and therefore having minimal compile time overhead, but generating poorly optimized code for long running processes), or alternatively pre-compiling and caching to have both (which is being worked on: https://github.com/JuliaLang/PackageCompiler.jl )

I'm not a Julia developer, but the great thing with Julia is that the libraries are exceptionally easy to read and extend, because the core developers make a strong effort to write clean code. Just look at ImageMagick.jl and you'll see, but I have the same experience with other libraries as well.

Also the language itself supports extending code (multiple dispatch really helps it to stay clean).

Why can you not call you imagemagick operation from julia? Can't you just `run(``)` it?

This is a part of a quite large system that does a lot of things. I do not want to wrap everything inside julia, that would force me to rewrite everything in julia. I just need to invert a few 4x4 linear systems from inside a shell script. I agree that this is not a very representative use case, and maybe it is irrelevant for most people. Yet, if I try to evangelize julia in my lab this problem blocks it, and it is better to stick to octave.

Reply on news.ycombinator.com