Hacker News Reader

Comment by amrrs | Hacker News Reader

I use both R and Python in my work but when we move our models to production it's not real time, just a batch execution like once in a day. I'd like to hear from anyone who uses Julia in their actual job/work. Is it worth learning Julia, hoping to use it in work some day?

View on HN

R and Python are still (currently) nicer for stats and ML, although there's a lot of not-yet-mature Julia libraries that are extremely cool (Turing.jl for Bayesian inference in particular, and systems for autodiff). I also think Plots.jl is the best-designed plotting system, although if you are a ggplot true believer you might not agree.

But for classical scientific computing, of the sort typically done by real engineers wearing coke-bottle glasses, using MATLAB and expensive toolboxes, it can't be beat. It's really easy to work with matrices and make your code super-fast. And it steals all the good parts of MATLAB syntax but with a good, modern, sensibly designed programming language.

Also, DifferentialEquations.jl is undeniably the best available piece of software for solving differential equations, in terms of both performance and ease-of-use. It feels like using alien technology or something.

I think sometimes people are distracted by Julia's performance, according to https://github.com/JuliaLang/Microbenchmarks Julia is not the fastest language/compiler (maybe LuaJIT is).

Julia is not only good for its performance, but also its multiple dispatch, its type system and more. Because of those features we have https://github.com/JuliaGPU/CUDAnative.jl, more elegant package interface, like https://github.com/JuliaOpt/JuMP.jl and there is more.

I still believe there is no silver bullet, if you want something Python then you get another Python. But think about this: is the full Python's dynamic feature really what we need in works like simulation, HPC, etc.? Probably no. I think Julia is kinda of balance for the related field (like for scientific computing). Just like in the old days, people start using FORTRAN, MATLAB, Lisp as the most advanced tools at that time. We start using Julia now.

In addition to the type system, what are some other features you believe are too dynamic (for reasonable performance)?

Yes probably just for reasonable performance, and I don't actually need to use Python's rich object model. Mostly, I just use Python to call C++ for easier interface (and C++ is for performance). And before Python 3 since there is no type annotation, refactoring is quite annoying.

I'm not an PL expert, so I would like to ref a previous discuss in discourse, which is more detailed

https://discourse.julialang.org/t/julia-motivation-why-weren...

I was just trying to say that if you want to develop something with reasonable performance, elegant interface in a short time. Probably under this situation, Julia is more suitable.

Those benchmarks draw a lot of hate, because anyone coming from language X will get offended at how unoptimized code in language X is. Also true if X == Julia. For example, they never turn off boundschecks, which disables vectorization.

The point is mostly to (a) show the difference between fast languages compiled to efficient assembly and (b) represent code someone new to a language may bang out to get something done (while avoiding performance pitfalls) and avoid the benchmark game.

That said, I agree. I try (poorly) not to advertise speed, because people coming from languages like R will rarely fail to write type unstable code that is slow, observe JIT compilations that make Julia a little laggy, and then come away disappointed.

Things like the type system and meta-programming shown off in your examples are amazing, and also not something you can reproduce in other languages by adding binary dependencies.

> … and avoid the benchmark game.

Escalating benchmarks that add increasing optimizations to continuously one up one another. In the "logical conclusion" of this left unchecked, the Fibonacci benchmark for example (d)evolves into a lookup table. Which is missing the original point, hence the desire to just leave everything un-optimized.

Unfortunately the arbitrary "leave everything un-optimized" also misses the point because in practice we don't.

> (d)evolves into a lookup table

We can make the arbitrary decision not to accept that, and instead try to use our best judgement on what optimizations to accept.

"One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of "reasonably idiomatic" Haskell. However, we have made abundant use of strictness annotations, explicit strictness, and unboxed vectors. We have, more controversially perhaps, used unsafe array subscripting in places. Are our choices reasonable?"

http://www.leafpetersen.com/leaf/publications/ifl2013/haskel...

> I think sometimes people are distracted by Julia's performance…

Perhaps because the Julia homepage distracts them :-)

"Julia is fast! Julia was designed from the beginning for high performance."

I used Julia in my day-to-day work at my previous job (Intel) and in my new position (Rigetti). There's quite a few companies that are using it in production now.

Glad to hear more about how are you deploying Julia in production.

That's really cool! I checked Rigetti's github but there seem to be no open source Julia projects there (although, somewhat surprisingly, lots of Common Lisp!). Are you free to say what Julia is being used for at Rigetti (and Intel), and whether there are any plans to release things as open source in the future?

At Intel I developed a few product modeling tools in Julia, including one that's now used in the introduction of every new product. The only real annoyances that I ran into were related to dependency management in Pkg2, but I think Pkg3 should solve all of that. I'm quite new at Rigetti, but I would suggest keeping an eye on our GitHub page.

In my opinion, Julia has everything you need to be productive in a scientific computing or academic arena right now. In fact, if you're in an atmosphere where there's a lot of chaining together various command line tools Julia actually does subprocesses and pipes quite well.

As a more general point though, I work in an academic environment and I see tons of languages whizzing by. I routinely see C, C++, Python, R, Matlab, csh, bash, and the list just goes on and on. I've even seen someone write a command line tool in PHP that had no justifiable reason for being written in PHP. But it works, so whatever. If it was written in Julia, maybe it would be uncommon, but also slightly more refreshing.

So, really, why not Julia?

We are using Julia to develop a clinical trial simulator for drug dosage prediction and personalized medicine applications as a joint MIT, University of Maryland Baltimore, and JuliaComputing project. It will be open sourced in January, and at the same time we will be starting a clinical trial to test the methods in a real-world setting.

Definitely post a link! Are you doing stochastic modeling, and if so how do you structure the input data? I’d love to use similar techniques for other data modeling.

Here's a link to the talk: https://www.youtube.com/watch?v=KQ4Vtsd9XNw . We are using stochastic models at two levels: at the population level using nonlinear mixed effects models, and at the individual level using discrete stochastic (Gillespie) models and stochastic differential equations via DifferentialEquations.jl. The input data has a standard structure which we read in with tools like JuliaDB, and for output the DiffEq solutions are table structures which can directly output to JuliaDB, DataFrames, or csvs.

I've been using it in production since version 0.5. It's a joy for creating high performance numerical code without having to leave the comfort of a productive and interactive scientific computing environment. Perhaps surprisingly it's also a great glue language, where I think it does a better job of replacing shell scripts than python.

I do love Julia, but start up and import time can make scripting a little rough. Hopefully PackageCompiler.jl will solve that soon.

If you haven't yet, you should check out a build of 1.0. The start-up time is _greatly_ improved. Basic startup time is twice as fast as ipython for me now:

$ time ipython -c '1+1' Out[1]: 2

real 0m0.493s user 0m0.401s sys 0m0.063s

$ time julia -e '1+1'

real 0m0.222s user 0m0.115s sys 0m0.101s

Yes, python itself is still much faster. I compare against ipython just because that used to be my go-to interactive shell for technical computing.

That's pretty cool! I never had to much of an issue with the startup time for interactive computing though. It mainly has come up as an issue when I was writing short scripts that could be run a couple dozen times in serial.

The real question is how long does `using Plots` take in v1.0?

I use it pretty much every work day to explore ML/DSP ideas and to experiment around before formalizing an algorithm. It's fast to run (which is important to be able to go through enough data to make conclusions), the code produced is reasonably readable, there are a decent number of libs, and it's fairly quick to write. I'd say if you're looking for something like a MATLAB/R/Python/etc replacement Julia is a good option, though it has been a bumpy ride at times while the language has been evolving.

I think I started using Julia around 2013 or so after getting fed up with the slower speed of octave, so I'm guessing that things should be a fair bit smoother from this point on if they've officially tagged 1.0.

Reply on news.ycombinator.com