Hacker News Reader

Comment by ashton314 | Hacker News Reader

I find Julia to be a beautiful little language. Applause to the team for this 1.3 release!

In one of my CS classes we wrote a LISP using Julia. Our professor wrote the parser for us, so we just had to focus on the actual interpreter bit. The pattern matching/multi-dispatch mechanisms were really nice. We didn't get to play too much with the parallelism mechanisms, but I liked what I did see.

I feel like Julia is a much richer language than, say, Python, and it would be a nice drop-in replacement for numerical computing tasks: it's faster, in some ways more ergonomic, and afaik can call out to Python libraries if needed. Has anyone here switched from Python to Julia for scientific/ML/AI purposes?

View on HN

I switched to Julia from Python due to facing the classic '2-language problem' of scientists. We wanted an easy to use API and good speed.

The main bottleneck was computing the Mittag-Leffler function. First we tried in Python, then we get a ~10x improvement by outsourcing that bit to Fortran. The Julia implementation of the Mittag-Leffler function is comparable to Fortran but writing the entire software package in Julia led to ~5x speedup due to avoiding language interop issues and taking advantage of other Julia-specific benefits.

So, you obtained a 5x speedup of the whole package compared to the Python version with some Fortran extensions?

Yes.

And for anyone wondering about the state of the Julia package ecosystem compared to Python, someone had already written the main two packages we needed in Julia (for Mittag-Leffler computation and Inverse Laplace computation). At that time there was no Mittag-Leffler implemented in Python or SciPy that I could find.

I could also add that we were computing integral equations with Mittag-Leffler function in the Kernels so in worst cases we needed O(n^2) calls to Mittag-Leffler function where n could be O(10^5).

If you need another speedup consider LuaJIT which is used at CERN for heavy number crunching and data analysis, see http://cds.cern.ch/record/2157242?ln=de. E.g. on slide 14: LuaJIT took 5.3 sec, C++/GCC 5.7 sec, PyPy 7.6 sec, Julia 35 sec and Python > 2 hours.

From the comments on that:

> No one should use global scope of Julia for calculations: [...] which is said that takes 35 seconds. > If you wrap it in a function and not had a type instability, You'd get 4.704 seconds...

Half-decently written Julia ought not to be much slower than C.

But it does also allow you to write quick and dirty scripts with no care for speed, which is also useful. It would be nice if people wouldn't benchmark the latter. The manual has a pretty helpful section for avoiding this: https://docs.julialang.org/en/v1/manual/performance-tips/

Indeed, I would imagine that well written Julia is on par with LuaJIT.

Even more importantly, from a cursory look online I cannot find any Lua implementation of either the Mittag-Leffler function or Inverse Laplace routines. And there is no reference to any wider ecosystem of scientific computing packages.

Sure I could call some C functions but then I'm back to facing a 2-language problem which I don't want.

Furthermore, it looks like there has been no new releases of LuaJIT since 2017. Development on it looks to have stopped.

If you can measure you don't have to imagine.

I watched the video, those benchmarks are just terrible. If one wants to use LuaJIT for a porject, sure go for it, but making up weird benchmarks like that which purposefully cripple competitor language performance is just embarrassing.

Well, weird benchmarks are probably all those where your favorite language doesn't look good ;-)

I would say in this case what's 'weird' about the benchmarks is that the optimization strategy taken across the languages is incredibly uneven. Indeed, simply wrapping the above julia code in a `let` block so it's not out in the global scope is sufficient to have it on par with C++ (assuming the same flags are used, e.g. fastmath).

I played around with Julia a bit last year. One thing that stuck with me was the non-negligible start-up speed. If you want tight feedback loops you have to figure out a way to reuse a Julia instance (like Jupyter).

On that train of thought, how interoperable is preexisting numpy/pandas code with Julia?

As for startup speed, note that in this round our research software engineer in the Julia Lab just brought the "first time to plot" to about 1/4 of what it was before. Julia 1.3 has a lot of these updates, and Julia 1.4 has a development sprint for bringing down the latency. I would be surprised if the common major latency issues still exist past next summer. Of course a JIT will always have some bumps compared to a pure interpreter, but Julia's compiler is definitely getting a lot smarter as to what to try and fully analyze (and what to cache)

You can ;-) The secret sauce in Julia is to use Revise.jl package. Is makes your life so much better. It observes source code changes while you are at the REPL. No need to restart everything.

How reliable is Revise.jl compared to ipython autoreload? My experience with ipython autoreload is... very poor, even though some people swear by it, so I have trust issues with this type of thing.

I use it extensively. In my experience it is very reliable. Onle once or twice in two years of using it has something strange happened with JIT compilation of functions. All I had to do to fix was restart the Julia REPL and load things in again.

+1 for this package. Total game changer.

One downside is Revise v2 actually itself has a fairly high startup cost. Best alternative is to pin it to Revise v1.

I personsally don't mind the startup too much. But I do get where people are coming from

> If you want tight feedback loops you have to figure out a way to reuse a Julia instance

I think Julia is meant to be used like Lisp:

- Step 1: Open the REPL.

- Step 2: Don't close it until you are finished (both writing and running your program).

My best guess is that people that complain about startup speed of Julia (and Emacs too for that matter) ignore step 2.

For myself, I write scripts that are part of automated analysis pipelines. So in my case (and probably others), using the REPL is mostly useful for testing code.

Julia has 0-copy numpy array interop through PyCall.jl if that's what you're asking.

It is. Thanks!

> One thing that stuck with me was the non-negligible start-up speed.

Yes! this is the biggest roadblock for me. I find python startup time unbearably long. I do not want to imagine even using julia. And this is sad because the language is orders of magnitude better (for math, at least).

If you're used to launching everything from your OS's command prompt, this is a killer. But if you get used to launching a REPL and running things from there, it's much better, because you only pay the startup latency once.

Some workflows require running scripts from the command line though

I agree and this still holds it back from being the main language in a research environment where one wants to have quick turnaround time.

On the other hand, it should work very well as a replacement for Numba. Numba already comes with a one-off jit cost, and it is much more limited in the sort of code it can speed up.

Is there any chance the course material is available? I've been looking to better my Julia meta-programming. Most of the resources out there are geared towards other languages. It would be amazing to see some simple parser/interpreter implementations.

Not being 100% interoperable with python make Julia doomed.

Library ecosystems have a human workforce value of billions of dollars, available for free. Any modern language that want to go mainstream must offer transparent and complete interoperability with an existing ecosystem. TypeScript -> js Kotlin -> jvm C++ -> C Elixir -> Erlang Hack -> php Crystal -> ruby

As we can see there are still Big ecosystems lacking a new modern language™ The python ecosystem The C# (CLR) ecosystem

It could be said that's mainly because those languages are already mostly "modern"

There are also dying big ecosystem and there would be economic value to make their old libraries accessible. Fortran Pascal Perl Etc

There are only 2 languages with really good Python interop (you could call it "as close as possible to 100%" since there is no such thing as "100% interop" except across languages sharing same VM like on the JVM Java & Clojure & Kotlin...).

One of them is Julia! - https://github.com/JuliaPy/PyCall.jl .

The other is Swift - https://github.com/tensorflow/swift/blob/master/docs/PythonI... .

And pretty sure the Swift team did it after seeing how well it went for julia.

You can use Python, C and Fortran code from Julia. More importantly package development is much faster in Julia than in Python. Which means the speed of which packages get added to Julia is much faster than Python.

And here is the kicker: Packages is Julia are far more interoperable than in any other eco system I have seen.

Than means 10 packages in Julia can quickly do more than 50 packages in other language. Say somebody makes a GPU processing package and somebody else makes a Machine learning package. They don't know about it each other.

Yet with Julia we frequently see that these guy can use each others package. The ML package can with little effort run on a GPU by including a GPU package even if it was never designed for it.

Julia just moves way faster than the competition and that is why Julia is NOT doomed.

>You can use Python, C and Fortran code from Julia. More importantly package development is much faster in Julia than in Python. Which means the speed of which packages get added to Julia is much faster than Python.

That's because Python already has tons, while Julia lacks tons.

Are the new ones added to Julia as good or better though?

>That means 10 packages in Julia can quickly do more than 50 packages in other language. Say somebody makes a GPU processing package and somebody else makes a Machine learning package. They don't know about it each other. Yet with Julia we frequently see that these guy can use each others package. The ML package can with little effort run on a GPU by including a GPU package even if it was never designed for it.

This sounds like magic bullet / handwaving.

Perhaps Julia's multiple dispatch helps here (?), but why would the above be the case and wouldn't be achievable in other existing languages as is?

>This sounds like magic bullet / handwaving.

It's not handwaving when it already exists. CUDAnative.jl gives native compilation of Julia code to GPUs, which allows CuArrays.jl to do efficient loop fusion among other tricks. This is then enough for DifferentialEquations.jl, Flux.jl (the big neural net library, think PyTorch), CLIMA (a MIT-CalTech next-gen climate model), etc. (too many packages to list) to all utilize the same underlying GPU framework. It's as if someone got rid of the GPU functionality of Numba, PyTorch, TensorFlow, etc. and just made one good enough Python library for everyone to share. As you can imagine, that reduces the amount of work to do package development but gets the same functionality in the end with a lot less maintenance issues. You can also say the same for how many JITs and transpilers to C there are in Python libraries vs the single Julia JIT.

The real danger of the Python monoliths is that when something as simple as a neural network package starts to include a JIT, a GPU compiler, and automatic differentiation library, and it's all written in C++, it gets really hard to attract developers because the code base is so hard to understand. This means that instead of getting new developers and more maintenance, people just spawn new libraries all of the time in Python, starting from scratch to write a new JIT, new GPU kernels, etc. I just cannot see how that's a good thing.

> Perhaps Julia's multiple dispatch helps here (?), but why would the above be the case and wouldn't be achievable in other existing languages as is?

Yes, ubiquitous, zero-overhead multiple dispatch is the reason, as explained in depth here:

https://youtu.be/kc9HwsxE1OY

Thanks, checking!

>> Not being 100% interoperable with python make Julia doomed.

> Than means 10 packages in Julia can quickly do more than 50 packages in other language.

Maybe you guys would like to compare your text editors now?

> As we can see there are still Big ecosystems lacking a new modern language™ [...] The C# (CLR) ecosystem

F#, obviously.

Not to mention C# is as modern as it gets...

Is it really? Class-based single dispatch OO with strictly explicit static typing—no gradual or optional or even Hindley-Milner type inference—is about as 90s a language design as there is. It’s like the “Friends” of PL designs. Which is to say, very popular and not bad, but certainly not modern or cutting edge.

I didn't say "cutting edge" though, I said modern.

Modern is not some "PL-community favorite with small following" a la Haskell, it's what programmers in the trenches are using. The same way a modern car is (pick a popular current model) and not some limited edition car with limited sales.

C# "keeps up with the Joneses", brings in ideas from functional programming (bringing in F# features incrementally), etc.

LINQ, pattern matching, dynamic, value types, tuples, destructuring, async/await, null-coalescing operators/assignment, streams, and much more...

Not only is up there with any popular language in the same domain (Swift, Kotlin, etc), and way ahead of Java, but compared to something like Go that gets people excited this is space technology.

(Plus is not like multiple dispatch is a new development, or gradual/optional typing is the "new" thing - we had those things for decades, and I don't see them catching on except the latter where necessary - e.g. adding gradual typing in JS/Python/etc variants like TS, where they can't have strict static typing for compatibility/historical reasons. The only language with gradual typing I remember in recent times was not very cutting-edge itself, Dart, and didn't manage to go anywhere).

C# supports multi-dispatch and gradual typing using the dynamic keyword since 4.0. It also does some type inference using var, but not full HM.

How is C# modern? It is extremely verbose and rooted in Java style 'OO' with lots of memory hopping.

Every iteration of C# has made it less verbose and adds more modern concepts [1].

What do you mean by memory hopping, reference types? C# has had value types since the beginning unlike Java.

1. https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csh...

Modern != less verbose

Reply on news.ycombinator.com