Hacker News Reader

Comment by The_rationalist | Hacker News Reader

In other words, mostly catching up features present in mainstream languages.

Seems like multi-processing is already better than Python, which is a hot mess. For things that you'd actually need multi-processing in, say, deep learning - like preparing and loading and then saving data, Python is difficult to use. And if you have to run something on windows, you are doomed.

I wrote a custom data pipeline for huge HDF5 files for some ai thing, which was a headache in linux but doable. Then it had to run on windows and I just wanted to kill myself.

Doing the same in Julia or even Matlab was easy, and faster.

And python is supposed to be easy to use :<

The only other dynamic language with this kind of multithreading capability is, wait for it... Raku (aka Perl 6). And my impression is that Raku doesn’t get very high performance with its multithreading, which is mostly designed to improve I/O throughput (and because it’s a cool feature and Raku has never met a feature it doesn’t like).

Even among static languages, it’s a relatively rare capability: Go has excellent support (of course), C extensions like Cilk and TBB support it, Rust has something similar with tokio, and I’ve heard people claim that Haskell can do something similar with appropriate extensions.

> And my impression is that Raku doesn’t get very high performance with its multithreading

Could you elaborate on how you got that impression? Do you have any benchmarks that the Raku core developers could look at?

I recall seeing various benchmarks of Raku that were on the slow side—slower than Perl 5 / Python speed, which are already what I'd describe as "slow languages". I can't currently find what specifically I read, but https://www.reddit.com/r/perl6/comments/btvkoz/is_perl6_stil... seems to have some discussion (and I think you're on that thread, so I'm sure you're aware). This doesn't specifically address threading performance, but if the runtime is 100-500x slower than C, it's not like threading can make up that kind of performance deficit.

The Raku runtime is still improving, and several object creation benchmarks now outrun Perl. A low bar, some might argue. But still.

Threading can make a difference on non-IO bound tasks if you're interested in wallclock rather than CPU: for instance, `say (1..Inf).grep( { .is-prime } )[9999]` (showing the 10000th prime number) runs 22 seconds on my machine, but the threaded version of that: `say (1..Inf).hyper.grep( *.is-prime )[9999]` runs in 8 seconds. That's just by adding the `.hyper` method to the chain!

Here's a comparable thing in Julia:

    julia> using Primes, Lazy

    julia> @time drop(9999, filter(isprime, Lazy.range(1)))[1]
      0.119811 seconds (1.25 M allocations: 23.502 MiB)
    104729

The Lazy package doesn't support using threads yet since 1.3 was just released, so doing the threaded comparison isn't simple at the moment. However, this illustrates what I was getting at: the sequential Julia code is 185x faster than the the sequential Raku code and 67x faster than the threaded Raku code. It's great that Raku threading gets some scaling here (not sure how many cores you have, so it's unclear if 2.75x scaling is good or not, but it's not nothing). But it's considerably easier to scale if there's already a lot of performance on the table. The faster each operation is, the harder it is to make a threading implementation that has low enough overhead to make threads worthwhile. From the performance perspective, any effort spent on threading in Raku would be better spent on sequential speed until that has been maxed out.

I should also note that this is not how one would actually find the 10000th prime efficiently. For that you'd use the `nextprime` function also provided by the Primes package, like so:

    julia> @time nextprime(1, 10000)
      0.010456 seconds (17.00 k allocations: 265.562 KiB)
    104729

That's another 10x faster than the lazy sequence approach. Which mostly tells me that the lazy code is impressively efficient—I would have expected a dedicated function to have more of an edge. Lest anyone cry foul about using C or whatever, this function is implemented fairly straightforwardly in Julia:

https://github.com/JuliaMath/Primes.jl/blob/ce0c1e388e1fd375....

Thank you for these goals :-)

Just for the record: yes, there are faster ways of finding the 10000th prime, but I use this example often as an example of a CPU-intensive task that can be spread over multiple threads easily.

Also, please note that all features of the example are built-in into Raku: no external module loading needed.

This is not a bad thing and it is quite natural considreing how young the language is. The same is true for golang and many other new cool languages. Being conservative when introuducing features in a new language is a very wise approach.

This is natural indeed but it should be more frequently said, as I feel too many developers will go to the new shiny language and loose so many times because of what I call software poverty. (lack of libs, lack of features, lack of stackoverflow, lack of documentation, and even lack of stability) Indeed new languages have new shiny features but they do not outweigh the poverty cost. Especially since there are modern languages compatible with big ecosystems like typescript or kotlin, I don't really see how new languages that are not compatible can be competitive.

Software poverty - Love the term.

> modern languages compatible with big ecosystems like typescript or kotlin

...if you do anything towards ML or datascience, typescript and kotlin are the poor relatives. For most tasks 90% of the ecosystem is useless to you and you can spend more time wrangling the dependencies of a nasty nodejs package than quickly coding smth from scratch.

That "richness" is also... "garbage".

If the language fills a niche sufficiently better than existing ones. Julia doesn’t bring too many novel concepts (JIT, macros, multimethods, parallelism) but ties them together nicely for scientific computing.

The best chance Julia has isn’t attracting average devs alone, but the library developers. Writing new python libraries for research purposes often requires writing C/C++ code. There’s already a number of Julia libraries focusing on interesting research topics in scientific, mathematics, and AI. Those lead to better and more useful libraries over time which can interact over time. Plus Julia Computing seems to be going towards building a successful commercial side of Julia. Personally I don’t find the software poverty thing much of an issue, outside a few core areas (database drivers, basic http, c interop, networking, text editors).

> I don't really see how new languages that are not compatible can be competitive.

What exactly would you like Julia to be compatible with?

Python? https://github.com/JuliaPy/PyCall.jl

JS? https://github.com/SimonDanisch/JSCall.jl

C++? https://github.com/JuliaInterop/Cxx.jl

Java? https://github.com/JuliaInterop/JavaCall.jl

R? https://github.com/JuliaInterop/RCall.jl

Not all of these are fully polished, but they're all being worked on (plus there are a bunch of others that aren't being actively worked on that I haven't put here).

Mathematica? https://github.com/JuliaInterop/MathLink.jl Matlab? https://github.com/JuliaInterop/MATLAB.jl

C/Fortran anything else that compiles to C-API supporting shared libraries is supported in built by `ccall`

JuliaLang is amazing at of FFI

One day Python will be as fast as Dylan.

Reply on news.ycombinator.com