Comment by Archit3ch | Hacker News Reader

Comment by Archit3ch | Hacker News Reader

Parent Full thread

> 6. Multithreading: Base language support is already pretty good, and there's OhMyThreads.jl [1] and data chunking libraries and many other supporting libraries around multithreading.

Agree on the rest, but multithreading in Julia is a let down.

The manual (https://docs.julialang.org/en/v1/manual/parallel-computing/) claims it's composable, but that's only true if you stay within Julia with the paradigm of Tasks. As soon as you interface with C/C++, you get a SegFault, as the Julia runtime expects to adopt [1] all threads calling into it. This is not always viable.

Julia should offer the option of C-style pthreads (or the Windows equivalent) and let others build abstractions on top of them.

[1] And that option was only added recently.

I haven't had to work on the intersection between FFI and multithreading, so I can't comment on that. I was more commenting on the ease-of-use of normal multithreading within a Julia program, and the availability of primitives around it. I've found it much easier to take an existing working program and parallelize it in Julia, than in most other languages I've worked with. (Granted, a lot of this is simply that Julia is a newer language and hence gets to design these into the language from the get-go instead of adding the parts piecemeal over years.)

> I've found it much easier to take an existing working program and parallelize it in Julia, than in most other languages I've worked with.

Unfortunately, this isn't possible when interfacing with proprietary, closed-source binaries. Especially when the C FFI is defined by standard, makes no mention of threading, and each implementation has different quirks.

Multithreading seems to work just fine with OpenBlas. It is also sometimes possible to wrap the underlying state machines from C/C++ code and then making it multithreaded in the Julia side.

None of that contradicts what I wrote. Note that it's calling into Julia from C/C++ that presents problems, the opposite is fine.

Reply on news.ycombinator.com