Hacker News Reader

Comment by death_eternal | Hacker News Reader

Like I said, the use case is heavy numerical workloads with, e.g. dataframes, in a context where the data is too big for something like python to handle. Using Nim for this is quite difficult too due to value unboxing overhead. It is easier to optimize for things like cache locality and avoid unnecessary allocations using this tool.

View on HN

I wouldn't reply except you mentioned this unboxing twice and I think people might get the wrong idea. I almost never use ref T/ref object in Nim. I find if you annotate functions at the C level gcc can even autovectorize a lot, like all four central moments: https://github.com/c-blake/adix/blob/3bee09a24313f8d92c185c9... - the relevance being that redoing your own BLAS level 1/2 stuff is really not so bad if the backend is autovectorizing it for you, and full SVD/matrix multiplying/factorizing can be a lot of work. Anyway, as per the link, the Nim itself is just straight, idiomatic Nim.

Parallel/threads is some whole other can of worms, of course. It is unfortunate that the stdlib is weak, both here and for numerics, and for other things, and that people are as dependency allergic as in C culture.

Anyway, "easier to optimize" is often subjective, and I don't mean to discourage you.

The goal is to not need to write C just to get performant code, especially for things like concurrency and numerics. While Nim can be fast without `ref object`, and you can guide the compiler to autovectorize in some cases, it often requires deep knowledge of both the compiler and backend behaviour. That’s not a good developer experience for many users.

Multithreading in Nim is bad to say the least and has been for a while. The standard library option for multithreading is deprecated, and most alternatives like weave are either unmaintained or insanely limited (taskpools, malebolgia, etc.). There's no straightforward, idiomatic way to write data-parallel or task-parallel code in Nim today.

The idea of the project is to make shared-memory parallelism as simple as writing a `parallel:` block, without macros or unsafe hacks, and with synchronization handled automatically by the compiler.

Course, performance can be dragged out of Nim with effort, but there's a need for a language where fast, concurrent, GC-optional code is the default, not something one has to wrestle into existence.

FWIW, I almost always have to look at the generated assembly (never mind C) to even verify things are vectorized. I hope you realize the promise of a PLang where that doesn't need checking or some kind of "autovec-miss" compiler flags or so on and wish you all the best.

I'm curious if you have looked at Chapel: https://chapel-lang.org/

If not, you might find some thing there inspiring.

Reply on news.ycombinator.com