Hacker News Reader

We're using it in a soft-realtime setting to monitor industrial chlorine production, and for us it has been a very pleasant experience overall. Yes, we've had some issues, but similar to other ecosystem IMO, and our support contract with Julia computing helped us in the one case we really couldn't solve ourselves.

Julia works really well for power users. There are no huge libraries full of C code like pandas or scipy. Instead there are dozens of small, well-tested packages that fill the same role, all hosted on github. That makes fixing issues so much easier.

Granted: outside of numerical/technical computing, the libraries can be lacking (eg. web development) compared to other languages. We're doing it anyway, but it's a more difficult decision.

>and our support contract with Julia computing helped us in the one case we really couldn't solve ourselves.

Could you elaborate? What was the problem and what was the solution?

See this issue: https://github.com/JuliaLang/julia/issues/30653#issuecomment...

Memory leak. As explained, it's not really Julia's fault.

This I also found amazing: https://github.com/JuliaLang/julia/issues/28726 . It took 4 hours to go from bug report about bad compiler code generation to a patch.

It seems that Julia is simply discovering the effects of memory fragmentation and the corresponding glibc malloc tunables.

I have commented with some details there:

https://github.com/JuliaLang/julia/issues/30653#issuecomment...

> There are no huge libraries full of C code like pandas or scipy.

Isn't Julia's LinearAlgebra a wrapper around BLAS/LAPACK implementations (OpenBLAS/MKL etc.)?

LinearAlgebra is mainly julia code. It does indeed call out to BLAS for many operations, but a huge amount of it is native Julia.

How much LOC did u guys have at the moment, and how much data are u monitoring? For smaller problems Julia was working perfect for us.

For data, we're monitoring ~1000 time series per plant, at about 2 points/minute. Julia's speed is not necessary there, but it is critical for historical simulations of algorithms.

Around 10K, split among 6 packages

> If u consider Julia as ur next language dont use it to any medium to big project outside pure scientific purposes it just isnt ready yet

My employer, Invenia, uses Julia for what I would say is at least a medium sized project.

~200k LOC, ~30 people concurrently working on that code base, literally the core product/system that makes us money.

Julia's not perfect, but I would not blanket discount it out of hand for medium projects. Like all technologies the tradeoffs need to be investigated with reference to the task (and team) at hand.

> near stable and/or fast as any other "new" languages (RUST, Nim, Golang, even Swift)

This is not my experience, at least for numeric code. It generates faster code than Golang, because it uses an LLVM backend (and actually supports macros and parametric polymorphism, so doesn't need to do the work at runtime), faster numeric code than Rust via @inbounds and @simd annotations (way more work to disable bounds checks in Rust), and faster than Swift because it doesn't have pervasive reference counting that can sneak in and destroy performance.

For numeric code sure but its because Julia are using BLAS (or any other instruction for CPU/GPU u give to LLVM). Julia (no BLAS) -> In matmul its on pair with Golang and Swift and a bit slower then RUST and Nim. If u need BLAS then just use lib with that :)

I'm having a hard time trying to follow your argument. You seem to have had some issues with Julia, and I would like to take those issues at face value. It would be nice if you could point to specific issues since we always like to fix things.

However, in response to discussion about Julia having an integrated set of abstractions that provide high performance - you are discussing a collection of features in Rust, Go, Swift, Nim, TensorFlow, Jax, PyTorch, etc. For any single feature, there can always be some other thing that does it better. But it is unclear how that makes one system better than another.

Numeric code doesn't just mean matrix multiplication. If I have a random nested for loop, it will probably run at least as fast as those languages(apart from maybe Nim, never used it) if annotated with @simd and @inbounds. If I'm operating on a small array/matrix, then Julia will blow Go/Rust out of the water via stack-allocated static arrays (https://github.com/JuliaArrays/StaticArrays.jl). These can't be implemented in Go because it doesn't support type parameters, let alone integer type parameters, and is only recently supported in Rust via const generics, which as far as I'm aware haven't stabilised yet.

Actaully, julia for loops can be made to perform at BLAS levels with this package: https://github.com/chriselrod/LoopVectorization.jl.

Here are my early experiments at making a pure-julia multi-threaded BLAS using LoopVectorization.jl https://github.com/MasonProtter/Gaius.jl. It absolutely blows a naive triple for loop out of the water and is quite competitive against OpenBLAS until you get to very big sizes.

For small, statically sized arrays, LoopVectorization + tripple loops is also much faster than MArrays. LoopVectorization doesn't support SArrays yet, because you can't get pointers to them.

MArrays will be stack allocated if they don't escape.

One of my in development packages also uses it's own "stack" (MMap a chunk of memory), so that it can have pointers to fast "stack-alocated" arrays.

I played around with LLVM's alloca a bit, but it seems like I could only ever use a single alloca at a time; if I ever used more than one, LLVM would just return the same pointer each time instead of incrementing it. If I have to manage incrementing the pointers myself anyway, I may as well use my own stack, too.

For (the problems I have tested and tuned it on), LoopVectorization produces faster code than C/Fortran, e.g.: https://chriselrod.github.io/LoopVectorization.jl/latest/exa... But it may be more fair to compare it with plutocc. In my early tests (which involved much larger problem sizes), plutocc does a lot better, because (unlike LoopVectorization) it seems to consider memory/caches rather than just registers allocation and instruction costs.

Ok let me get it here cause there are a lot of misunderstanding my post on top.

Swift have built in simd same about Rust https://github.com/apple/swift-evolution/blob/master/proposa...

Nim have arraymancer https://github.com/mratsim/Arraymancer

@inbounds it is just compiler option u can use it in any language with LLVM backend i would be suprised if Julia will be faster then Rust/Swift or Nim in this regard. But True about GO in that particular case.

>@inbounds it is just compiler option u can use it in any language with LLVM backend i would be suprised if Julia will be faster then Rust/Swift or Nim in this regard.

I agree it wouldn't necessarily be faster, but it also wouldn't be slower. Plus in Rust at least disabling bounds checks requires marking code as unsafe, which really gets the community's hackles up.

It only requires unsafe if the compiler can’t figure out that removing them is okay. Often bounds checks are not emitted, even in safe code.

Providing a well functioning BLAS experience in a language is a non-trivial task. Saying "just use lib" seems to severely understate this.

How many go or swift or Nim issues discuss matmul vs the Julia repo?

Nim have Arraymancer https://github.com/mratsim/Arraymancer sure it isn't easy but still most of top dogs have some libs supporting BLAS/CUDA etc.

Sorry to hear that. I do hope you filed issues or chimed in on existing issues. On deep learning, Flux has a different flexible approach. This is great for certain people. For people who want a library of existing models, Julia can already call TensorFlow and we didn't see the need to reinvent Tensorflow.

I'm also fighting the GC all the time, although I love Julia.

I wish I could tell the compiler to compile a function only if it can do it without using memory allocations. Often I would rather see a compilation error than the GC making performance of a function unpredictable.

Yep, you're not alone in wanting tighter control over memory allocation and GC. It's something that we're planning to work on this year.

It's really cool.

Of course borrow checker would be helpful, but I guess that's too hard to implement, that's why something like this would be a good balance.

BTW, does the core Julia team maintain an active list of this and other priorities?

I have not seen one, but I would really like to! It's one thing I envy about some other open-source language projects.

I also think Julia would benefit a lot from a list of "we want these but haven't had time for them". People like contributing things they know are desired, and with Julia it's hard to know whether people want or will accept something until you make the PR.

I really like the idea of Julia, and my biggest hope is that it can still replace the use of R in statistics and scientific computing, but I have to agree it is a bit unwieldy in larger projects or as a general programming language and that nim is really promising in this regard.

Sounds like a great opportunity to contribute fixes and improvements.

I have to admit that over the past 20 years, java is like the most stable robust, huge project, multiple teams, IDE friendly, type safe, infinite tooling and libraries language of choice!

Java is good for big projects but not for Machine Learning purposes (Also im C# fanboy so still will prefer C# over Java :D)

Type safe? NPE en masse.

IDE friendly? IDE support is only good because you absolutely need an IDE for this language.

It's an unwieldy, verbose mess of a language.

You need an IDE for any non-trivial project in any language.

This just isn't true. There's a lot of tooling nowadays that provides IDE-like features in text editors, or via command-line tuning. And different people work differently. Where some people use IDEs, others never leave the terminal and just use e.g. vim/emacs, a build system, and a debugger.

agreed - it's almost as bad as Java/Scala for needing a heavy IDE to make it work

Not too surprising. Java was _designed_ by ex Unix engineers at a time that the word "engineer" meant something. Now we have the cloud/web stack and _Javascript_.

The bar was low enough for Julia to sound good at the start, but it gives the impression of being "designed" "on-the-fly".

Luckily in some parts of the world engineer still means something by law, even in informatics, and one is not allowed to use it just because they went through a six weeks programming bootcamp.

Engineering implies scientific rigour and high intellect. So, no wonder software developers, usually a derided and mistreated collective, just love being called engineer...

Engineering (as in PEng in the US, but my jurisdiction just calls it Engineer) also implies liability, and a risk of losing your license if you fuck up.