Hacker News Reader

I agree with almost all of what you said. Except personally, I'm much more productive in something like Julia (or R or Python) than a language like Rust for the type of development I usually want to do. I mostly write Python and C++ for my dayjob, but would have a hard time justifying Julia for production at the moment, though I love the language.

Even more than Rust or Go, I think Julia could take a page from Typescript in terms of tooling because JS is so dynamic and the tooling is so nice.

I do think compiler latency has gotten way better in more recent versions of Julia, but I think better ahead of time compilation, faster interpretation, and code caching are all important.

I think Julia's tooling used to be better in Atom (Juno), however, Atom is dead.

I agree with you. Julia has gotten better but it’s still way slower than Python or R. And the workflow of running tests just make it painful. I think I’m familiar enough with Julia and I can write code and it’ll likely just work, but I’m not a 100% confident. With rust (and Python or Typescript with type hinting), I’m much MUCH more confident.

This is a good, well-thought out critique. While much of it rings true, I can't say I completely agree:

Wrt. static type checking, JET.jl does the job for me. I agree it still feels too immature with slowness and important edge cases it fails to consider. More importantly, the ecosystem itself needs to fully embrace type stability for it to be optimally useful, since otherwise you will get a shitton of false positives from functioning, yet unstable dependencies. The good news is that all of these things are improving quickly for JET.jl - it's getting faster and more robust. It still feeds immature since it's actively being developed.

There is also the linter, which is available in VSCode. It's pretty good. Nowhere near the level of rust-analyzer, but still decent.

You can easily have error types in Julia - see the package ErrorTypes.jl. It's not standard to use it, but I use it in my work code, and it works quite well.

You can absolutely write tests next to code and just run one test - see InlineTests.jl and ReTest.jl

HOWEVER:

* The latency is still terrible, and it really, _really_ hinders development. I don't see that being significantly solved in the next many years, only marginally improved. This can reasonably be a showstopper for some people. You learn tribal knowledge to lessen its blow (wrap things in modules, use Revise etc), but there is no sugarcoating it: It blows.

* The lack of statically enforceable interfaces is a real problem, and I think this will NEVER be solved in Julia. There just doesn't seem to be any real motivation from the core team, and no agreement in the community on what such a solution even should look like.

* The lack of separation of tests is very annoying as well. That might literally be impossible in Julia since part of its promise is that you can dynamically change definitions that affect other modules. It's what contributes to its great composability. But it also means you need to test ALL of Base if you change one thing.

Thanks for the reply Jakob. I agree with what you are saying.

When JET.jl works it is great. But for large projects it didn't even return when I tried it. I must have waited at least half an hour once.

VSCode seems like it is offically blessed and maybe I should just cave and use it for Julia. I use vim/neovim for EVERYTHING else though. Taking me out of my extremely custom tmux + terminal + vim workflow just for Julia development annoys me unnecessarily much. Do you know if there's work on getting this to work in vim/neovim?

ErrorTypes.jl, InlineTests.jl, ReTest.jl, SetField.jl all are awesome and I've used them in personal projects. I want to see them in the core language. Maybe ErrorTypes.jl is asking for too much, but tests sorely needs improvement. A SetField like feature literally had a PR open for years and then was just closed

https://github.com/JuliaLang/julia/pull/21912

Without these features in the base language, adoption is going to be so slow. Convincing my team to add a dependency is a lot of work, including updating internal documents, best practices, internal training for how and when to use a certain feature, etc.

I don't know how well Julia works with other IDEs to be honest. Much of the hard work of VSCode's extension is editor-agnostic, e.g. the LSP, the type checker and the linter - but whether someone has actually implemented a Vim plugin using these, I don't know.

Good luck with hoping for adopting those features into Base. Even debugging requires its own package in Julia.

But to be fair also Rust is guilty of keeping the base small and leaving important stuff in external crates (serde, tokio,...).

I think it's fine to architect a language in this way as long as there is a good 1st party effort at a comprehensive standard library. The failure mode I see most often of a small core is that there ends up being 100 libraries of varying quality and completeness to do common tasks e.g. parsing JSON.

There is interest in improving testing in Base. Retest seems really nice, but has a few caveats. If you have ideas, bring them up in Discourse/Slack.

Can you please open up an issue on Jet.jl? Even if you can't provide a MWE, it would be helpful to share these experiences

> when using Rust, you can write a unit test function right next to the function you are testing.

I'm not sure that this is a good thing? With Julia you can also write doctests which work great, but having a separate test set as a Julia _project_ offers a lot of flexibility. For instance, we're now developing a Julia wrapper for Playwright to build end-to-end/UI tests which themselves have multiple dependencies. Having a test _project_ allows having as complex test cases as needed, with properly managed dependencies out of source. How would you handle that with an approach like Rust's?

A test project means that you can easily extend the testing capabilities of Base Julia - for Genie we use ExtendedTestSet which actually allows us to pick and run only some of the tests. Take a look at that.

I hope this helps.

> there's a LOT more language features required to help tooling in the ecosystem improve.

Aside from interfaces, what would those be?

I’ll first say that this is my opinion, and that I don’t have good suggestions for how Julia as a language should implement this, just that it should:

* Better language level support for testing

* Better language level support for docstrings - currently doc strings are just normal strings stored in a global struct, and are missing crucial information about the type signature of structs or functions or methods that are being documented

* A way to type hint without using multiple dispatch / interfaces - I want to be able to say this argument has fields x, y, z but I don’t want to use multiple dispatch to enforce this interface. Multiple dispatch is useful with the implementation details of your code depend on the data structure. But I want better static type checking for a handful of fields or methods. Typescript is absolutely awesome at this.

* Support for using @copied s.x = 1 where s is an immutable struct. Users are literally incentivized to use mutable structs instead of immutable structs because that results in shorter cleaner syntax, even though performance drops through the roof in that case. With immutable structs you’ll have to recreate the struct by calling the constructor. In most cases, the compiler will compile as this away anyway but users are going to prefer shorter cleaner syntax when writing code.

* Better macro support. Currently macros have to be valid Julia code, meaning you end up with weird dsl that only kind of sort of matches the domain being modeled. In rust and nim I can literally copy almost any syntax and paste it in a macro block and it’ll just work. This is more work for macro developers and probably will make inference harder but will improve readability.

* Language level support for extending Array or letting users define arrays that are as efficient as the built in ones.

* Similar request for String buffers.

* Rewrite parser that is currently in scheme in Julia, I suspect better tooling will fall out of this

* Support for a subset of Julia that is fast to compile and run and produces small binaries.

I think more generally I want Julia to be a slightly different language than it is ( or possibly ever can be ). Currently a user of Julia needs to be cognizant of too many things that could accidentally introduce type instability. Almost none of these problems are going to be encountered by someone that understands how compilers work / how computers work, which in my experience is not a lot of domain scientists but is a lot of the core team working on Julia. Domain scientists have PhDs or Masters in highly niche subjects after years of writing Python, R, or Matlab but are more often than not terrible software engineers. I can’t tell how many times in almost a decade of being in this field have I seen 10,000 line scripts in a single file without any comments written by extremely smart scientists. Guiding these folk from the get go to make better software engineering decisions is important imo. Currently, if I’m leading a project, I can’t trust a fresh graduate or junior engineer to write idiomatic type stable fast and extensible Julia code, even if they are the experts of their domain. They have to be experts of their domain AND awesome software engineers which is so rare to find. Given additional constraints about funding, writing proposals, documentation etc, it’s honestly kind of amazing that Julia’s libraries are as good as they are.

In my opinion, Julia really shines when one or two people have to write code to tackle a specific problem. Code is super concise and clean and it all fits in your head and everything is fine and dandy. But when you have an engineering team that needs to work with databases, authentication, logging, deployment, scaling, machine learning, data analysis and plotting, and do most if not all of that in Julia … it’s so easy to write something that gets checked in that passes tests and code review that just destroys performance benchmarks.

Some points have simple answers:

* Support for using @copied s.x = 1 where s is an immutable struct.

I believe the standard package for this is this:

https://github.com/jw3126/Setfield.jl#usage

* Currently macros have to be valid Julia code,

There are two kinds of macros, and it sounds like you want `macro foo_str(x)` which accepts anything. But the tradeoff is that you have to parse it, rather than getting an expression tree.

https://docs.julialang.org/en/v1/manual/metaprogramming/#met...

Thanks for all your comments, they've been really enlightening. As a data scientist who sometimes wishes for something better than Python I did have my eye on Julia, but I was unaware of what issues could arise putting Julia into production. Sadly, I think that kind of puts me at square one ... there doesn't seem to be anything, even niche, that can replace Python for our business use cases.

We're a very Spark-heavy shop so I've tried to encourage Scala, but that got no traction, and it would raise its own set of problems. I like Rust but it's simply too complex to expect scientists and data scientists to wrap their heads around; it's hard enough to get them to use Python type hints.

Oh well.

Very good points. Agree with most of them - especially the lack of interfaces. I'm currently working on a ~30k LOC Julia codebase which was mostly written by myself. This is split into ~80 packages within a monorepo. I've never ran into problems with tests taking too long as tests for a single module take a reasonable amount of time to run.

I'd really love a PackageCompiler that would "just work" though. Julia 1.6+ made a lot of progress in pre-compile times. However, nothing beats being able to easily release a binary...

I'm a fan of Julia and have used it and will continue to use it for a lot of my daily work which involves running numeric simulations, but I agree with your critique.

I feel like the ideal language would combine the strengths of Rust (static type checking, great LSP, better threading, better package manager, easy compilation without a large runtime, etc.) with the focus on the scientific computing domain and all the associated trade-offs that Julia has. Maybe it's not possible to have the best of both worlds, but it sure would be nice.

> * Language level support for extending Array or letting users define arrays that are as efficient as the built in ones.

* Similar request for String buffers.

* Rewrite parser that is currently in scheme in Julia, I suspect better tooling will fall out of this

These 3 are all desired and totally doable. They just need some developer time. If anyone wants to help on them, get in touch.

To make those bullet points work, put blank lines between them to make paragraphs.

Thank you. I just did. I’m a tad new to commenting on HN :-)

I feel like if you're going to go monorepo in Julia (which I'm not sure I would, personally), you'd at least want to have multiple modules in that repo. Then you can run tests separately for each module, etc.

We do but tests still take a really long time to run. Sometimes changes are so local, and running compiling and running few hundred tests to check that a 50 line patch works is debilitating.

Ah, gotcha, fair

I'm bookmarking this and showing it to everyone that claims Julia is the next big thing. Kudos for really exercising the language despite all of these painful shortcomings (you must be a very patient person).