For a scientific / academic focused language, I can’t help but feel like blog posts about Julia always seem to evangelize performance in the language in a biased or subjective manner. Anyone else get the same sense?
I love working on performance. It's fun and challenging. Trying to best high scores (times/benchmarks) is fun and gamifies it.
Personally, I answer tons of questions online (e.g. the Julia discourse and Slack) related to things like SIMD because I find it fascinating and want to share this excitement with others. Same thing when it comes to a blog post or announcing a package; I focus on what motivates me most.
I think my comment was mainly that if I were viewing it objectively (for example, knowing nothing about Julia), I think a lot of Julia's evangelizing comes off too strong. This is definitely not directed at you btw, but just that the community as a whole is just overly positive and ONLY positive.
For example, if you search for Rust on hacker news or even browse /r/rust, you'll need 80-90% pro rust, but you will also definitely see another 10-20% of posts being very critical about Rust. And these critical posts are even from prominent members of the Rust community. Additionally, there's a roadmap for Rust posted very often.
In Julia however, I can count the number of "negative" Julia posts I've seen on one hand. Don't get me wrong, I really like Julia, but there are A LOT of things that need to be improved that I know about and probably more that I don't even realize. It would be nice to have experts post and comment on that, and the core Julia members describe how they are going to tackle that. IMO, that discussion does happen, but it is largely on Slack and forever lost to Slack's history.
I've definitely been making an effort to temper any discussion of benefits with associated drawbacks.
Though, I just want to express WHY I'm so enthusiastic. I've used both R and Python for DS. I'm familiar a bit with matlab.
I've played with Keras and Pytorch.
Aside from all the technical and productivity benefits Julia just feels GOOD to write in, compared to all of the above. It's a real quality of life improvement + a feeling of freedom in that I can express my ideas, compose them with others all without worrying about a different compiler or framework or array type system or having to code units in c++.
This is freeing and honestly, it would be sad if it's relegated to niche status forever. There's definitely a selfish component here, but I'd like everyone to feel the same benefits.
It's hard to go back to pandas or tf or pytorch after using chain.jl, dataframes.jl and flux.
Yes, there are drawbacks with compile times and things like that, which can be annoying. But I keep working in julia because the benefits are worth it, those aren't intrinsic to the language, and most importantly, there's a roadmap to fixing them: https://discourse.julialang.org/t/precompile-why/78770/8 with progress that is tantalizingly close to the end goal of easy dev and easy deployment.
Maybe this my biggest gripe when someone claims Julia is objectively better than Python or R. The answer to this question is highly subjective, heavily depends on whether you use VSCode or not, and people that usually answer this question are heavily invested in getting other people to use Julia.
But the answer is usually presented as an objective answer, and I guess that just kind of rubs me the wrong way.
As someone who spends 98% of his time in Python and the remaining 2% in R I'd love to switch to a more nicely designed, ergonomic, high-performance language. Adopting Julia at work is a nonstarter for now, but what really holds me back from learning it is how thoroughly academic all the evangelism seems to be. Most working data scientists aren't doing things like physics simulations and won't be terribly interested in solving differential equations; even in grad school the topic rarely came up for my econometrics research.
To put it bluntly: reading about Julia kind of makes me feel a bit dumb, not excited.
When I do work with neural networks, the networks just aren't going to be small because the value in them -- for the kind of work I do -- is in their capacity to use large amounts of data to find useful representations. So the benefit of Julia over PyTorch described in the link is only interesting to me in a "huh, I guess that's a bit cool" kind of way.
I, speaking for myself, will keep this in mind next time I set myself to gushing about Julia.
You mention r/rust downthread - r/Julia is quite inactive in comparison. However, the places where Julia discussion actually happens - Discourse, Zulip, etc. - do contain a lot of criticisms, wishlists from other languages, roadmaps for improvements, etc. But these are not places you would randomly come across if you're not involved with Julia, like a subreddit would be.
And the scientific/academic focus of the language that you mention is actually another reason this happens. When the language's users are (primarily) developers, they (we) nitpick and think about alternate designs and blog about them for no reason at all. The average Julia user instead would just ask in the discourse/other forums, maybe complain, and then move on with their research/engineering problem. So the blog articles that end up getting posted here are most often by the core language/package developers announcing new features, breakthroughs, and other positive news.
Part of what Julia offers is that you need less tweaking, and the whole stack is in julia so it's quite hackable.
An example I've asked about previously: When comparing a recursive implementation of an LAPACK operation with OpenBLAS, is that using the RELAPACK(?) implementation in OB, and if not, how does using a similar algorithm in C or Fortran compare? Generally, I want details of measurements.
I’d quite like to like Julia, but that’s one of my sticking points.
I think Julia is stuck in a spot where they are better than say R/Python for scientific computing on a fundamental language structure level and package manager level, but that it's not good enough to make up for the fact that those languages have a more robust ecosystem.
I'd argue though that this mindshare is not there due to Python/R as a language (and platform) being better, but simply because there were no better alternatives 10-15 years ago and by now the sheer inertia makes it impossible to stop.
You'd need to convince a sufficient portion of people to move to Julia at more or less the same time. Few people want to be first movers. These tend to be the ones who actually care about the qualities of the platform, not just "get the work done and clock out".
If you are a data scientist, you won't be paid for moving to Julia. You'll be paid for coming up with working models. And you take a serious risk by moving to a new platform with little adoption. The platform might die, taking your tools and processes with it. You won't be able to rely on your colleagues advice about technical issues. You can run into bugs more frequently simply because fewer eyes have looked at the ecosystem.
The end result is that few people take the plunge.
Why aren't we moving, you ask? Because moving to another ecosystem does not put bread on the table. I cannot go to clients and say: this past 6 months we made no improvements to our strategies, but look, we migrated to a new programming language that is used by a fraction of a percent of our peer group.
The field attracts clever people and there are many greenfield projects. But just because we start a new project, it doesn't mean that doing it in a tiny ecosystem is sensible. Some firms can do it. Jane Street has the means to basically be "OCAML The Systematic Trading Language". This is not a luxury afforded to most.
And thus we get a chicken and egg problem, where nobody wants to be the sole first mover as there's little advantage to it. At the same time, we all see that everyone would be much better off if we moved.
Julia still has a lot of empty library space compared to R or Python, and isn't perfect, but my guess is it will catch up. R did when it was being compared to SAS, fortran, C/C++, lisp, and so forth.
I'll be honest and say that I wish something else more general-purpose (to the point of say, having a bootstrapped compiler) would be in its spot but I can't really complain right now. Maybe something else will catch up.
I think for me personally is that R and python is a bit in denial about its performance limitations when it comes to hard problems. You pretty much have to drop down into C/C++ to address them, and for certain things, julia really does do many times better time-wise, without the weeds of C/C++. I think there's some people (myself probably included) that are tired of being forced to choose between the C/C++ and R/python worlds. I think there is a bit of overhyping and/or cult-like nature of julia but I also think some of it is trying to convince people that you don't have to choose between expressiveness and performance.
It might not be a perfect relationship - there will be lagging effects and other factors, it will vary based on the specific task, and some of it will be subjective - but you can’t pretend that there is no relationship at all.
There are lits of external factors like costs, available support, availability of compilers for certain hardware, support of the existing tool chain, learning curve, career chances etc.
Python and JavaScript aren't the best but sufficient enough
This isn't true. Sure, you can't unilaterally pick whatever language you want when you work with other people. But for every piece of software ever built somebody had to decide what to build it with. And even though Julia has existed for 10 years, almost nobody is picking it over the alternatives.
You need to gather a certain critical mass of users to get popular. Otherwise Java , JavaScript or Python would have been replaced already.
Most of the time accessibility beats performance.
What do you think why Visual Basic was so popular and why Python is now?
Java started with a first implementation in 1991, but I didn't find it usable for serious production until about 1999-2000. That isn't quite 10 years.
10 years from inception to mature-enough-for-production seems to be kind of the rule. Building support systems and a community is hard, hard work.