PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly) | Hacker News Reader

PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)

dev-discuss.pytorch.org

265 pointsthetwentyone283 comments

Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust.

Rust has been eating C++ lunch. Same rapid rise of ecosystem story.

Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in Julia.

Julia is great for numerical computing, it needs faster time to plot and more hands in the ecosystem. The former will be solved and the latter seems inevitable to me. Pitch in!

Wow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too.

I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.

this is just false. Julia has partir based threading (like go).

> I'm not sure what advantage Julia

While I'm really not a fan of 1-based indexing, Julia's multiple dispatch is not something easy to match in Python.

[EDIT]: one thing that's still not solved in Julia is code startup time.

Many people will sell you some sort of workflow that works around the problem, but it's the same old tired arguments people would use to defend traditional compiled languages, and I'm not buying.

I really wish they would find a way to truly solve this.

> [EDIT]: one thing that's still not solved in Julia is code startup time.

> Many people will sell you some sort of workflow that works around the problem, but it's the same old tired arguments people would use to defend traditional compiled languages, and I'm not buying.

I mean, Julia has a REPL so you can basically edit code as the process runs, which definitely makes startup time less of an issue. The fact Julia can produce such fast code is also pretty nice.

Starting a new process every time you want to run a snippet of code isn't getting the best out of a dynamic language...

I would say the almost every version of Julia 1.x has better in terms of code startup.

as in 1.7 > 1.6 > 1.5 > 1.4 > 1.3 > etc...

it's especially goten way better since julia 1.5, so really mostly in the last few years.

In julia 1.8, what's interesting to me is that the julia runtime will be separated from the llvm codegen; https://github.com/JuliaLang/julia/pull/41936

the immediate effect is to allow small static binaries without a huge runtime (namely the LLVM ORC), but the side effect is probably that the interpreter will also get better in cases where you don't want JIT.

Most definitely not single threaded. From one of my codes

# Threaded inner loop, each thread has no dependence upon others

  Threads.@threads for t=1:Nthr

        inner_gen_cpu1!(psum,ms,me,cls,2)

  end

That's all you need. You don't need pthread create/join, you don't need installable language extensions, you don't need to appeal to external tools/libraries to enable threading. Its built in to Julia. And it is trivial to use.

But is it real kernel threading or the same kind of cooperative threading that Python also supports?

It's real kernel threads. Julia also supports cluster computing too.

According to this discussion thread, Julia requires the number of kernel threads to be specified at startup: https://discourse.julialang.org/t/does-multithreading-requir... This seem to be more like pyprocessing and much more limited than the pthreads interface.

I didn't go deep on Julia's multithreading, but what he is saying is that Julia uses an MxN threading (I think nowadays if you don't specify it at startup it will just use one for each cpu thread), which is the same as a language I did most of distributed programming (elixir/erlang), and as far as I know it's the same as Go.

Having 1 kernel thread for each CPU thread means that your program can use all available CPU threads at the same time (so you get all the parallelism available within the machine), and having a language based scheduler for each thread means you can have minimal overhead (no need to do a system call) to create a new concurrent execution (meaning lightweight/green threading similar to what python allows, except being automatically distributed by the language within all kernel/cpu threads). In Elixir this means you can create millions of processes even though the OS will only see one thread per logical cpu thread, and I never felt the limitation of this abstraction over multiprocessing (of course, Julia is definitely nowhere near as mature - and maybe never will due to stuff like preemptive scheduling and parallel garbage collection that is easier to implement in a language with only immutable types, though it seems to be moving along, and in Julia 1.7 the processes being able to move between kernel threads solving the issue mentioned in that discussion you linked).

But that scheme is the same as green threading and has the same faults. Start Julia with one system thread. Run one infinite loop in one green thread and another in another green thread. One of the loops will run while the other will wait until the other completes (which it never does). This is inferior to Java and C# which both uses system threads by default, allowing both infinite loops to run. Erlang/Elixir has the same problem. You can run thousands of green threads, but if one of them is stupid enough to call a blocking C function then all others have to wait.

That issue actually doesn't happen with regular elixir, since it's immutable and stateless, the scheduler doesn't wait for the process to voluntarily wield (it's always safe to switch, so it just gives some fixed time for each process, which makes it very low latency and very reliable but not very efficient at any individual task due to all the switching). Calling other languages from erlang/elixir create dirty processes that can't be scheduled this way and may cause those issues.

Since I didn't use dirty processes in elixir it did make me forget about this obvious issue you pointed out, that for a mutable language like Julia can happen in every thread, but that's not something that limits the expressiveness of the model, but something that requires consideration to avoid while programming and language level mechanisms to protect the thread (at the very least the ability to define timeouts that can throw an exception on any spawned process) or maybe a future framework on top of it that handles this in a safer way (something like Akka). I can only hope Julia can achieve the full potential of it's multithreading model.

> Start Julia with one system thread. Run one infinite loop in one green thread and another in another green thread. One of the loops will run while the other will wait until the other completes (which it never does). This is inferior

If you want julia to use multiple system threads, why are you suggesting one not use system threads for this test? All you have to do is start julia with multiple threads and it'll use those threads for your infinite loops.

Python threads are built on top of "real" preemptive OS threads. The presence of the GIL does not change that.

GIL can be released and it is released e.g., for IO, during heavy computations in C extensions--Python program can utilize multiple CPU cores.

https://docs.julialang.org/en/v1/manual/multi-threading/

As has been pointed out, Julia is not limited to a single thread. It's actually got a pretty good support for parallelism both at the thread and process level.

And no, Julia is not too similar to Python. Julia has multiple dispatch, Python does not.

Another point that is often lost is how mathematical Julia code can look, especially for matrix functions.

Yeh this is superficial, but so are 200 dollar sneakers and they do just fine.

Honestly there is a real pleasure writing code that looks like it could be on the blackboard. The numpy / numba world in Python just feels... not great.

I actually consider that a negative. Mathematical notation is fine when you're dealing with limited space on a blackboard, but if you're programming please name your name variables and take the time to model your domain explicitly.

I don't understand - what is wrong with condensing the python representation `sum_of_lambda_for_each_psi` to `Σλ∀φ`? It seems that 4 symbols is much quicker and easier to read than a slew of snake case stuff.

For each phi?

I'm not completely sure that you aren't joking. Maybe for a mathematician the latter would be "very readable"?

Indeed it is (slight improvements not excluded). I would love to be able to write things like M<sup>T</sup>A (w/o the HTML rubbish seen here - flavour of the same problem) for two matrices directly as code (can Julia do that?). Instead, I've been forced to learn row and column precedence in OpenGL, DirectX, Numpy, and whatever, paying attention to it in endless constructs.

Even after 25 years in the business, it's still a pain.

What about everything that isn't yet represented by widely-adopted mathematical notation? I'd hate to have to learn which Greek letters correspond to which operations every time I dive into a new codebase. The advantage of English words is that everyone knows them.

Edit for details: `Σλ∀φ` is a chore to read for me. These letters only make sense if I've absorbed the author's notation beforehand, like in a math proof. I parse the snake case version instantly, because my brain is trained for that. And I understand it much more quickly, because it says exactly what the function does in the way I would represent it in memory.

We’re talking in a context of scientific software here.

> I'd hate to have to learn which Greek letters correspond to which operations every time I dive into a new codebase.

You have to do that anyway, because you’ll have to connect the code you’re working on with the scientific paper describing it. When the code uses variable names too far removed from the mathematical symbols, you have to do make two steps: figure out how the words connect to symbols and then figure out the symbols. This will be especially difficult for the mathematician/scientist without a strong coding background: they’ll have much less friction reading code that matches the symbolic notation and Greek letters that they’re used to.

> The advantage of English words is that everyone knows them.

Not when it comes to “math” words.

> `Σλ∀φ` is a chore to read for me.

Right, but that’s because you have a different background. For me, `Σλ∀φ` is much easier to read and understand. More importantly, a symbolic notation is much denser, which allows to parse long expression that would be very hard to understand if they were written out in words.

Again, this is for the very specific context of highly mathematical/scientific software that Julia excels at and is primarily used for. In a more general context (when the software isn’t a direct representation of a scientific paper), I’m 100% on board with good, descriptive variable names

How about single-letter Chinese labels? because that is my background and that is what I am comfortable using.

If that’s what’s most appropriate for the field you’re working in, sure, why not?

Of course it will limit who will be able to interact with your project, which can be a good thing or a bad thing. For the math-unicode in numerical software, you may not even want someone without at least a minimal math background (enough to understand Greek letters and math symbols) working on the code. Likewise, a project that’s inherently Chinese, where it doesn’t make sense for the users/developers not to know Chinese, should feel absolutely free to use Chinese symbols in their source code.

On the other hand, if you do it gratuitously, you just unnecessarily limit collaboration. I’m ok with that, too, personally: it’s your own foot you’re shooting.

I also don't see Chinese symbols as fundamentally different from non-english ASCII. It's pretty common for Physics bachelor students to show still naming their variables and write comments in e.g. German (my teaching background). I'll push them to kick that habit pretty quickly and switch to English, but this is no different from them still writing the bachelor thesis in German and then switching to English for the master and PhD: there's nothing really wrong with it, but you won't have any reach within the scientific community if your work is not in English.

There's really nothing wrong with beginners starting out in their own language. Why shouldn't a 14 year old Chinese kid write their first programs using Chinese characters as identifiers? I'd much rather have a language support full Unicode they way Julia does than to force everyone into ASCII.

> `Σλ∀φ` is a chore to read for me. These letters only make sense if I've absorbed the author's notation beforehand, like in a math proof.

It's a hell of a lot better than reading `Sigma`, `lambda`, etc. and having to do both the verbose English -> character translation in your head while trying to understand the mathematical form. At the end of the day, you do need to understand the math in order to understand mathematical code, and making it have as few translation steps really helps.

As a mathematician by training - even with a now almost full career in professional software development - it's simply one of the handful of levels of abstraction I like to switch between. Much more concise then than usual code. I did all these things, massaging bits down to machine code, designing systems on the other end. Math has its place there, and if the toolbox supports it, I'll consider....

I remember that one of the first things I did when learning C++ was to implement a matrix library, operator overloading and all (oh, how naive when it comes to getting it right!).

This is a valid question. Others have provided a much better answer, the thing I want to add is that being able to express computation in software the same way as in research papers makes the code much easier to read and reason about. This language was invented a long time ago to express computations concisely, and today you will see a lot of math built on top of these abstractions (since they’ve been around for sooo long).

So the ability to express computation in standard mathematical notation rather than having to invent pseudo symbols to do it makes it much easier to read for people who already have that training. And for people who don’t… it does require pre-reading to understand how these symbols are used but you have the benefit for hundreds of years of math literature and community to look it up!

So, one-letter English labels were deemed bad coding-style for decades, but suddenly one-letter Greek labels are good because some X-language (Julia) supports it. Seriously? How about single-letter Chinese labels? because that is my background and that is what I am comfortable using.

well, you can always program in wenyan: https://github.com/wenyan-lang/wenyan

But seriously, it's 2021. We are no longer slave to ASCII, or even English.

Most of the newer math oriented langs have heavy use of unicode (julia, agda, lean, etc).

One-letter variables aren't always bad coding style. Some single letters are meaningful. It's conventional to use 'i' for an array iterator, for instance - or i and j for two dimensional iteration. x,y, and z are cartesian coordinates. It would be downright perverse not to call the base of the natural logarithm 'e'.

And so it is with greek letters. Many of them carry intrinsic meaning, or a loose collection of related meanings - they effectively are labels. It's not that people want to write dense and impenetrable code, it's that they would rather write μ than coefficient_of_friction because that is its proper name and how it appears in formulas and textbooks.

Completely disagreed. I was skeptical originally, because I wasn't used to seeing unicode symbols in code, but in certain types of mathematical code, it makes it /far/ clearer because your brain can link to the appropriate symbols in textbooks.

To me, it's a matter of familiarity. clearly human brains can process large amount of symbols. Just look at some east asian languages. Historically, there has been an ASCII bias in computer languages, but that is history.

Mathematics is a /language/, and one of the most universal in humanity because it models certain human reasoning.

To your point, for personal projects in Elixir I’ve started using Japanese characters for function names because they can be so much more terse.

This is really useful when relating Julia code to a paper or providing it as part of a paper - there is very little lost in translation.

Julia is much, much faster than Python. For numeric code it's almost as fast as C++. They've used lots of cool tricks to become crazy fast on numeric workloads. It's also definitely got support for multithreading, clusters, etc...

For numeric code, comparing to pure python makes no sense, since people use numpy+numba to do that. Is julia much much faster than numba? I don't think so :)

yeah, then you try to use numba + scipy and then sure many things work but you're never too far away from a tableflip and wanting to curse your fucking life out.

Probably not faster in an absolute sense but things like loops in Julia can be properly optimized and will sometimes be more readable than structuring your program entirely around NumPy constructs.

For numpy, it's correct. But numba can optimize the loop. So optimized and readable loop is not an advantage of julia compare to numba.

Not in real-world contexts. This is spelled out in Julia for Biologists (https://arxiv.org/abs/2109.09973) which does the operation counting to show why using Numba with SciPy is still an order of magnitude slower in scientific operations like solving differential equations compared to Julia. An order of magnitude on widely used scientific analyses is pretty significant!

Julia is generally in the same ballpark as Numba (depending on the application they should be within 2%). The difference is that Julia is a full language, while Numba breaks if you try to use it with anything else.

Yeah, Numba usually delivers on its performance promises when used right, but can be such a huge hassle oftentimes as to make it not really worth it.

Yes, Numba is just barely less awful than managing a C build process.

Julia is a much better solution to the two language problem, here's hoping it can overcome the ecosystem inertia.

I do like JS quite a bit, I was a hater until a few years ago when I had to start working with it more than a snippet here and there, but I still prefer python on the backend. I think JS makes a lot of sense for frontend with the way everything is async.

>rapid replacement of backends in JavaScript

Really? I thought that was a ~2017 thing.

Ha it's still going strong. Now bifurcated between node+next and the rise of Deno.

It won't stop either because the road between between JS client dev and JS server dev is so smooth. Path of least resistance type thing.

It’s not really bifurcated. Hardly anyone is using demo.

I think Slack is using deno now.

https://api.slack.com/future/tools/cli

One has to be in a pretty tiny bubble to believe there’s rapid replacement of Java backends with Deno, of all things.

That was my first thought in reading the "JavaScript backend" thing.

I don't know, x being a potential replacement for y doesn't say all that much. You could have been writing Java or C++ for the past 25 years and got loads of stuff done, solved problems, shipped software, made money etc.

Languages are fun to think about but you don't always need to be concerned with every vocal minority of programmers that like to talk about how their language is better than yours. Sometimes that replacement is better and sometimes those people are wrong. But even when they're right, being marginally better isn't that big of deal, or nearly enough to make for a viable rewrite or change of language.

I wholeheartedly agree. X instead of reasonably-similar-Y isn't convincing in and of itself. I'm not even convinced Julia will be the dominant numerical programming language.

That said I'd like it if it develops a robust and large ecosystem because I personally like coding in it. It has built-in matrix ops, parallel ops, dynamic dispatch etc that are really nice to work with in the numerical space. Like Matlab but well rounded and fast.

So I admit my comment is less argument and more cheerleading. "Hey folks let's make this the case so us numerical people can have a slightly improved experience".

In the grand scheme of things this is as noble or ignoble as any.

** like Matlab - but free and open source ** Perhaps an important aspect that allows it to be relevant

I think Python is used by a lot of people now because elementary schools and universities teaching it, easy to start with and you can gradually convert to more advanced (compact) language, you can prototype easily without having to remember a lot of syntax, and you come a long way without an IDE (+ Jupyter Notebook is great).

I love that Torch is converted to Python. I think it’s more importantly with a large ecosystem than the most efficient language.

Choice between Java and native languages (for the backend at least) is obvious due to the performance concerns. Same with Python/Julia.

For Java that's mostly a myth. Modern Java isn't measurably slower than C++ in many cases and faster in some.

Nevertheless, every time I run JetBrains IDEs it works much slower than QtCreator, for example, and eats way too much memory compared to the native equivalents. Hardly a myth if I can see it with my own eyes. Same with Ghidra vs Binary Ninja or IDA Pro. And so on.

What about memory usage?

This might have been the case in the past, but the JVM is now good enough that you get very close to native (and sometimes better) performance with Java.

> it needs faster time to plot

Time to plot is much improved in 1.6 and should continue to improve in 1.7. It's definitely being addressed.

Python’s strength isn’t being the best at anything. Its strength is being top 2/5/whatever in a ton of things. Moving to julia for this just trades some pain points for others.

100% this. I love that in python I can handle most all problems reasonably well. I don't need perfect, I need a good, versatile language that I know how to code. Even if Julia is better in some things, the switching cost of learning the language, libraries, nuances, the massively fewer online resources, just is not there.

I've been using C++ and Python since the early 2000s. Julia these days is my go to free time language to hack together stuff in, partially because I've been impressed with what comes out of that ecosystem despite the smaller community. I think this is at least a testament with how composable things are relative to C++ or python.

I love many things about Julia, but there are definitely growing pains for the language and I can see why it may be a hard sale to people used to modern python or C++, both of which have improved since Julia came out. However, Julia has it's own goals that might make certain communities veer towards it. It'll be interesting to see if it gets more adoption in industry.

The way I see Julia now is how I saw python ~12 years ago

I agree, but it's probably closer to 20 years IMO, which makes sense given that Python has been around for 21 years longer. By 2009 Python was already in widespread use at Google/YouTube and (I think) Mozilla, plus startups like Reddit and Dropbox. Hell, Steve Huffman wrote "Lots of people have written web applications in Python, and there's plenty of code from which to learn" as one justification for Reddit's Python rewrite in 2005 [0]. No one is saying that about Julia today, it's seeing very little use in the private sector at this point.

[0] http://web.archive.org/web/20051230163903/http://reddit.com/...

I agree, personally I think it's a problem with Julia's initial "product" placement, which was probably a overfocus on the "technical computing" aspect.

Personally, the first version of python I used was 1.5.2 (in 1999). I learned it initially instead of the current 'scripting language' hegemon, which was perl. Since CGI was in extensive use at that time, this was the start of the python web backend ecosystem.

In the late 2000s, I was able to sneak in Python rewrites of MATLAB or Fortran code written by scientists, even in situations where Python was not officially sanctioned for that type of work (I was mainly a C++ developer, but would also write python bindings). I feel like it really was the strength of python's strength as a universal ducktape that caused the ecosystem growth of both the scientific _and_ web stacks.

Nowadays, I've really come to appreciate Julia from a novelty standpoint, and I'm intrigued at what it can do, especially in terms of probabilistic programming and other emerging fields, but in order to justify using Julia in more general 'professional programmer' settings, it needs to focus on what's made modern Python so successful. Otherwise, I could see it's fate as "merely" replacing a more niche language like MATLAB (which Python has /already/ managed to do for the most part).

Julia has a decent amount of penetration in the private sector, just not in "tech". If you look at JuliaComputing's clients, you'll see a lot of private customers, they're just not the type that typically talks a lot about the tech they're using.

I'm glad you qualified this with 2/5/whatever. Usually people say #2 in everything, which is of course wrong.

Python more like #100 in terms of speed, #100 in terms of correctness, #100 in terms of sound abstractions, #10 in terms of readability for large programs. Its real strengths are quick hacks and a decent C-API.

The community is smug, conceited, does not value correctness and in general is intoxicated by Python's undeserved success. Many posers and incompetent people.

I think Python is great for exploration and method development, while I would prefer something more efficient like Julia for production systems. On the other hand, Julia requires compilation while Python allows open-heart surgery on the production system (for the masochists among us).

I personally don't like Python that much because every library does things differently and sometimes it feels like learning a completely new (sub) language. E.g. NumPy DataFrames allow to do the same thing in multiple ways (e.g. adding an index column, or removing a column). Often when I need to look up how to do a particular thing I end up finding many solutions that simply don't function with the version I am working with. Sometimes looking even into old code of mine doesn't work any more and requires either me using an older library of relearning how to do things.

That being said, a friend of mine has been quite fond of Julia lately. Which put Julia on the top of my list of programming languages to do a deep dive.

Just to be clear Julia is "AOT-JIT", that means that the methods are compiled when used the first time. With Revise.jl (a package for interactive development) you can make a fully interactive Julia process, rewriting things on the fly, while benefiting from fast code

Running Revise in production sounds unwise (sorry for the pun).

Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).

You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.

In certain common lisp deployment you might do that.

There are legends from Common Lisp users where this is very much done.

> When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone.

> Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun.

Taken from https://sep.yimg.com/ty/cdn/paulgraham/bbnexcerpts.txt?t=163...

I know that this is not possible in Julia like that. But just because it's not possible in Julia doesn't mean what they did was somehow wrong when it worked well for them.

Call me a cynic but I would hope almost no devs had the power to live update the production codebase. That sounds crazy to me. To me, that kind of thing should only be done with (approved/verified) versions which are cut and tested before deployment, and deployment should act like an unchangeable immutable compiled binary until the next binary is ready. Not only is this legally required in many scientific cases, but also production should have a lot more care.

This kind of thing sounds fine for a dev server though, in which case Revise+RuntimeGeneratedFunctions will get it done in the cases I've seen.

I understand where you’re coming from, it’s the production robustness paradigm of someone who has worked with such systems and knows what kind of controls are required to release code. eg live editing like that would almost certainly go against SOC2 controls.

However, I’m also tempted very much by a system that allows for fixing things so quickly, because it provides for a delightful customer experience. Most customers don’t report most issues, only when something is persistently wrong or is something they can’t get around. So being able to quickly fix it this way seems genuinely amazing. If I were to take this idea further… maybe some sort of customer isolated canary deploy/ feature flag might be the way to express this customer experience in todays world.

The problem is always that you might break other things with your fix. Maybe a battery of testing before canary deploys can raise confidence about the validity of the fix.

most famous one being NASA JPL remote patch

This is a binary view of development-verses-production. I appreciate languages that treat this as a continuum and let me make my own trade-offs.

Common Lisp, Smalltalk, and Erlang all treat the process as running indefinitely. This gives those languages a certain something special.

I was hoping to find the same thing in Julia. It's not there. It seems like a significant missed opportunity to me. These dismissive comments make it sound like you guys might not know what you are missing.

We legally cannot do things like say live code change the production system underlying the analysis of ongoing clinical trials without going through the whole software development lifecycle required by the FDA (I.e. the Pumas example again), so even if it did exist all a lot of the production systems we're running legally couldn't use it. But also, you can do it with Revise+RuntimeGeneratedFunctions, obviously that solves the mentioned world age issue so I don't see what the fuss is all about? I don't see how offering a solution is dismissive?

but you can adhere to legal requirements and go through all the code and test reviews and still have live update capability. i don't see why they are exclusive. moreover, there are scenarios where taking the system off-line for an update would cause a critical failure

Okay, so then use RuntimeGeneratedFunctions+Revise. Done. Julia can do it, but having a setup so that people can do instant review of changes on the live production system is still a little bizarre and I wouldn't do it even though we could.

i was just explaining luke's point. i think he was hoping that julia would do more to enable interactive development a la common lisp/smalltalk

Yeah. This forum doesn't seem like the best place to have the discussion, alas. Maybe the corner of a room at JuliaCon with a few Lisp refugees one day :).

Coming from a Lisp background Julia fits like a glove in the beginning but some aspects, like the inability to redefine structs, I just crash into like a brick wall. It makes it hard to imagine writing (say) a Blender/Paraview type of application in Julia.

I suppose that Julia is heavily influenced by Scheme and Dylan but not especially by Common Lisp (or Smalltalk) where the idea of being unable to redefine types at runtime is jarring.

I wonder if Julia advocates would do better to just acknowledge that this point of view exists, even if experience with Julia might change it over time, rather than dismissing it (yes, very much so) as empty "fuss." But there aren't that many Lisp/Smalltalk hackers in the world so you can probably afford to alienate them if you want to...

It's acknowledged. The full redefinition of any function in any context was solved by RuntimeGeneratedFunctions and we use it extensively throughout SciML, ModelingToolkit, Pumas, etc. so it's fair to say struct redefinitions are the only thing left. That said, "struct redefinitions" are done all of the time in many contexts: if you know you may need to do this, you can just use a named tuple which acts just like an anonymous type and obeys dispatch on a named form that you can then hijack at runtime. With smart engineering then you can be redefining everything on the fly, and we do this through MTK quite regularly. That said, it would be nice to do the last feat of full redefinition of non-anonymous structs, and the latest proposal for how to do struct redefinition is https://github.com/JuliaLang/julia/issues/40399. So we both built and provided solutions for how to do it, showed how to do it in libraries, tutorials, videos, etc. How is that not acknowledging the point of view and doing something about it?

You might want to try and acknowledge the other point of view where I note that, hey, we did make this work but it was a smaller deal then we thought because we legally cannot employ it in many production contexts. We're still going to work out a few details for fun and ease of debugging, but given that we have extensively looked into and thought deeply about the whole issue, we have noticed that the last little bits are less useful than we had thought (at least in the contexts and applications I have been discussing, like clinical trial analysis). That doesn't mean we won't finish the last pieces, but given how much of it you can already do and how many teaching materials show how to do work around the issues in any real-world context, and how little of a real-world application the last few bits have, it shouldn't be surprising that the last pieces haven't been a huge priority. So instead of looking narrowly at one factor, I encourage you to take a more global view.

I'll concede that there seems to be a lot of work done on this in Julia that I'm not aware of. Thank you for the references, I'll check them out and then gladly concede the rest the next time this topic comes up.

Clojure has a high performance data frame library that leverages new JVM vector API and high quality apache arrow protocol.

Talk related - https://youtu.be/5mUGu4RlwKE

https://github.com/zero-one-group/geni-performance-benchmark

> Java has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang.

I guess in some startup scene, it has been Java and .NET over here and no signs of changing, despite the occasional junior projects that eventually get rewritten back into Java/.NET stacks when they move on.

This seems so silly to me.

It’s PyTorch-if they said “the next version of PyTorch will be in Julia, the ecosystem would shift accordingly.

They’re practically saying “this language has every feature we need and want, most of them already existing, but we’re going to continue re-inventing them in this objectively less suitable language because we clearly wish to make life harder for ourselves”

Or I read it as "We want to make life as easy for our userbase as possible, so we will put more work on ourselves to make our users lives easier" which is an attitude I very much appreciate.

agreed, and this always been the driving philosophy of pytorch, and perhaps why it kind of won so much brainshare against tensorflow despite _long_ odds when torch was ported from lua.

Soumith Chintala had a keynote talk in juliacon where he focused on these points; https://www.youtube.com/watch?v=6V6jk_OdH-w

were those odds really long? TensorFlow is a beastly mess; and there were some very serious breaking issues between minor version revisions, it was so bad that I used to work at a company where we joked "this company only exists because people can't install tensorflow". We also used to joke that in order to install TensorFlow, step 1: install the JVM.

Exactly.

PyTorch is not only easy, but is a joy to work with.

Among researchers, TensorFlow is rapidly losing ground to PyTorch, and, I think, will keep losing ground until it becomes a niche and only used by Googlers and some others.

https://horace.io/pytorch-vs-tensorflow/

In the long run I think moving to Julia would make a lot of sense.

I have used MATLAB, R, Python and Julia extensively for doing all sorts of data related things during the last 20 years. Julia is incredibly easy to work with, very elegant and really efficient.

R and Python have always felt clumsy in some ways, and hard to write really performant code, even if you are more proficient in Python! As a seasoned Lisper and MLer, even after having a lot of Python experience in my belt, Julia felt much easier to work with from the very beginning.

Furthermore, most Julia libraries are written in pure Julia which simplifies things a lot and enhances composability. While there are great libraries around, the DL landscape is a bit lacking. Flux is great, but I would not use it to build e.g. transformers as it changes too often and has too few maintainers behind it. Hence a potential migration of Torch to Julia would be fantastic.

Flux dev here. There's Transformers.jl which has prebuilt transformers built on top of Flux. While the package does change, we have been more careful about ensuring we don't break code all that often.

But Julia doesn't have a good story for anything else besides that.

You can take a Python web server process, have a request call a task that uses NumPy and OpenCV and scikit-learn, get that back, and you're done, all in the same language.

Julia's community does not seem to have aspirations beyond high-performance math code, which is great for its use, but I'm not going to learn Julia just for that when I can implement the entirety of a development pipeline in Python and have all the other niceties that come with it.

That's just not true.

https://github.com/GenieFramework/Genie.jl https://github.com/JuliaWeb/HTTP.jl https://www.youtube.com/watch?v=xsxJt4prFG4

And with upcoming improvements to binary size and structured concurrency (it already does go-like lightweight threads) it will get even better.

A notebook doesn't make for a line-of-business web app.

And the two other links?

I don't know why I didn't see them. I read through the docs with Genie; it seems to be about where Django was 12 years ago as far as feature development goes. Enough to be very productive for some use cases, not sufficiently productive to consider it a step up from the existing tech that's out there.

Does keeping as much of the codebase as possible in Python (or keeping the fast parts in C++) actually make things easier for the userbase, or do they just care about having a first-class interface in Python regardless of the implementation language?

Almost certainly the latter. Python excels in this because it’s really easy to learn, so non python libraries provide python functions that you can call which may not be implemented in pure python.

Language choices are less about language to me and more about ecosystem of libraries. Python has generally been very strong in the ml/data science realm. I know Julia is catching up but am unsure just how much it covers. Example of python libraries I would consider needed as part of the data ecosystem, numpy, pytorch/tf, batch and stream processing frameworks like spark/flink/beam, workflow orchestration like kubeflow/airflow, data formats like pyarrow, etc. My company does most of our ml in google cloud so gcp libraries are also quite helpful. How much of that does Julia have equivalents to? How much coverage do those equivalents have? We’ve done some ml things outside python before and one general issue for most languages is leaving python there’s a high risk that something will be missing. Because of that if we use another language it’s preferred to keep scope of it’s project small. I think only big exception is on the data engineering side sometimes Java is better as a lot of batch/stream frameworks have best coverage in Java, although ml libraries is weaker there so our usage of Java is mainly data pipelines.

Another issue is pytorch/tf in python are very dominant in research/projects. Often we clone relevant recent projects and try experimenting with them to see if they help. Swapping to Julia would hurt a ton in that area.

edit: Also while I'm fond of python I'd be very open to seeing another language win. There are language design choices I dislike in python, but I like enough of the language and ecosystem as been too strong to leave most other languages worth pondering. If Julia grows enough that my coworkers start asking for Julia support I'd be happy to explore it. My pet preferred language is crystal (ruby like readability + types + good performance) but ecosystem wise it's tiny.

Python is the middle manager of languages. It sucks at everything, but always knows a guy.

I saw a quote once that stuck with me: python is the second best language for everything, first at nothing.

What are the best languages for "executable pseudocode" or "glue language" roles?

And there's 1 guy he knows that's a 10x developer and always willing to take one for the team.

I'll leave it to HN to figure out what that means :P

Call native libraries?

Write a metric shit ton of unmaintainable, underperforming pandas code and call it an effective solution

One of the really nice things with Julia is some of the ecosystem needs disappear. Python needs a lot of ecosystem because none of the packages work together, and the language is slow so you have to make sure you are doing as much work as possible outside the language itself. To answer your question more specifically:

Numpy -> Array + broadcasting (both in Julia Base)

pytoch/tf -> Flux.jl (package)

batch/stream processing -> you don't need it as much, but things like OnlineStats exist. Also Base has multithreaded and distributed computing. Spark in particular is one where it lets you use a cluster of 100 computers to be as fast as 1 computer running good code.

pyarrow -> Arrow.jl (there's also really good packages for JSON, CSV, HD5 and a bunch of others)

Let me know if you have any other questions. Always glad to answer!

Good spark support would be a good answer for batch/stream processing. I am a little scared of definition of support. Apache beam supports like 5 runners (flink, spark, data flow, etc) but the quality of runner support is extremely inconsistent. I’ve also noticed even for python flink sometimes have very useful operations only in Java with no wrapper. Although honestly having data pipelines in one language and downstream users of the data in a different language works pretty well in my experience so mixing data pipeline languages is somewhat ok.

What’s workflow orchestration choice? That’s main one you didn’t touch. My work area in on an ml training platform and a lot of my work can be described as wrapper work on kubeflow to allow dozens of other ml engineers to manage experiments/workflows. For python the main choices are kubeflow/airflow. Ray kind of but ray workflows are still quite new and missing a lot of useful features. I need some system to run hundreds of ml workflows (one workflow being like 5-10 tasks some short some long) per day and manage there tasks well.

Broader area also includes libraries like weights and biases, bento ml, etc (experimentation management libraries).

In theory you can have workflow manager in one language and workflow code in a different language. Main downside is it makes debugging locally workflows harder (breakpoints are a little sad across most language boundaries), but it is doable and we debated migrating to temporal (Java workflow system) before.

I don't use much in terms of orchestration, so I'm probably not the right person to ask there.

One of the really big potential benefits of Julia is that it lets you remove language barriers which is especially nice if you are doing ML research (or playing around with new model types, etc). Since the ML stack is Julia all the way down to CUDA/BLAS/julia loops, you can really easily inspect or modify everything in your stack.

> What’s workflow orchestration choice?

We’ve moved away from language-integrated orchestration entirely at my work: we use Argo Workflows on Kubernetes, so we’re just orchestrating containers and aren’t beholden to language-specific requirements anymore so you can use whatever language/tool you want provided it packs into a container and accepts/returns what the rest of the workflow expects.

By "the ecosystem" they mean the python ecosystem, not the PyTorch ecosystem.

PyTorch is a small part of the python ecosystem. The python ecosystem is not going to change at all if PyTorch moves to Julia.

I've been thinking a lot about trying to use functional dialect of Python like Coconut[0] and Hy[1] w/ JAX so I can write functional DL code.

Glad to see functorch[3] as PyTorch is the library I have the most experience with.

[0] http://coconut-lang.org/

[1] https://docs.hylang.org/en/alpha/

[2] https://github.com/google/jax

[3] https://github.com/pytorch/functorch

Also see:

https://github.com/adsharma/py2many/blob/main/doc/langspec.m...

See also, Hissp: https://github.com/gilch/hissp

Loving pytorch but the reasoning remains strange to my ears: Python at all cost, not Julia, because of the ecosystem, well OK.

But all bullet points are about things that are easily done right now with libtorch (pytorch underlying C++ core code), and the hassle is... Python.

Well rational conclusion would be, just do everything in C++, and bind to Python. Make C++ first citizen here, since in all cases it'll be needed for performance, forever.

If Julia had wanted to be taken seriously then it shouldn't have dropped the ball at the very first hurdle by having 1-based array indexing.

This is such a non-issue and I'm tired of reading a version of this comment in any thread where Julia is mentioned.

This is a total non issue as indexing is an operation that is subject to multiple dispatch. For a humorous example see https://github.com/giordano/StarWarsArrays.jl

The person who comes to a Julia discussion and says "so, how's 1-based indexing?" is the same person who walks up to people in the office and goes "so, how's the weather today?" every single day. It's the Smash Mouth Allstar of programming language conversations. The only things interesting about the conversation at this point are the ever more elaborate attempts at semi-comedically changing the topic towards something that's not so overly repeated.

Julia (and Fortran) have a concept called offset arrays where you can basically start on any sort of index: https://github.com/JuliaArrays/OffsetArrays.jl

IMHO, one of the biggest advantages of Julia _is_ arrays.

Julia does not have offset arrays. There is an external Julia package that can mimic Fortran's intrinsic behavior (arbitrary start index).

julia's package manager is quite modern (and very well designed relative to the mess that is either C++ or python, no experience in modern fortran for me), so something being external or in the core language is not as important as it is with other languages. this is also the same model as in for example, Rust or Javascript.

in fact, it appears to me that they intentionally made the core language with fewer amounts of intrinsics relative to other languages.

Many of the packages in JuliaArrays/ might as well as be in the core language, especially things like StaticArrays: https://github.com/orgs/JuliaArrays/repositories

I happen to know that pytorch is a pain to maintain in packaging systems. It has a complex build system, many non-python dependencies, and massive build times. I don't know how this factors into the dissatisfaction with a C++ implementation, but I wouldn't be surprised if it were a factor.

In other words, python binary wheels are harder to maintain than source-only python packages. And pytorch uses more than a few. I can't imagine Julia makes the problem much simpler. The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code.

I don't know what it would take for this sort of pain point to improve significantly. Some standards around how C, C++, and Fortran projects are packaged would help. This would allow projects to build on top of existing natively compiled tech a lot better. Maybe the biggest reason those languages don't have the same "ecosystem" as python is utter lack of packaging standardization.

I would recommend you never try to compile tensorflow then. Honestly pytorch might take ages and be relatively complex. But 9/10 tries it builds on a recent Fedora, when you get a new release. tensorflow then ... I mean it's not me to judge that, as I don't understand why you would vendor a ton of C-libraries while depending on a single-file v0.0.3 python library...

Julia actually makes this problem way better for 2 different reasons. the first is that you don't need nearly as much C/Fortran when you are working in a fast language. the second is that Julia has binarybuilder which is a really good system for delivering reproduceable binaries that can be distributed. To show how well this works, try adding the Cuda Julia library. it just works without any of the issues python has.

> The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code.

Are you talking about something like BinaryBuilder.jl[1], which provides native binaries as julia-callable wrappers?

[1] https://binarybuilder.org

The more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy).

It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language.

I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.

100% this. I wish it weren’t so.

What do you think the major drawbacks are? Speed would be the top of my list, but most projects to not need anything more than what python can currently pump out.

Basically everything. If I had to pick two things I hate the most it would be dynamic typing and Python's deployment story.

I wrote a somewhat tongue-in-cheek rant blog post. https://www.forrestthewoods.com/blog/things-i-like-about-pyt...

Aside from speed, one thing that really eats at me is that it makes any sort of functional programming overly verbose, unfun, and just not very idiomatic. Also the the vast majority of python programmers simply don't understand the best practices in their own ecosystem.

I was recently writing code using Reactor/RxJava in Java 11 w/ Lombok. I don't think I've ever been so productive or lead a team as productive as when we were going ham on the functional/reactive Java. Now that I'm back in Python land, I am constantly frustrate on a daily basis with both the language and the runtime at every turn. Even with the asyncio we are working on, it feels like the absolute minimum viable product when compared to the java, node, or rust I have done.

There are some fantastic python enhancements that bridge some of the gaps like PEP 484 (Type Hints) and PEP 517 (holy crap an actual build systems that are not dogcrap) but it feels like the python community does not care.

Never seen Python described as verbose. Rx has libraries in Python libraries. If you think scientific programmers give a damn about build systems you don’t know what you’re talking about

You can't dismiss the fact that hiring python is hard. You think you're getting a good programmer, because they know all the leetcode tricks, but that person turns out to be a dud.

Isn't that true for any language? Why are you hiring for language skills rather than problem solving aptitude and conceptual fundamentals?

> Why are you hiring for language skills rather than problem solving aptitude and conceptual fundamentals?

All I'm saying is that signal to noise for the common tests you give for 'problem solving aptitude and conceptual fundamentals', is much lower when you are hiring for a python position. You think you're hiring for those things, but you're actually hiring for leetcode-optimizers.

I mean, I'm not trying to do hire like that, and I think I have an interview that tries to test that effectively, but I have had to deal with the downstream effects of people who are doing hiring like this, and that has been a real problem for me.

> when you are hiring for a python position

If you instead hire for "engineering" positions, without caring about what languages the candidate knows, you can interview for their ability to solve practical programming exercises [1] in whatever language they are most familiar/comfortable with. Maybe this only works at FAANG-level hiring, but in these contexts, top tier candidates can get things done in any language, and that's really what matters, no? But more to your point, I've generally found candidates that pick Python (or Ruby/Perl/etc) can actually accomplish more–and therefore prove their capabilities–in the space of an interview simply because they're picking a more expressive language. Bad candidates will prove they are bad candidates no matter what language they choose.

1: Eg, reading/manipulating files, merging/filtering/sorting data, generating and processing multi-dimensional data structures, etc.

I personally have an interview that essentially tests if you can produce well-organized code for a challenging algorithm that is persented as-is (the steps to the algorithm are provided, so no cleverness is necessary, you are dinged in my interview for using cleverness if it introduces errors). But, I do not control hiring for all people I have to work with.

Can you elaborate? If someone can pump out leet code I would assume they would be a half decent programmer and it would just take some time for them to be as productive as you wanted. Then again I’m mostly self taught and never done leet code and I still manage to be a good programmer according to those I’ve worked with

I personally think that leetcode-interview-passers that I have had to work with do poorly with actually useful tasks like, writing organized code, writing tests, documenting, etc.

Probably but those are things they can pick up on the job pretty quick, while it’s a lot harder to teach leet code style problem solving while solving actual business problems.

Maybe you have a different experience, but none of the actual business problems I deal with are leet-code style. Then again, I have recently stopped calling myself a software developer (and have long since stopped calling myself a software engineer) in favor of the more accurate "software plumber". But I imagine, even coding tasks that require algorithmic knowledge and insight, are benefitted from well-organized and well-designed code. Doing that correctly is often a matter of insight and judgement, and I think often those are not "easily learned".

> You can't dismiss the fact that hiring python is hard.

I deny that hiring Python is hard beyond “hiring is hard”.

> You think you're getting a good programmer, because they know all the leetcode tricks,

Unless I want someone for a role that is very much like reproducing leetcode tricks, I don't think I would think someone is good for it because they are good at those. In fact, leetcode is mercilessly mocked as being almost completely irrelevant as a positive signal for hiring, though it may be useful as a filtering tool to reduce an overwhelming volume of applicants to a manageable one where high rates of both false negatives and false positives, but some slight advantage over just randomly discarding applicants, is tolerable.

Do you have absolute control over the hiring of every person you work with? Not all of us do.

> Do you have absolute control over the hiring of every person you work with?

That's...a complete topic switch and irrelevant. The discussion I was responding to is about the challenges facing whoever does have control.

Well considering I made the initial statement, no, it's not. A company I used to work for once hired exclusively from the must-have-Python-experience pool (not my decision) and the cto fawned over how well he solved this "well-known-leetcode problem", and he utterly failed my problem, which tests for actually useful competency... of course he was hired -- and turned out to be a complete lemon. I remember that hiring round distinctly, everyone we interviewed for that position (n~10) was competent for the leetcode problem but never did basic things in my interview like "write tests", "don't try to make a complicated algorithm", etc, even when told explicitly to do/ not to do those things.

Outside of that I interviewed several of my friends (I know them from a non-programming context, so I don't know their competency) who were predominantly python devs, and completely noped out of them for the same reasons (and these were my friends).

Bad languages like Python, JavaScript, PHP are responsible for powering large part of tech revolution. Ability to write bad code easily is IMO large part of why they’re so popular. Low barrier to entry helps to build huge ecosystem.

Sure, but I get the feeling that the pendulum is swinging from a period where being able to write code easily is relatively devalued to being able to read code easily. Ironically, python itself benefitted from the last swing of that pendulum, as it was widely regarded as an "easier to read" PL than, say C (well, yeah) or perl (well, super yeah) or PHP (super-duper yeah).

I would claim the tech revolution happened despite those terrible languages rather than because of them. The languages are popular because of inertia, not because they're good.

Python is popular because of the ML revolution. If ML didn't take off neither would Python's popularity. Is ML successful because of Python or despite Python? Well, the world is probably further along with Python than if it merely didn't exist. But if a different language that sucked less existed we would, imho, be further along than we are.

I'm not annoyed Python exists. I'm annoyed that its inertia is so monumental it's inhibiting further progress. We're at a local maximum and the cost to take the next step is really really really high. That's not Python's fault mind you, just the way things work.

No these languages are the drivers of the tech revolution. PHP was widely adopted because it allowed people to rapidly build personal websites and then those people went on to build companies with it. Python is popular in ML because of its syntax being close to pseudo-code and allowing people who aren’t programmers to interact with all the old math libraries you used to have write C and Fortran to use. JavaScript is popular because it made the web more interactive and thus a ton of people who made websites learned it as their first programming language when they were young.

Python and PHP are so big because of the languages themselves and their implementation, Js is a bit different in that regard I’ll admit.

It's true that the tech revolution would not have happened if all code had to be written in assembly. And it's true the machine learning revolution would not have happened if all code had to be written in C.

Is Python better than assembly and C/C++ for ML? Absolutely. Is Python good? I don't think so. Other people might use that term, I do not. I think Python is a bad language that would be designed very differently it was built today. And we're stuck with its decisions because the inertia to change them is monumental.

It's not really the language's fault. As an industry we've learned a lot in the past 30 years! It would be a travesty if we hadn't made language design process! Unfortunately we haven't figured out how to effectively apply lessons to encumbered ecosystems. Case in point: the transition from Python 2 to Python 3 was an absolute disaster. And it only made tiny, incremental improvements!

As someone who uses Python since 2004, you got it backwards.

Python was popular before because it's very nice language. People wanted to use it for science to, so they wrote very good scientific libraries for it.

R was very popular for non-neural-network ML some years ago, yet it wasn't picked up for NN, because R kind of sucks for general programming. As the joke goes, the best part of R is that is a language written by statisticians. The worse part of R is that is a language written by statisticians.

Python was growing at accelerated speed year on year well before neural networks.

The runtime of R is it’s biggest downfall. Its single threaded nature kills it. I say this having used future::promise in a Plumber API to get some semblance of concurrency.

> Python is popular because of the ML revolution.

Python was popular, including for scientific use, before the ML revolution; in fact, the only reason it is associated with the ML revolution in the first place is it's preexisting broad adoption in scientific computing.

I would say that those are not bad languages. People are just elitist and think if your language isn’t strictly typed, functional and gives first year CS students a headache it’s a bad language and “creates spaghetti code.” The only thing wrong with dynamic typing is it’s slower and is harder to debug, but people are able to be way more productive in these languages you call bad.

> The only thing wrong with dynamic typing is it’s slower and is harder to debug, but people are able to be way more productive in these languages you call bad.

Not even close to "only thing". Dynamic language is a net loss of productivity once you reach a certain level of scale. Refactoring a codebase with millions of lines of code in a dynamic language is an absolute nightmare.

Opinions vary at what level of scale this happens. My personal experience is that once you hit just a few thousand lines of code that dynamic typing is a drag on productivity.

There's a reason things like TypeScript are increasingly popular.

I don't care what language first year CS students use. I care what languages I'm forced to deal with on a day-to-day basis!

It's not just Python, Julia is dynamically typed too.

Indeed, and static type analysis is the single biggest feature I'd like Julia to add first class support for.

i agree, this is without a doubt the number one thing that julia needs.

nearly every dynamic language has trended towards this in the last 7-8 years. it's probably had the most profound effect in modern Javascript (typescript) and python. It's incredible how different production javascript and production python are compared to say, 2014 javascript or 2014 python.

JET.jl is a interesting project to do some error analysis through abstract interpretation: https://aviatesk.github.io/JET.jl/dev/jetanalysis/

more so than the goals of JET itself, the abstract interpretation framework it exposes should allow for TypeScript-like gradual typing.

This is important, imho, if Julia is ever to go out of a niche area because most of the rest of the world has already moved on to see the benefit that the improvement in application reliability that static analysis allows.

JET.jl it's an interesting case indeed. Its being used retroactively in the Julia base code to look for any undesirable dynamic type situations

This is one of the nicer aspects of Julia. It starts out being a great language to work in. Its easy to implement algorithms that are generally difficult in other languages.

Its important to remember that most of the python ecosystem, isn't written in python. The functions are often thin wrappers/objects around the real computation, which is often written in a faster language, C/C++/Fortran.

Julia excels in composability, performance, ease of development. You don't need to recode your algorithms in another language due to the performance of Julia, as is needed in Python's case.

Generally speaking, I see Julia's biggest fault, time to first plot, being addressed far sooner than python being redesigned to have Julia's capabilities. For the record, I use python daily in the day job. And I use Julia there for analytics there, often with jupyter notebooks. Easy for users to consume and interact with.

Let’s not ignore the giant elephant in the room: 1-based indexing. I don’t particularly care since I use R and Python but Java, C, C/C++, C# all used 0-based indexing. It’s truly a bizarre choice Julia made there.

It really is a rather small elephant. It can be jarring at first (unless you think of R, MatLab and Bash), but then you just stop thinking about it because it legit doesn't matter.

People should stop wasting time bikeshedding this insignificant detail, but for some reason it is to programmers like a red rag to a bull.

Really its only an issue to some programmers, who prefer specific languages.

When you have to deal with a wide range of languages, stuff like this is small potatoes, compared to, say, indentation based structure. The latter can result in the completely non-obvious change of program flow due to a single errant space or tab.

You mean, the little ant on the floor?

I barfed at 1-based indexing for about a week, but now it is as natural as anything.

I would compare 0-based and 1-based indexing with whether you put semicolons at the end of each line or not. Either way doesn't really change the feel (semantics) of the language.

Also, fortan is 1-based, iirc, and a lot of numerical code is in fortan.

Oh, and many many beginning programmers and scientists have a hard time with 0-based indexing. Not sure why, but such you hear, so the choice is really not that odd.

The reason beginners have a hard time with 0 based indexing is that humans count from 1. Seriously, I've spent weeks trying to tell people "yeah, we want rows 4 and 5, so that's, uh, rows 3 and 4..." and they think it's nuts, and I now think they're right.

Right. Zero based indexing makes zero sense, unless you explain the underlying technical reason, that it’s an offset relative to a memory pointer (spend a week teaching pointers first!).

It makes sense in certain context (and in languages like C that have a low-level mental model). For scientific computing at a higher level of abstraction where the mental model of a multidimensional array is a tensor, and not a memory offset location, zero-based indices really get in the way

> Zero based indexing makes zero sense

The sensibility of the index choice is equal to the starting value of the index.

StarWars indexing makes 4 sense.

yupyupyup, you got it. :)

Precisely. Indexing makes sense in a context, and it is trivial in general to switch. This said, telling people that the first element they want is the "0th", is completely unnatural.

You can turn any 1 based array into a 0 based array using a wrapper type that just gets inlined away.

Also have a look at https://github.com/giordano/StarWarsArrays.jl

Mose Giordano hit the nail on the head with this one. Who cares about 1 vs 0 based indices? If anything its a silly and minor design decision.

I worry about someones ability to solve real problems in any language if they can't get their head around an +1/-1 when indexing into an array.

> I worry about someones ability to solve real problems in any language if they can't get their head around an +1/-1 when indexing into an array.

This. Think about what this signals to employers and interviewers if someone throws a hissyfit over this.

1-based indexing makes sense to computational scientists (target audience). Fortran, Matlab, etc. make very good use of that. Moreover, you can change to zero based if you wish.

So this "very big elephant" is, in reality, a nothingburger.

For me, the very big elephant in the room is the semantic formatting. It has and continues to trip me up time and again. A touch of a space bar can change the flow of a python program. That is a huge asteroid of a problem.

I don't think Python is a bad language, it just often gets used where it shouldn't.

Python is one of the few languages that has a balance of ease of use, ecosystem, ubiquity, and useable type system. It's a fantastic glue language and it's extremely flexible.

Python is not a bad language, programming languages do not have to be unreadable or have a steep learning curve to be good. The problem with python is that it’s implementation is slow and offers a ton of hang ups that you have to know the language in and out to even know they’re there. There’s a post here about once a year that details some of the funnier things.

I have had the opposite experience with python, the more I use it and learn the standard library and ecosystem the more I love it. What exactly makes you think its a bad language?

For me I think the packaging ecosystem is bad, we need one package management tool like poetry built in. We need a built in typing system like typescript. Lastly we need to remove the GIL.

I’m pretty sure all of these are currently being addressed by the community.

I switch languages a lot and things like functools, itertools, dunder methods, list comprehensions, dict comprehensions are things I sorely miss especially in typescript. In particular list and dict comprehensions when used with care are a great deal easier to work with and reason about when transforming data.

I'd be moderately happy with Python if it had full static typing, improved error handling, fixed packaging, fixed deployment, and removed the GIL.

I like to think that containers only exist because deploying a Python application is so %^#(&*# complicated that the easiest way to do is to deploy an entire runtime image. It's an absolute nightmare and travesty. So bad. So very very bad. https://xkcd.com/1987/

I'm not optimistic on TypeScript for Python. That'd be great if such a thing existed! I'm not optimistic on packaging or deployment. There is recent progress on GIL removal which is exciting! There is hope, but I'm proceeding with very cautious optimism.

Comprehensions are kinda great, but also hideous and backwards. Rust iterators are a significant improvement imho. The fact that no recent languages have chosen to copy Python's syntax for comprehensions is telling!

Oh, and I think the standard library API design is pretty poor. Filesystem has caused me immense pain and suffering. Runtime errors are the worst.

> I'm not optimistic on TypeScript for Python. That'd be great if such a thing existed

MyPy exists, Python officially supports type annotations.

I do think comprehensions are a weird feature for Python, particularly from the “one way to do it” perspective. And also because the language so strongly discourages FP patterns outside of comprehensions.

Overall I would say the language is a lot better than the ecosystem, and that it suffers a lot from having a bad packaging design. I’m not a fan, suffice it to say. It’s best if you can stick to the standard library.

> Oh, and I think the standard library API design is pretty poor. Filesystem has caused me immense pain and suffering. Runtime errors are the worst.

Do you mean os.path, pathlib.Path, or something else? I use pathlib.Path all the time and never get filesystem errors anymore, it's really practical.

Someone wanting to promote a new language could wrap Python as a growth hack.

Julia has a package that does this fairly elegantly (not perfect of course, but you can wrap on top of this) https://github.com/JuliaPy/PyCall.jl.

> Julia says:

> A language must compile to efficient code, and we will add restrictions to the language (type stability) to make sure this is possible.

> A language must allow post facto extensibility (multiple dispatch), and we will organize the ecosystem around JIT compilation to make this possible.

> The combination of these two features gives you a system that has dynamic language level flexibility (because you have extensibility) but static language level performance (because you have efficient code)

Given those constraints, the first language that comes to mind is Java. Why is Java basically not a player in the scientific-computing game?

no data types, no operator overloading, no extendability of new functions to existing objects, no C/Fortran interop, mediocre performance. Need I go on?

Also no (official) REPL.

Actually Java has had a repl called Jshell since JDK 9.

The Julia crowd really need to hang less value on the REPL - people writing serious production code don't use one, and scientists using Python have moved on to notebooks. At this point, having Julia be so focused on the REPL is a weird affectation.

Julia has great notebook support, the Ju in Jupyter stands for Julia after all. There's also the fantastic Pluto.jl for reactive notebooks.

And the VSCode extension isn't quite IDE level, but that's just a matter of resources.

IMO, the REPL is really nice for exploration. I would never write code in it, but it serves as a really good doc search tool and quick way to check things. It helps that the Julia REPL is more inspired by the Lispy ones than python/R.

I used notebooks when I did my PhD using Python, but have moved to the REPL using Julia on my PostDoc. The Julia REPL experience is so much nicer than Python's, it removes the need for notebooks completely for me.

I'm a scientist, I use REPL a ton. Notebooks are fine too, but there's a thing called exploration and nothing beats REPL speed for that.

In future it would be great to mashup something like repl.it with Jupyter notebooks, maybe with Medium-style wysiwyg. That might really win.

Mediocre performance compared to what?

Because people in that game value simplicity of use over anything else (performance, safety, maintenance) and Python is a top performer in that KPI while Java is not very good at this (though it is getting better but still)

This is a big part of it. People who are scientists write code that we would think is disgusting and don’t care that much about abstraction outside of mathematical functions. There’s a lot to learn with Java. I helped classmates in my intro to programming class when I went to college, because I already knew how to code, and I have no idea why they picked that language as an introduction language. After weeks people were still struggling because on top of the ideas like loops/variables/control flow now they are having to learn classes.

Python was written with people like scientists in mind. Professionals write fast C libraries and then people who know just enough to get by use python to glue it all together.

> People who are scientists write code that we would think is disgusting and don’t care that much about abstraction outside of mathematical functions.

- plenty of scientists write good code; I think the "scientist can't code" meme is harmful.

- most of the people that write PyTorch code aren't necessarily scientists - they're software developers (data scientists, ML engineers, research engineers, whatever title - but their main job is to write code)

> Python was written with people like scientists in mind.

Python had nothing to do with scientists when it started. It was written to be readable and easy to use and it started as a hobby project. It existed for over 15 years until the data science / ML ecosystem started growing around it.

I'd flip that meme slightly; it is true that scientists write shitty code...but it turns out computer scientists do exactly the same, and with less humility.

The difference is programmers write shitty abstractions to abstract shitty code

I should have been more clear, I didn’t mean python was written for scientists I meant it was designed with simpler syntax so people can pick it up and write a script in a few hours, scientists need to do some simple algorithm so they use python instead of fighting with a language with more terse syntax. People wrote their math libraries for python because it was a simple language (on the surface) with a C api. It was at the right place at the right time.

> Python was written with people like scientists in mind.

Having just taught a bunch of scientists Python, I'm skeptical of this.

Stuff that is obvious to me, but really hard to explain to coding beginners:

* why `len(list)` but `str.join(...)`? I.e. why are there two ways to call functions?

* why does `[]` do so many different things? List comprehensions, list construction, subsetting, sometimes it secretly calls a method....

* why in the name of god is the first row called 0?

* and then why does [0:3] mean rows 1, 2 and 3, not rows 0, 1, 2, 3 or 1, 2, 3, 4?

* ... except in pandas where sometimes [foo:bar] includes bar...

* in general, why is pandas so damn complex?

I thought it'd be easier to teach than R, but I'm not quite so certain now.

The main reason is that a lot of scientific computing depends on ancient libraries written in Fortran or C, and a lot of newer libraries depend on GPUs for their matrix operations so need to be able to make calls to C/C++ modules that run directly to the GPU. This probably isn’t impossible for Java but that was never meant to be it’s niche, it doesn’t compile to machine instructions and prefers portability.

I know FFI in Java is certainly possible and I’ve seen it at least a few times personally; is it that much harder than in Python or just less common due to the need?

Python doesn’t compile to machine instructions either and there’s nothing that prevents GPU access from Java. In fact I’d bet in many cases pure Java beats Python + C library though it obviously depends on how much has to be written in pure python.

Java falls apart on the point about post factor extensibility.

See this talk for examples: https://www.youtube.com/watch?v=kc9HwsxE1OY

For once it's impossible to work in Java without an IDE and without creating a project.

But you can just write a simple 20 line Python script to do some data mangling, no project with 30 IDE files required.

Between 1996 and 1999 there were hardly any IDEs available for Java outside Windows, and Forte was Solaris only.

Visual J++, Visual Cafe and JBuilder were the main ones but not everyone was eager to buy them, while the JDK was free beer.

Julia is designed with interactive use and REPL in mind. Java is designed with enterprise software written by large teams and IDEs in mind. The first one, among other things, leads to valuing terseness. The seconds leads to valuing verbosity. I wouldn't want to deal with the usual Java boilerplate in an interactive session.

That’s a really good question and I’m not sure I fully understand all the reasons. One big one is that Java intentionally has made interop with native libraries quite difficult. One cannot take a language seriously for numerical computing if you can’t easily call BLAS, LAPACK and FTTW just to start, and there’s a neverending supply of such efficient native libraries. Julia, on the other hand makes it easy to write efficient code in Julia but also makes it easy to call native libraries written in other languages. Easy interop with numerical libraries was pretty much the first design criterion.

There’s also some unfortunate choices Java made like standardizing one specific semantics for reproducible floating point code. That’s unfortunate because adjusting for native SIMD widths sacrifices reproducibility but improves both accuracy and speed. The only choice if you want perfect reproducibility on all hardware that Java supports is the worst performance model with the worst accuracy.

There’s also the fact that Java integers are 32-bit and Java arrays are limited to 2GB, which was reasonable when Java was designed, but is pretty limiting for modern numerical computing.

I also think that the JVM object model is quite limiting for numerical computing. They still don’t support value types, but value types are precisely what you want to represent efficient numerical values like complex numbers, or quaternions or rationals, and so on. Java forces all user-defined types to be heap-allocated reference types. Julia solves this by defaulting to immutable structures, which is exactly what you want for numerical values: the semantics are still referential (if you can’t mutate you can’t distinguish value semantics from reference semantics), you just can’t change values, which is exactly how you want numbers to behave (you don’t want to be able to change the value of 2).

Lack of value types in Java also makes memory management unnecessarily challenging. You can’t make a user-defined type with an efficient C-compatible array layout in Java. Because the objects are references, so the array is an array of pointers to individual heap-allocated objects. The ability to subtype classes forces that, but even with final classes, the ability to mutate objects also forces it, since pulling an object reference out of an array and modifying it is required to modify the object in the array (reference semantics), and that’s incompatible with the inline array layout.

And finally, this same lack of value types puts a LOT of pressure on the garbage collector.

> I also think that the JVM object model is quite limiting for numerical computing. They still don’t support value types

This is mostly true, but the primitives are value types and you can get some things done with them. (Not enough to make Java good for these use cases, no.) I.e. write float[] instead of Float[] and you have a contiguously allocated region of memory that can be efficiently accessed.

the fact that they couldn't make primitive vs object invisible is just incredibly dumb though. it's one of the many ways c# is purely superior.

C# is certainly more flexible in this regard but I wouldn’t say they made the distinction between primitive and object invisible, they just allowed user-defined “primitives” in the sense that users can define C-style structs that have value semantics. The type system is still bifurcated between value types and reference types. That’s ugly but livable in a static language where you can just disallow mixing the two kinds of types. And it’s not just theoretically ugly: it means you cannot write generic code that works for both kinds of types. Or you can yolo it like C++ templates and just allow whatever by textual substitution—to hell with the semantic differences. But of course then you get all kinds of surprising differences in fundamental behavior based what kinds of types you try to apply your generics to. Which is exactly what happens with C++ templates.

What do you do in a dynamic language where no such separation can be enforced? Or in a static language where you just don’t want that kind of ugly anti-generic bifurcation of your type system? The classic static PL answer is to just make all objects have reference behavior. (Which is the approach Java takes with a pragmatic ugly exception of primitive types.) This is the approach that ML and its derivatives take, but which is why they’re unsuitable for numerical computing—in ML, Haskell, etc. objects have “uniform representation” which means that all objects are represented as pointers (integers are usually represented as special invalid pointers for efficiency), including floating point values. In other words an array of floats is represented in ML et al. as an array of pointers to individual heap-allocated, boxed floats. That makes them incompatible with BLAS, LAPACK, FFTW, and just generally makes float arrays inefficient.

So what does Julia do? Instead of having mutable value and reference types, which have observably different semantics, it has only reference types but allows—and even defaults to—immutable reference types. Why does this help? Because immutable reference types have all the performance and memory benefits of value types! Yet they still have reference-compatible language semantics, because there’s no way to distinguish reference from value semantics without mutation. In other words immutable structs can be references as far as the language semantics are concerned but values as far as the compiler and interop are concerned. And you can even recover efficient mutation whenever the compiler can see that you’re just replacing a value with a slightly modified copy—and compilers are great at that kind of optimization. So all you give up is the ability to mutate your value types—which you don’t even want to allow for must numeric types anyway—and which you can simulate by replacement with modification. This seems like a really good trade off and it’s a little surprising that more languages don’t make it.

> C++ templates and just allow whatever by textual substitution

The Julia approach is indeed elegant. An in-between position taken by F# is statistically resolved type parameters, where type safety is maintained and semantics largely preserved by constraints on members. It's a pain to write but trivial to consume though errors can be obstruse. More type level flexibility on the CLR (which seems to be planned) will further improve things when it comes to generic programming.

one day OCaml will win ;-;

Not with its Unicode handling.

What stage of Julia denial is this?

Well, it's agreeing that Julia's goals (promises? aspirations?) are worthwhile. Whether Julia itself actually delivers on those promises is a different question that isn't addressed in the original post.

I like what Julia is doing but I just dislike the syntax. It seems to resemble ruby, whose syntax I also think is ugly, which to me resembles a modern form of basic.

Hmmm... this seems to be an odd first impression.

There is the use of @ (but to signal macros), but otherwise, the syntax is much closer to a cross between Python and matlab except nicer for doing math.

I tried writing a few programs in Julia and got sucked in by how effective it is. The real surprise is that just a few weeks in instead of pulling up R to do a quick calculation my fingers decided they wanted Julia.

Similar story for me. I did last years Advent of code in Julia and it was great.

I know what you mean. I especially dislike the use of an "end" keyword everywhere without a corresponding "begin" keyword.

I think that's inherited from matlab.

Huh okay. I’ve never used Matlab but one of the first languages I played with was a basic for simple game development

Yeah that’s exactly what I’m talking about

begin is a Julia keyword though it.

But yes end closes begin, for, function and while.

Does it really matter though? I mean, whether it's a closing bracket or a whitespace demarcated control flow - does it really effect anything you do in the language? Julia solves a lot of problems, like real problems - quibbling over begin/end vs {/} or white space seems kinda silly.

Meanwhile I have actually seen real world lost productivity due to white space in python. Curly brackets not so much. Never seen lost productivity over begin/end but I'm sure it's happened. It seems silly to me either way - it doesn't really effect anything.

Personally, I think when you write Julia in shorthand form, it looks nothing like ruby or basic.

for example, check out parts of the stdlib: https://github.com/JuliaLang/julia/blob/master/base/operator...

but in the end, julia really is a lisp: https://www.youtube.com/watch?v=dK3zRXhrFZY

With the slightest hint arising that Julia would be the future of ML and DL, I learned it.

But, then what?

I could not use it anywhere I worked. The ecosystem was lacking.

Julia is good, but for what exactly?

People involved with Julia are always big with words, but when will I see it in use somewhere?

You can apply to a job here,

https://juliacomputing.com/case-studies/

The ecosystem is a superset of Pythons: https://github.com/JuliaPy/PyCall.jl

It's probably more fair to claim that R/Python/Julia have the same ecosystem now if you are willing to deal with speed bumps and some clunky interfaces. Because there are packages going in both direction for all three.

Why do people keep claiming that Julia's ecosystem is limited? Julia can use all of Python's libraries. They all work great. No need to reimplement in Julia, just use PyCall and your favorite Python code.

It feels a risky proposition at this juncture to go short python. The arguments against/for have been rehashed a million times, the redeeming features of julia have been articulated very cogently...

What has not been accounted for is that the huge community / network effect of the python ecosystem is very far from exhausting itself. If anything, it is just starting as the exponential growth has mostly been the last few years (tautology, he he)

A major investment to eliminate python technical debt would make more sense if things were stagnant and the re-engineering would open up entirely new domains.

I've used Julia for quite a few years now. It's biggest flaws in my opinion are basically cultural and not technological. It's been adopted mostly by serious domain experts rather then typical software engineers and more 'normal' people. I don't know say junior or senior scientists. This has lead to amazing results but also has it's own detriments.

Some portions of the ecosystem are rock solid, especially the parts where JuliaComputing makes money from consulting(not all but some). Other parts are beds of sand/permanent research projects. The median experience is usually someone points you to a package and it doesn't really do what you hoped it would so you end up adapting it and rolling your own solution to a problem. Maybe you try to make a PR and it gets rejected because of "not invented here"/academia mindsets, either way you made a fix and your code works for you.

What makes this barrier hard to overcome for adoption is: trust, and blind spots. People who aren't experts in a casual work area (maybe computer vision) realize they can't use a tool to do something `basic` and run away to easier ecosystems(R/Python). People who are experts in other areas, check credentials of packages see that an ivy league lead researcher made it and assumes it's great and usable for a general audience. So you'll get a lot of "there's a package for that" but when you go to use it you might find the package is barren for common and anticipatable use cases in industry (or even hobbies).

This makes Julia best positioned as a research tool, or as a teaching tool. Unfortunately - where Julia actually shines is as a practical tool for accomplishing tasks very quickly and cleanly. So there's this uncomfortable mismatch between what Julia could be and what it's being used for today. (yes Julia can do both not arguing against it). The focus on getting headlines far outsurpasses stable useful stuff. Infact, very often after a paper gets published using Julia, a packages syntax will completely change - so no one really benefits except for the person who made the package.

Interestingly, 1 person(with some help of course) fleshed out the majority of the ecosystems need for interchange format support(JSON), database connections, etc. It's not like that person is jobless spending all their days doing it - it was a manageable task for a single smart person to kick off and work hard to accomplish. Why? Because Julia is amazing for quickly developing world class software. That is also kind of its detriment right now.

Because its so easy to create these amazing packages you'll find that a lot of packages have become deprecated or are undocumented. Some researcher just needed a 1 off really quickly to graduate, maybe the base language(or other parts of the ecosystem) changed many times since its release. Furthermore, if you try to revitalize one of these packages you'll sometimes find a rats nest of brilliance. The code is written very intelligently, but unpacking the design decisions to maintain world class performance can be prickly at best.

One of Julia's strengths is it's easy/clean to write fast enough code. One of its downsides is, this attracts people who focus on shaving nanoseconds from a runtime (sometimes needlessly) at the expense of (sometimes) intense code complexity. Performance is important, but, stable and correct features/capabilities mean more to the average person. After-all, this is why people use, pay for, hire for: Matlab, Python and R in the first place - right?

Most people don't want to have to figure out which ANOVA package they should use. Or find out in a bad way some weird bug in one of them and be forced to switch. Meanwhile in R: aov(...).

Do I blame Torch for not using Julia? No. Should they consider using it? Yes, absolutely. Does Julia's cultural issue need attention before risking Python(or anything else) reinventing a flavor of Julia that's more widely used for stability reasons alone - in my opinion, yes (see numba, pyjion, etc). Still love the language, because technologically it's sound, but there are blemishes. I'd chalk it up to growing pains.

This is a great comment, I've had exactly the same experience. For a simple, concrete example of the fragmentation issue, the canonical JSON parser seems to be JSON3.jl, but there's also JSON.jl, which is slower and has other subtly different behavior. Neither mentions the other in its documentation, neither is deprecated, but if you search for "json julia" only JSON.jl comes up on the first page of results, but if you ask a question about JSON.jl in Discourse or Slack they'll probably tell you to use JSON3.jl instead.

(To be fair, Postgres has an extremely similar issue with JSON data types and it's doing fine.)

The state of tabular data formats is similar but instead of 2 libraries there are 20, and some of them are effectively deprecated, but they're not marked as deprecated so the only way to find out that you shouldn't be using them is, again, to ask a question about them in Discourse or Slack. You can check the commit history, but sometimes they'll have had minor commits recently, plus (to Julia's immense credit) there are some libraries that are actively maintained and work fine but haven't had any commits for 3 years because they don't need them. I assume this will get worse before gets better as the community tries to decide between wrapping Polars and sticking to DataFrames.jl, hopefully without chopping the baby in half.

I feel like the "not invented here" mindset contributes a lot to that fragmentation. It's easy to write your own methods for types from other Julia libraries because of multiple dispatch, which seems to have resulted in a community expectation that if you want some functionality that a core package doesn't have, you should implement it yourself and release your own package if you want to. So we have packages like DataFramesMeta.jl and SplitApplyCombine.jl, not to mention at least 3 different, independent packages that try (unsuccessfully IMO) to make piping data frames through functions as ergonomic as it is in R's dplyr.

Despite all of this, I still like the language a lot and enjoy using it, and I'm bullish on its future. Maybe the biggest takeaway is how impactful Guido was in steering Python away from many of these issues. (The people at the helm of Julia development are probably every bit as capable, but by design they're far less, um, dictatorial.)

Kindred spirits it seems. Yea I think there is a serious future for Julia. It's my R&D and prototype workhorse by preference :).

Again, completely agree with the sometimes confusing state of the ecosystem. Sometimes I wish a bit of democracy existed, but people are people. I proposed some solutions to that problem a while ago but that's a story for another year.

Academia does create a very different kind of reward system that is often counter to community progress. IE: get there first, publish, obfuscate to thwart competition, abandon for new funding. Tends to reward people the highest for not giving credit, or sharing progress.

Meanwhile, people relying on alternatives to julia are more like: load in trusty xyz, use it in trusty way, I'll upgrade when it makes sense, and check the docs not the code when I am unsure of something.

Not to say industry is much better(I keep saying `academia`), but industry projects do tend to appreciate/honor free labor a little more kindly. That or they close the OSS gate and you get what you get.

Novelty is a driving force, but too much entropy and not playing well with each other can destroy a meaningful future quickly. It'll work itself out, one way or another but only because the technology is good :D.

I was surprised when browsing PaperSpace.com (a gpu host for ML training) that Fast.AI is now considered a "legacy" software? I've built a few small classifiers / ML projects but not really enough to really branch out of an intermediate tutorial.

With how quickly these frameworks change it's overwhelming to keep pace! Anyone have advice for solid frameworks that can reasonably leverage GPU's without too much heavy lifting?

Keras is integrated into TensorFlow and it's as solid and easy as it gets if you need a high level API for deep learning. If you need to write your own modules PyTorch is probably a better choice.

They must be talking about fast ai version 1. Version 2 is used everywhere now and development is on-going as usual.

I'm surprised that no one brought up using a subset of python with an emphasis on static typing, efficiency and transpilation can give you both the ecosystem and the efficiency.

There is Cython--it is a superset of Python. https://cython.org/

I'm aware of it, but prefer a subset accepted by static type checkers over a superset.

Neither language have proper tail call elimination, which, is absolutely insane to me.

Yall really just write procedural code for everything?

You must be in quite the functional bubble (I envy you for that, though)

Just Ruby...

I use Python plenty, just not in large enough doses that I have to actually make peace with it.

Actually, I've recently watched a talk by Guy Steele where he makes a case for tail call elimination being essential for object orientation: <https://www.youtube.com/watch?v=0hlBkQ5DjaY>. He demonstrates how tail call elimination enables better separation of concerns, allowing you to write code where objects don't need to "peek" into one another.

Another non-functional application of tail call elimination is finite state machines. Writing them as functions calling the next state in tail call position is very elegant, legible and efficient.

ES6 used to require tail call elimination, and it was even shipped by Safari and Chrome back in 2016.

Were it not for Firefox and Edge teams who torpedoed that feature, it would be a part of the major language of today.

Maybe it still will be.

A true scotsman rolls their own callstack.

I've never seen recursive looping over a thing get turned into efficient SIMD code in any language. Starting with loops has no good reason to be better able to achieve that but for practical compilers it makes a huge difference.

Julia code might also uses a lot of in place operations which would be hard for a compiler to infer as safe.

This is why I find it so annoying that CS programs seem to worship functional programming (at least MY program did!).

No, I AM going to write procedural code, and it WILL be faster than your "high IQ" 1 line recursive solution. Also funny to see how little recursion gets used in CUDA/Pytorch/GPU programming - which is what we are seeing to be more and more important over time.

Functional programming is a lot more than just recursion and there's nothing about GPUs that precludes an immutable first array based language with functional programming as core. Google Research's Dex is an example.

Speaking of functional programming and GPU's - check out Futhark :D. Very on the fringe but very interesting.

And functional paradigms have a way to express looping over a thing that's much better than recursion: filter / map / reduce.

> I've never seen recursive looping over a thing get turned into efficient SIMD code in any language. Starting with loops has no good reason to be better able to achieve that but for practical compilers it makes a huge difference.

Well, for example at the very least in Common Lisp you'll have much more joy with higher-order functions than with loops. The simple reason for that is the existence of compiler macros (http://clhs.lisp.se/Body/03_bba.htm) which can replace function compositions with arbitrary code. And it's much easier to figure out what the function composition does than to write a loop vectorizer.

Maybe loops are not fun in common lisp the? I had much fun with loops in PicoLisp.

Julia also has similar macro capabilities to common lisp.

> And it's much easier to figure out what the function composition does than to write a loop vectorizer

I probably agree but complicated loop vectorisation would probably expressed in Einstein tensor contraction notation such as Tullio.jl provides. I wonder a Fourier transform would look using function decomposition.

@tullio F[k] := S[x] * exp(-impi/8 (k-1) * x) (k ∈ axes(S,1))

Loops are a lot of fun in Common Lisp, what with LOOP (https://gigamonkeys.com/book/loop-for-black-belts.html), ITERATE (https://common-lisp.net/project/iterate/doc/Don_0027t-Loop-I...), and FOR (https://shinmera.github.io/for/). But their fundamental problem of trying to extract a "bird's eye view" of what you're actually trying to do remains here.

Maybe a new iteration construct specifically supporting parallelization would help here. Some of these things might already work quite well, for example it should not be difficult to discern from (let me use ITERATE's example)

    (iterate (for el in list)
             (finding el minimizing (length el)))

that you're computing argmin using a pure function, which should be easy to do in parallel, whereas the way you'd typically write this in C would be probably difficult for the compiler to confirm as such (it would have to look for pattern such as "if (f(current)<f(best)) { best=current; }" or something like that. (BTW have you seen another loop construct supporting argmin/argmax in another language? I haven't so far.) Whether this could be done somewhat generally and extensibly is something I have to ponder on.

if you want tail calls in Julia, there is a 3 line macro that gives it to you.

Could you point to it? Thanks

Here is one:

https://github.com/TakekazuKATO/TailRec.jl

It works by inspecting the code and rewriting a function to turn tail calls into loops.

The interesting bit is that it was very easy to write because of the strong macros in Julia.

This is eye opening. I might take a look at this in depth and try Julia again.

This repo should interest you https://github.com/TakekazuKATO/TailRec.jl

Real world code rarely uses recursion, and if it does it's the kind that doesn't allow tail call optimization.

I've been using FP strategies for around thirty years, so pardon me if I don't find confidence in your perspective.

Your case is covered by the "rarely" word in my comment.

Tail calls and while loops are essentially equivalent, so why care whether a language prefers one or the other?

Because if you like functional programming, then avoiding loops and imperative code is desirable. Some of us hate seeing `for` and `while` because they also imply mutation of variables. Recursion is beautiful.

It's like, why use one synonym and not the other? Sometimes it's because writing things in an immutable manner makes things clearer. Other code that can benefit are certain coroutines and generators.

> It's like, why use one synonym and not the other? Sometimes it's because writing things in an immutable manner makes things clearer.

That's true in many situations, but in this case the equivalence between the two is so straightforward that I can't see recursion gaining you any clarity. Perhaps you have a concrete example?

I can't think of a specific example but I do know that there have been times where I have chosen to write a loop as tail recursion and times where I have used mutable variables and a while loop. Why one or the other might be recursion better capturing intent, preferring immutability, efficiency or just a matter of taste.

Sometimes it makes more sense and is more flexible to package functionality as a recursive function, then mutation and variables feel like clutter. Other sometimes using recursion instead of a loop gains nothing expressively. Or sometimes structural recursion + pattern matching is the much more elegant approach in terms of communicating and capturing the essence of the algorithm.

They're not "essentially equivalent". A while loop can't begin in one module and end in another. A tail call sequence can. Loops are not modular.

Just to be clear, are you suggesting mutual recursion across modules?

Why is that unreasonable? I might be missing something (I have no formal CS education), but I can imagine it, e.g. a base graph datastructure library, and then two libraries dependent on the base library being used together for some graph traversal work (where callbacks are used).

Just to be clear, I'm using the word "module" here in an extremely generic, language independent way. Basically in the sense of "things that can be compiled individually". In many languages (for example in Lisp), those are functions. A syntactic loop construct ordinarily can't even cross a function boundary.

If your language of choice features formalized modules in the Modula-2 sense, its loops most likely can't span multiple functions even within a single Modula-2-sense module, even if you wanted that (for example, in a state machine with named states as functions which tail-call each other to switch state by transferring control, or something like that).

it's fairly easy to just write a macro to do tailrec in julia at least; https://github.com/TakekazuKATO/TailRec.jl

Lack of TCO is a JVM problem. All JVM languages are incapable for this reason. It's a very hard problem to fix apparently, I remember even asking one of the Sun JVM developers at a talk at my university and he said one of the issues was the Java security model and stack ownership IIRC. This was a long time ago though so I may be incorrect. The problem remains hard though.

Neither PyTorch nor Julia run on the JVM.

Why not go? Go beats Julia in parts where python is not good at. Is it because fb vs Google?

Try using Go for any serious math project, then do the same using Julia. Report back as to how both approaches went :P. From someone who uses both languages for very different tasks regularly, I would never try to write Torch from scratch in Go. I can't envision a way for it not to be a serious maintenance or performance disaster. Maybe that's a lack of my own creativity, but I'd much sooner use C++ rather then write any large portion of it in Go. If only for template generics...

It's because Go is a 90s (procedural) language in 2021 and I would maybe use it in a parallel universe where many other languages don't exist.

>Go beats Julia in parts where python is not good at.

I have not seen good results from differential equation solvers in Go.

I don't think you ever will unless Go >2.0 is a completely different language.

Yup exactly. I hate these super general statements like "better ecosystem". These blanket statements obviously make no sense since you can pull out 20,000 counter examples off the top of your head. Instead of "Go beats Julia in parts where python is not good at", name the one example where this might be true, maybe possibly, if there is one (is there is one?) to have a concrete discussion on. The internet and its vacuous statements are better for comedic value than discussion. We're at least on the same page.

Reply on news.ycombinator.com