Hacker News Reader

Comment by aborsy | Hacker News Reader

Julia is great for numerical simulations. Companies such as Google and Facebook should have supported Julia for machine learning, in my opinion.

Numpy is OK, but verbose and a bit of a pain.

View on HN

> Companies such as Google and Facebook should have supported Julia for machine learning, in my opinion.

Given that ML is becoming so important and Julia has ergonomic handling of numerical code, why don't they explicitly support it? Is it inertia?

You didn't look hard enough.

ML on CUDA is very much part of Julia already, and Meta seems into that lib as well. =)

Is it good? PyCUDA was so bad (as of 2018) that after 3 major bugfixes from me and no end in sight it still almost sank a major project until I gave up on it and went with native CUDA.

Anecdotally, the Julia approach certainly seems... nicer via e.g. KernelAbstractions etc. I don't know if the performance/flexibility is quite there vs. native CUDA, but going from zero to GPU kernel programming in Julia is as close to painless as I've ever seen, especially given how modular everything is (for example, you can use OffsetArrays.jl directly in GPU kernel code).

"Is it good?"

Compared to the CUDA dumpster fire at a cat food factory, it is often trivial and nearly transparent syntax for the users familiar with ML.

Really depends on the use-case =)

The conventional options are fairly well documented:

https://sciml.ai/

https://fluxml.ai/Flux.jl/stable/gpu/

https://github.com/SciML/DiffEqFlux.jl

https://github.com/alan-turing-institute/MLJ.jl

They have a 100,000 Python, C++, and Java devs. My assumption is they don't care, i.e., inertia like you said.

The big mistake was Microsoft not having .NET cross-platform from the beginning and not pitching F# as a Python alternative.

> F# as a Python alternative

To play devil advocate, i think when F# was out (and still being developed), the Zeitgeist was really enamored with the idea of dynamic language, and the envisioned productivity gain from them. So F# was really too early to the dance.

Since then, things have change quite a bit with most major dynamic language adding a proto-type system.

Right now, i think we understand better what made language such as python and ruby so popular and are able to design better statically typed version with better tooling around them ( rust, zig, julia etc...)

The developers of Pytorch put out an article about this a few years ago and they admitted Julia would make sense from a pure ML perspective but the weight of the Python ecosystem was too critical.

I wonder if GPT-4 could be used to efficiently start porting more stuff over to Julia. I feel like Python is really a poor fit for deep learning. Julia is much more pleasant.

The only reason PyTorch exists is that Yann LeCunn couldn't convince people at Facebook to use the original Lua implementation.

So yeah, I'd be surprised if FB ever moved away from Python.

From my point, there is still too much change around automatic differentiation libraries.

There is Zygote.jl, which is used in Flux.jl. However, it's more in maintenance mode. At some point, Diffractor.jl was hyped but it didn't take off yet. And then there is Enzyme.jl which people hype now.

But for me as a user, it's not clear what I should really do to make my code well differentiable for those libraries.

If you stick with torch, jax or tensorflow, everything seems to work better regarding AD.

[0]: https://github.com/FluxML/Flux.jl [1]: https://github.com/FluxML/Zygote.jl [2]: https://github.com/JuliaDiff/Diffractor.jl [3]: https://github.com/EnzymeAD/Enzyme.jl

Note that Meta recently added a Julia ruleset to its buck2 prelude set. An important step toward working with Julia on prod environments.

https://github.com/facebook/buck2/tree/main/prelude/julia

Instead we got Tensorflow for ... Swift, which nobody asked for.

Apple probably asked and paid for it. Isn't there potential in accelerating LLMs and transformers on smartphones? Most of what they are good at fits the smartphone use case well.

Not at all, it was more related to well known Swift folks at Google.

I tried using the latest stable Flux.jl with the latest stable Julia interpreter about a year ago, and most of it didn't work. Some of the simplest model usage examples led to a traceback with dozens of frames, ending in an incomprehensible internal error that I couldn't possibly understand as a beginner.

It seems to me that Julia is still moving too fast for its own good. At some point, something needs to stabilize for it to be worth adopting.

Use of Python in ML was bottom-up, not dictated by the CEO.

Numpy is not really "OK" for everyone, it is still classed as a functional compatibility layer for a fundamentally broken language paradigm.

People often conflate popularity with good design. Have a great Monday =)

Are there any good references for what a library like Numpy should look like?

Most of Erlang, Julia, and Go... As they tended to re-factor dependencies and trend toward a homogeneous distributed ecosystem.

Python is the core issue rather than specifically Numpy, as it was bodged on (SWIG) to try to fix real use-cases... much like how 30 years of bodged on GPU mailbox structures made people biased to implement ridiculous solutions that necessitated ridiculous software paradigms.

I don't think any one person has the will and resources to resolve the core problem. However, Julia will likely also initially bind the same nonsense for awhile due to compatibility needs, but users tend to re-factor nonsense out of the ecosystem with better native solutions over time.

"All software is terrible, but some of it is useful" =)

I am specifically looking for Numpy-like libraries and designs, not languages that have generally well-designed libraries. For example, Elixir has Nx.

It seems to me that, even given Python's constraints as a language, that a nicer wrapper could have been developed in Numpy that still called out to bindings to do the heavy work.

> functional compatibility layer for a fundamentally broken language paradigm

What did you mean by this? Is it mainly the fact that Python is a bit an undesigned language in general that calls into unmanaged languages like C/C++ and Fortran in its libraries?

"I am specifically looking for Numpy-like libraries and designs"

Why would one bottleneck a design with polyglot stacks even before a single line of code was implemented? This sounds like naive nonsense.

"What did you mean by this? ... Python"

Python was never designed to handle threads or parallelism properly, and has performance issues Numpy tries to address though its wrapped C/C++ libraries.

Python became 30 years of spiral development, and implodes into a new implementation every so often. Depending on the use-case it may prove appropriate, but never optimal. =)

I was wondering about libraries that do Numpy-like things, not libraries implemented like Numpy.

There are many native packages and wrapper scaffold libraries like Numpy. It will depend on your problem domain, but Julia often transparently supports its broadcast operator on most core data-structure math ("using LinearAlgebra" and the like).

https://gist.github.com/AlexanderFabisch/6343090 (small feature comparison list, but ignore the incorrect opinions)

https://sciml.ai/

https://fluxml.ai/Flux.jl/stable/gpu/

https://alan-turing-institute.github.io/MLJ.jl/dev/about_mlj...

Julia!

Yes, the paper in the original post has a few examples. Julia uses expressions that are based on operators we use in regular arithmetic, even in plain Python. So, just ordinary operators instead of np.matmul, np.ling.expm, etc.

Matrix multiplication works with the @ operator, e.g. A@B. (Or with the * operator if defined as np.matrix, but nobody uses that and probably shouldn't). For matrix exponentials you need scipy.linalg.expm, but there is nothing in Python preventing why it couldn't be done with e.g. e**M if wanted (or even e^M if wanted but probably shouldn't). You can even implement it yourself in a few lines.

You don't seem to know much about what you're criticizing.

Lol Good luck working in a project where people have decided to create wrappers like you suggested.

For expm? The matmul as @ is numpy standard and used widely. If you have something that needs expm so much it needs to be an operator, good luck working in a project where people can't understand one-line class definition.

I simply refuse to work with code written by people who have had a hundred such harebrained ideas. No luck needed, but thanks!

Why is it any more or less harebrained for the language devs to overload an operator in the core?

Won't there be too much cultural inertia for all the people who invested their brain into numpy ?

Reply on news.ycombinator.com