Nx: Multi-dimensional tensors Elixir lib with multi-staged compilation (CPU/GPU) | Hacker News Reader

Nx: Multi-dimensional tensors Elixir lib with multi-staged compilation (CPU/GPU)

github.com

Hi everyone, co-creator of Nx here!

I have just published an article on Dashbit's blog with a bit more context on Nx and the design decisions behind it: https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-...

It is hopefully a more in-depth reference than the README. If you have any questions, I will be glad to answer them!

Just wanted to say thank you, and how much I appreciate your work for the Elixir and Erlang communities.

A common complaint with Elixir and Erlang is that it isn't suitable for numerical computations, well now it is!

I'm excited to try this out, especially after seeing the XLA support.

Where do you see Nx going in the future? I saw there's an MNIST example in the repo. Do you plan including higher-level features for neural networks, similar to Keras/PyTorch?

Sean is looking into higher-level features for NN - it will likely be a separate library on top of Nx. For general direction, I briefly discuss that at the end of the article above. :) If something is unclear, please let me know!

As someone working in the weather field, the reference to xarray at the end of the article is very exciting. I imagine that eventuellay it will be possible to read NetCDF files and create dataframes based on Nx.

Great initiative!

One bit that sticks out to me is that the "standard" math functions like `exp` or `ln` aren't imported despite `defn` being quite maths specific. It seems that they could be imported as default functions inside the scope of `defn`. That leads me to a more important syntax question though.

Do you have any thoughts or plans on operator syntax for elementwise operations vs matrix level operations? One big "win" for numerical Python was having `@` become an infix operator specifically for matrix dots [0, 1]. The syntax in Julia (which follows Matlab) is to have most math operators also have an element wise version. It's generalized to "broadcast" operations in Julia [2]. R appears to have ``, `%%, `%o%` infix operators for elementwise, inner, and outer matrix multiplication.

Really writing any degree of matrix (ahem Tensor) maths and equations becomes very tedious and illegible if you have to use function calls to distinguish elementwise vs matrix operations.

The pipe operator really does help significantly, even with simpler stats oriented code (I've tried a few different formats [4]). Still it's still harder to read if you're familiar with "broadcast" syntax from other mathematics oriented languages.

0: https://alysivji.github.io/python-matrix-multiplication-oper... 1: https://www.python.org/dev/peps/pep-0465/ 2: https://julialang.org/blog/2018/05/extensible-broadcast-fusi... 3: https://www.statmethods.net/advstats/matrix.html 4: https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8...

Elixir allows overloading the operators or creating custom ones, but only certain symbols are recognized in the parser, to add new operators the Elixir parser needs to be changed.

You can see them here https://github.com/elixir-lang/elixir/blob/master/lib/elixir...

Not all of them are defined in the language, the list of the unused infix operator includes

    \\, <-, |, ~>>, <<~, ~>, <~, <~>, <|>, <<<, >>>, |||, &&&, and ^^^

to define a custom infix operator you can use this syntax

    def left <|> right do
      IO.puts("#{left} <|> #{right}")
    end

    # usage
    left MyModule.<|> right

    # or
    import MyModule
    left <|> right

if the operator is already defined in Kernel, you can exclude it from import and use your own

    import Kernel, except: [{:+, 2}]
    import MyModule

    left + right

as you can see @ is not among them, @ it's not a binary operator and is defined as Kernel.@/1

Hi, those are great points and exactly the type of discussions I expect to have with the greater Elixir community over the next months. I will answer them with my current perspective.

Given that Nx has its own numerical kernel, we can import whatever we want. :) I have decided to keep the imported keywords to be a subset of the Elixir's built-in Kernel but there is nothing stopping us from changing that. For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too.

Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream.

Thank you!

Thanks for the response! Glad to hear it's on your radar and ready to foster those discussions. :-)

> For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too.

Matlab and Julia import lots of math defaults because that's their primary target. In Python it varies, and it's pretty common to use `np` to avoid cluttering the namespace. Though partly it's because there isn't a good way (AFAICT) to make a scope with default custom namespace like `defn` gives you! It's a bit of taste really though. I'd suggest it's worth considering.

> Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :)

Well now, you give me way too many ideas. ;) I'll at least post some issues on the Nx tracker. My initial thought would be that perhaps a `<>` operator for "dot" operation would fit in with Elixir's other operators and is comparable to R's `%%` operators so there's a similarity and precedence. If it were low risk to add then it'd be a pretty decent option.

It wouldn't solve the whole elementwise issue though. but with syntax trees there could be ways to use Einstein notation instead [1]. That'd be fun to play with!

> If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream.

I'll create an issue on Nx project and basically repost my thoughts there. Then see what other people are thinking!

1: https://en.wikipedia.org/wiki/Einstein_notation

Just FYI `<>` is already the string concatenation operator. Just in case you're not aware. :)

Ah thanks, I think HN ate my character. It’s ‘<*>’ with a asterisk in between.

P.S. a potential sweet spot for Elixir in the broader scientific computing community comes when you combine native numerical libraries _with_ Phoenix LiveView! Realtime scientific dashboards are amazing. There's nothing I've found that makes interactive dashboards as easy as with LiveView. Actually there are projects in R, Julia, and Python that are great but they're much more difficult to deploy and scale IMHO.

The Nx architecture is quite good. I started work on a backend that avoids requiring jvm/bazel/python toolchain (substituting in a zig toolchain):

https://github.com/ityonemo/ez

the two languages feel like they were made for each other. Despite the verbosity of doing some things in zig (the dynamic tensor broadcast function is a bit of a beast), due to both language's similar compile-time metaprogramming facilities, getting to the point of plugging in Nx's tensor addition + tensor multiplication is about 200 LoC.

Cool project. I'd never heard of Zig, but it looks like an interesting solution to the need you've identified.

I'd definitely agree that it seems valuable to have an Nx version that comes with a smaller, simpler install for development purposes. XLA is amazing, but it's not obvious that you want to have that toolchain setup for local development in all scenarios.

Would be great to see several different backends created for Nx as a way of exploring what feature sets are really needed and ensuring that the integration mechanisms are sufficiently simple and general to support multiple solutions.

Well another place is for elixir nerves. You probably don't want to figure out how to shipping the xla architecture to your embedded device (I think it has to JIT the code); with nerves you can trivially cross-compile the nif shared library with zigler and deploy to arm architecture.

But kudos to José for building an architecture at a very good level of abstraction, you know, just in case xla falls out of favor (it is a Google project), nx won't have a hard dependency, and there will be trivial migration paths out.

I think there could be some other interesting nx backends, like "remote computation", or "transpile elixir and run on a Julia worker node"

A bit petty, but the first example is an unstable softmax implementation:

  defn softmax(t) do
    Nx.exp(t) / Nx.sum(Nx.exp(t))
  end

See https://ogunlao.github.io/2020/04/26/you_dont_really_know_so... etc.

Thanks for the info and the reference, that was a good read!

What's really amazing to me is Nx is just installed as normal hex package. `defn` is a standard Elixir macro: https://github.com/elixir-nx/nx/blob/main/nx/lib/nx/defn.ex#...

I love how this is implemented on top of existing Elixir utilities: just some macros and fancy AST transforms. The fact that you can do all this with just a library and not any sort of super low-level language pragma speaks to the elegance of Elixir's core design.

I think my dream job would be something like what José Valim does: designing a gorgeous, performant language that delights developers. Some day…

Also just above (https://github.com/elixir-nx/nx/tree/main/exla), exla is an "Elixir client for Google's XLA (Accelerated Linear Algebra). It includes integration with the Nx library to compile numerical definitions (defn) to the CPU/GPU."

As if I needed another reason to just stay in elixir fulltime. Machine learning/deeplearning was one of the few niches I was looking outside elixir for.

Elixir as a language is really approachable. If you're looking to dive in I recommend the book `Elixir in Action` but the official docs are also quite good.

Jose is so prolific on such varied kinds of projects. I expect he'll team up with John Carmack next to solve General AI...

I know the newest BEAM release has a new JIT compiler for faster calculations. Is that work related to this at all? I'm guessing not, since it seems like the speed up here is from moving the calculation out of the BEAM.

it's not related to the new jit.

As a programmer with no ML/GPU experience, this reminds me of the Spiral language: https://github.com/mrakgr/The-Spiral-Language

I wonder if Nx will succeed due being "just" a library for a popular language (Elixir) instead of being a separate language entirely.

"Don't make a new language if your idea could be implemented as a library" is good advice indeed.

As a regular full stack developer writing Elixir full time, what is the use of such a library? I have heard of Tensorflow, there is mention of tensors in this article, but what would be the practical use of such a library, in the real world? When would one write code like the one shown in the examples?

I'm happy that Elixir is becoming more and more mainstream. It's the best language I've used in my 15 years in the business.

At a company I work for we manipulate a lot of weather data which comes in the form of big multi-dimensional arrays of numbers. We use Elixir but we delegate this data processing to Python using the xarray library. Nx is the foundation of an ecosystem that would allow to build similar libraries for Elixir.

If I may ask, why is your company using Elixir? Wouldn't it make more sense to use Python everywhere on your web stack if you do scientific work on weather data?

In fact the vast majority of our data processing is in Python but we also have some R and shell scripts. Elixir is great for orchestrating parallel data processing and building reactive UIs showing what's happening in real-time. The Erlang VM has interesting performance and introspection capabilities and Elixir being a younger language its tooling is a lot less messy.

I learned something new, a creature called the Numbat.

I thought it was a pun on "wombat"

I too learned about the number from this project. I also learned that it’s endangered. The GitHub README has a link to where you can donate to help out.

If I understand correctly, Nx.defn effectively calls a backend compiler for that function, and that will presumably result in only one round-trip of data to the accelerator hardware (e.g. GPU). If you have two defn functions, do you 1) lose compilation between them and 2) round trip twice, or can you nest defn functions and get compilation of the composed function?

I believe that your understanding is correct: defn marks potential boundaries between "jit compiled" or even AOT compiled, and if you call a defn in a defn it will be fused.

Correct! Consider when you call "defn" for the first time to be the entry-point. Everything inside that is fused.

Tangential mini-rant: yes. naming things is hard. But we already have 'Nx' -- the monorepo tool.

I don't find this useful in any way since Elixir is weakly typed and a worse Lisp than Python..

How exactly is Elixir "a worse Lisp than Python"?

Elixir is a functional language and it even has Lisp-style macros.

deleted

douches get downvotes

https://games.greggman.com/game/dynamic-typing-static-typing...

The hard evidence that static typing is an overall-win is lacking. Period.

or, more succinctly,

https://i.imgur.com/zbU1wwH.jpg

how is static typing lacking evidence? don't we use that in rust to prevent the existence of large swaths of errors vs say C? i know for sure when turning on typescript for some javascript projects at work we found many issues, those all seem like wins? Why when I look at some dynamic languages say for example clojure i see stuff like type annotations in their core libraries, and then spec's to help provide a lot of what a type system can do? do you have maybe a counter argument to this fellow https://www.youtube.com/watch?v=XTl7Jn_kmio

The usual things that are brought up in this kind of HN thread are that studies don't show an advantage in delivered software, and tradeoffs / limitations of type systems are bad tradeoffs, and the vast majority of things caught by static type systems are trivial errors that are easily caught anyway, and that these days a lot of interfaces are APIs between systems that the static type systems don't cover anyway.

Thanks for the interesting sounding talk link. It's "Types are like the Weather, Type Systems are like Weathermen - Matthias Felleisen" from ClojureTV, for those who rarely click on youtube links. Mathias Felleisen a well known Racket guy, author of How to Design Programs, etc.

It probably helps in Rust.

Perhaps it helps more in certain specific languages or contexts.

The studies found no general “overall cost” improvement; in other words, the extra correctness came at the cost of tons more boilerplate and “fighting the type system”

Elixir is not weakly typed, it has strong typing on the contrary. The word you're looking for is "dynamic". Javascript and PHP are weakly typed, neither Erlang nor Elixir.

PHP is weakly typed but it has a strict type option nowadays, and class properties can also be typed.

Combined with function parameters typehinting (which easily beats typespecs) it's a great and productive developer experience I must say.

First: Elixir has strong dynamic typing. JavaScript has weak dynamic typing.

Second: As someone who uses Elixir for their day job, Racket for PL research, and Python for some CS classes, I can state with 100% confidence that Elixir is much closer to Lisp than Python. Elixir has macros that operate on the level of abstract syntax trees! Indeed, I think that is how most of this project was implemented.

I don't find this comment useful in any way since you are so weakly-informed and strongly-belligerent.

Elixir and Erlang are strongly- BUT DYNAMICALLY- typed. Next time, please check yourself before you wreck yourself.

Reply on news.ycombinator.com