Elixir and Machine Learning: Q3 2023 roundup

dashbit.co

39 points·josevalim··5 comments

Thanks for the hard work everyone who contributed to this ecosystem, from someone who just wishes to focus on applying the tech to build shit and solve problems in this space.

Been standing on the sholders of much of this for a while now from prototyping in livebook and for various ML/LLM backed prod projects with things like self hosting embedding models (thanks to NX + bumblebee + OS Models from ) and using meta fais for vector storage neatly via elixirs solid native bindings story, to generally elixir being great for streaming output from LLMs or building more complex agents on top of them using the process model / OTP.

Hi everyone, author here. This is a roundup from multiple projects and collaborators within the Elixir+ML ecosystem. I will be glad to answer questions where I can. :)

What is the state of NX on Apple hardware?

It runs fine, but using metal is complicated. We provide bindings for both libtorch and XLA. Libtorch has better metal support but still incomplete, with types and functions missing. Apple released a driver for XLA and it is, unfortunately, closed source and tied to a particular XLA version, so I wouldn’t particularly recommend it.

Very exciting, there are so many things happening in the Elixir Machine Learning space, with all of it focusing on being easy to deploy and test yourself.

I just love that you can create robust ML services. Which for me, isn't as easy to do in Python, even though I've created async service with core Tornado before asyncio was a thing.

Reply on news.ycombinator.com