I like your breakdown of the requirements in three abilities. I think Elixir provides all three of them (with a caveat) and I will refer to two videos for those who want to learn more:
* This video shows how you can use Livebook (open-source computational notebook environment) to explore pre-trained neural networks (from Hugging Face), learn how they work, and then deploy them as part of a Phoenix web application: https://news.livebook.dev/announcing-bumblebee-gpt2-stable-d...
* This video shows how to scale a neural network serving, first to run it concurrently, and then have it run distributed over multiple CPUs and GPUs: https://news.livebook.dev/distributed2-machine-learning-note...
The caveat is about your second ability. The production scale models are, at the end of the day running native/cuda code, using the same backends as Python (Google XLA and LibTorch). What we do is that we compile your Elixir code into a graph which is then compiled to the CPU/GPU just-in time (in an approach similar to JAX). You can find more information here: https://github.com/elixir-nx/#why-elixir