Modular is building a next-generation AI developer platform

Okay.

Why do people always bring up this shit? Solutionism…Sure it will solve problems but it’s going to create a lot of issues too. Imagine the mass surveillance, robotic enforcement of dissidents etc.

People already know what needs to be done to solve a lot of these problems we have, it’s called sharing and compassion. It’s not robotics.

I’m serious they the likely outcome of all this tech is going to be the matrix, so people can imagine their egos get as big as they like while being drip fed vitamins. One can only hope I guess.

Interesting, I wonder if Chris Lattner learned from the failure of Swift for TensorFlow project that he lead.

1. The iteration cycles are really slow. Waiting for models to train is O(days) for the most common models that can still do something useful.

2. It's very hard to predict whether a change will make your model better or worse. You typically have to try a bunch of things out the whole way through. Even a 10% idea success rate is pretty good, but it takes a lot of time and GPUs to find out which ideas do work.

3. The programming of a training pipeline is really error prone. The most common error is that your tensors are different sizes, but even the bolt-on static type systems don't help you prevent tensor size mismatch.

4. It's true that lock-in is pretty bad. If NVidia had real competition then maybe prices would come crashing down.

5. If the set of primitives in pytorch or tensorflow work for you, then great! But if you need new primitives, then you're crossing the Python-C++ boundary, learning CUDA, and also grappling with build systems (either praying that Python will find CUDA, or learning more than you ever wanted to know about Bazel).

6. All the data prep and final analysis tends to need specialized scripts and pipelines. All these tools are written by people who just want to get back to the model, and are run infrequently. You're lucky if the scripts still work today, and especially lucky if there are any tests at all.

Firstly it'll be a universal low level software layer, that can run on top of any cloud hardware, which can then be developed against to enable maximal cloud reach when training models.

This sort of virtualisation might be considered slower than direct access, but they've also created a DSL on top of python, which looks to enable the compiler to make smarter decisions about how to allocate memory and compute during training. So both together presumably producing a speedup worthy of the hype.

Kudos to them if they deliver on their promise.

Very hype. Pretty pictures.

> A new programming language that gives Python superpowers

This is the part that I find intriguing.

the usual gist, another framework