The author states that Spiral "was made for the sake of making a deep learning library which was too difficult to do in F# itself." I wonder what he thinks about DeepML (http://www.deepml.net/), which is a deep learning library written in F#.
The way these projects go is that you start of with an AD library and then when the type system and the language starts getting in the way you either do a lot, and I mean a LOT of engineering with inferior tools or you make something better.
The 'make something better' is always picked and can take various forms. Generally, people would not make a decision to work on a language for a ML library for almost a year before they write down the first line for it. They instead take the middle road of starting with an AD library and gradually extending it.
First they realize that they cannot really express tensors properly with the confines of the language so they make everything symbolic, then they build engines to run such symbolic AST, then they realize that they need JITs and other optimizers to make everything run fast. Tensorflow and PyTorch are currently at this stage. There are more stages after this.
Of course the task of working with ASTs is what a compiler does. I think it is a great pity that Tensorflow and PyTorch are written in C++ under the hood. C++ is probably the worst language I can imagine for working on compilers, so I applaud the authors of DeepML for picking F# instead. Had the TF team picked statically typed functional language for this they could have cut the size of TF by over 10x and saved themselves over a million lines of code.
[ML](https://en.wikipedia.org/wiki/Standard_ML) derivatives like it were made for that sort of thing and I cannot imagine doing Spiral in a non-ML styled language.
The idea is to keep a tape (you can think of it as a list) and then record all the operations on it as you step forward through the program. Then at the end you execute the operations backwards from the tape.
I take this approach in Spiral's ML library.