It seems like a ton of engineering effort has been put into these neural network frameworks. How didn’t they explore sparsity yet? With numerical linear algebra that’s, like, the most obvious thing to do (which is to say, you probably know beforehand if your problem can be mapped to sparse matrices).

(Edit: just to be clear here, I’m not saying I expect the whole field is full of dummies who missed something obvious or something like that, I don’t know much at all about machine learning so I’m sure I’m missing something).