Hacker News Reader

Big difference between knowing the math in theory and actually using it to do something useful with your data.

That difference approaches zero as programming more symbolic and as implementations become more available.

Libraries like TensorFlow, Theano, etc. are great because then I can focus on what I want (math) instead of spending my time telling a computer how it should multiply things together in an imperative sense.

Agreed, but that Sword of Damocles cuts both ways. The thing you do with your data which starts out useful at first can and will easily erode into infrastructural debt if not built on sound foundations, which cannot be found off the shelf in standard scientific computing libraries that have been around for decades.

Or more likely in actual implementation (than direct decomposition of A^T * A), an SVD of A.

There's no such thing as "simply" in numerical linear algebra.