Hacker News Reader

Comment by eimrine | Hacker News Reader

> Substitute certain overhyped programming terms, such as "Deep Learning", and "Machine Learning", and "Artificial Intelligence" with more accurate replacements such as "Linear Algebra", "IF Statements", and "BS That Doesn't Exist Yet", respectively.

There was an article that NNs are just a decision trees, sounds better then "if statements".

View on HN

If you expand a NN to decision trees, the resulting decision tree(s) can take up orders of magnitude more space and take orders of magnitude more time to run than the original NN.

The paper is discussed here: https://www.youtube.com/watch?v=_okxGdHM5b8

So...compressed decision trees?

at that point most things are compressed decision trees.

Possibly related: learning is isomorphic to compression, and the two may be in fact fundamentally the same thing.

Oh, wow. I'd long thought of my own learning as "when reality surprises, isn't merely what you'd expect, isn't "obvious", then fix that, with emphasis on simplicity and generality". I'd not recognized that as compression. Tnx!

Counter argument: For every trainable NN, there exists an equivalent, sparser [edit] trained NN.

https://numenta.com/blog/2019/08/30/case-for-sparsity-in-neu...

https://web.archive.org/web/20211130211241/https://www.bhaut...

https://arxiv.org/abs/1803.03635

NNs are decision trees only if all the activation functions are step functions, which makes sense intuitively.

Reply on news.ycombinator.com