Comment by steppi | Hacker News Reader

Comment by steppi | Hacker News Reader

Having once studied just enough algebraic geometry to appreciate the true excellence of Mumford’s work in the field I’ve wondered before if I should study his Pattern Theory book [0] to mine it for insights that the ML community may have missed as a whole. I'd forgotten about that though.

Has anyone here worked through it? It seems like much of it is rooted in methods that have kind of fallen by the wayside due to the incredible success of modern techniques. I’d also be very interested to know David Mumford’s thoughts on modern deep learning and how he thinks it fits into his pattern theory framework. It looks like he goes into this to some extent in a 2019 blog post on whether AI can achieve consciousness [1] and another on the convergence between ML methods and the human brain [2].

[0] https://www.routledge.com/Pattern-Theory-The-Stochastic-Anal...

[1] https://www.dam.brown.edu/people/mumford/blog/2019/conscious...

[2] https://www.dam.brown.edu/people/mumford/blog/2020/Astonishi...

It seems like much of it is rooted in methods that have kind of fallen by the wayside due to the incredible success of modern techniques.

TBH, that's one reason I find this interesting to begin with. It's a pet interest (obsession?) of mine to look for ideas that have been "left behind" and forgotten about prematurely for whatever reason. I suspect that from time to time there's a technique / idea / whatever lying out there in the sands of history just waiting for somebody to pick it up, dust it off, and turn it into something amazing.

Along those lines, have you ever checked out the book Neurocomputing, Foundations of Research [0]? It's a compendium of old foundational papers that I think would be up your alley.

Researchers will find Neurocomputing an essential guide to the concepts employed in this field that have been taken from disciplines as varied as neuroscience, psychology, cognitive science, engineering, and physics. A number of these important historical papers contain ideas that have not yet been fully exploited, while the more recent articles define the current direction of neurocomputing and point to future research. Each article has an introduction that places it in historical and intellectual perspective.

[0] https://mitpress.mit.edu/9780262510486/neurocomputing/

Along those lines, have you ever checked out the book Neurocomputing, Foundations of Research

Yes, I picked that up (both volumes) late last year. Same for Neural Smithing[1] which I just acquired a copy of about two days ago. I also read that Talking Nets: An Oral History of Neural Networks[2] book last year. Lots of threads to follow based on that. Actually, now that I think about it, I think that's where I found out about Neurocomputing.

[1]: https://mitpress.mit.edu/9780262527019/neural-smithing/

[2]: https://mitpress.mit.edu/9780262511117/talking-nets/

Reply on news.ycombinator.com