Comment by version_five | Hacker News Reader

Comment by version_five | Hacker News Reader

Parent Full thread

Deep learning is very good at certain problems, much better (in some sense, particularly the ability to generalize with less fine tuning and feature engineering) than methods that came before it. It had undeniably opened up tons of new research applications, but has of course fallen short of the hype around it. I'm still very bullish on the post-hype potential of deep learning for solving real problems, it just requires a much clearer understanding of what it does and doesn't do, and adapting solutions to work within it's constraints.

All that to say, it's easy to find flaws with it, but it has been game changing. I'm naturally skeptical about someone who has just published a book on something hes been working on for 50 years that he believes is "better" than the current (admittedly fad) technology. I'd like to see some real examples about the specific kinds of problems his tech solves, and how it solves them better than other methods. If he's saying it's better than deep learning on things deep learning is good at (e.g. can it beat a neural network model at classifying imagenet?) then I call BS absent a compelling demo. If it's better at some other thing, let's hear what that is.

> absent a compelling demo

I think the progress was the result of competitions that kept everyone blindfolded to test datasets. No one could exaggerate the performance of their methods, allowing deep learning to emerge as a winning solution. I think deep learning has fallen short of the hype because researchers are exaggerating their results, and what is needed to propel progress is a renewed focus (with funding) for competitions.

Exactly. If it's really better than deep learning, he can publish a library implementing it. Plenty of us are hungry for high performance, stable, explainable ML and the place for ART's approach would quickly be recognized.

I’m struck by the juxtaposition of these cogent, reasonable assessments of new tech (ML, blockchain) I see a consensus on and looking for jobs on LinkedIn. Sometimes all I see are jobs for new companies based on ML/blockchain.

We have these tools that solve specific problems in specific contexts yet all these companies are on the hype marketing train. Are these investors not doing due diligence or am I missing something?

ref: Theranos

Off Topic : I am interested in learning DL, not superficially (like most of online courses), but a more theoretical foundation, one that enables me to think and envisage new architectures, new techniques or combinations. Can you help me with any self study stuff that will enable me to learn as such.

Just my opinion, but look into the "Overparameterized machine learning" community[0]. If you are interested in the theory, this gets at what is fundamentally different about deep learning.

I may be reading too deeply into your question and you don't actually mean that theoretical. For something lighter, I'd suggest studying some of the common model architectures in the area you're interested in (like you can look at the pytorch lightning or torchvision or detectron2 models or etc source code) and understanding why things are the way they are.

[0] https://topml.rice.edu/ (this is an annual conference, can look at last years talks and references therein, search for topml, etc)

There is very little theory to DL. It's almost completely about empirical results.

You will be very disappointed if you expect that there is some grand theory of Deep Learning.

My suggestion is to follow Arxiv Sanity: http://www.arxiv-sanity.com/top

> but a more theoretical foundation, one that enables me to think and envisage new architectures, new techniques or combinations

Advances in DL have been mostly empirical not theoretical from what I have seen. It's less about being able to "envisage" and more about empirical testing.

Reply on news.ycombinator.com