Hacker News Reader

The algorithms to train, initialize the networks, new architectures are far more important than the hardware advances. If people knew how to train NNs 50 years ago we would live in a different world.

View on HN

We did. they just didn't have the same computation abilities back then.

I find it really interesting that when Richard Feynman did a sabbatical at Thinking Machines when they were developing the early parallel execution hardware that's really not worlds away from modern GPUs he got them in touch with one of the leading neural network theorists as an obvious use for the tech. When he wasn't fixing their hardware designs using systems of differential equations.

It would be an interesting thing to know more about.

the basic concepts underlying DNNs have been known for decades, it has been exponential increases in compute power that have made them practical

Reply on news.ycombinator.com