Hacker News Reader

Comment by nextos | Hacker News Reader

Can anyone point to any reference where models depart from "closely-supervised end-to-end learning"?

In the article, LeCun & Manning argue this paradigm has some limitations, and I do agree. I'm thinking the field will evolve to systems becoming a combination of probabilistic logic-based engines (which represent formal causal reasoning) plus lots of deep models (which represent intuition, hypothesis generation and specialized tasks like vision).

View on HN

"Unsupervised Neural Machine Translation", https://arxiv.org/abs/1710.11041

Generative Adversarial Networks (GAN) and Autoencoders are the ones that come to mind. There are also all the models used in research leading up to the "deep learning" craze: Helmholtz machine, Boltzman machine, Deep belief networks.

http://pyro.ai/examples/vae.html

http://www.scholarpedia.org/article/Boltzmann_machine

http://www.scholarpedia.org/article/Deep_belief_networks

Geoff Hinton's Coursera course goes pretty deep on the unsupervised models

https://www.coursera.org/learn/neural-networks

Reinforcement learning probably counts too (Deep Q Learning, Policy Gradient, Actor-Critic networks might be equivalent to GANs?)

http://pytorch.org/tutorials/intermediate/reinforcement_q_le...

https://github.com/vy007vikas/PyTorch-ActorCriticRL

There are lots of examples. In the NLP space, Word2Vec and other embeddding models learn analogy relationships in an unsupervised way. Things like the OpenAI unsupervised sentiment detection model[1] show there is a lot more that can be done in this area.

[1] https://blog.openai.com/unsupervised-sentiment-neuron/

Reply on news.ycombinator.com