Hacker News Reader

Comment by sarthakjain | Hacker News Reader

Deep learning architectures built by machines (so we no longer have to design architecture to solve problems) https://arxiv.org/abs/1611.01578

Transfer Learning (so we need less data to build models) http://ftp.cs.wisc.edu/machine-learning/shavlik-group/torrey...

Generative adversarial networks (so computers can get human like abilities at generating content) https://papers.nips.cc/paper/5423-generative-adversarial-net...

View on HN

I read the infoGAN paper yesterday. It blew my mind. https://arxiv.org/pdf/1606.03657v1.pdf. This is a way to do disentangled feature representation learning without supervision.

All these are definitely cool, but I think we're still a long way from leaving the "look at this cool toy" status and stepping into the "I can add value to society" status.

Furthermore, if we consider that most of these DL paper completely ignore the fact that the nets must run for days on a GPU to get decent results, then everything appears way less impressive. But that's just my opinion. I love working in deep learning, but we still have LOTS of work to do.

It's true that there aren't too many practical applications of GAN's yet, but I'd argue that transfer learning is already pretty powerful. It's fairly commonplace to approach a compute vision task by starting with VGG/AlexNet/etc and fine-tuning it on a relatively small dataset.

Could you elaborate? After running for days / weeks/ months the output is a net that can do inference in seconds, or with some now-common techniques milliseconds with only small reductions in accuracy. These nets can then be deployed to phones to solve a rapidly increasing number of identification tasks, everything from plants to cancer.

The time from theoretical paper to widely deployed app is smaller in DL than in any other field I have experience with.

There is a LOT of investment in model training right now, with frameworks, specialized hardware (like Google's TPU), cloud services, etc., not to mention the GPU vendors themselves scrabbling like mad to develop chipsets that accommodate this more efficiently.

It's going to take less and less time and money to train a useful model.

I have a hard time reading most research papers on deep learning, as they are geared towards mathematicians who already know that lingo.

>> Generative adversarial networks (so computers can get human like abilities at generating content)

Does this only apply to artistic content, or also to engineering content ? Say PCB layouts, architectural plans, mechanical designs, etc ?

Artistic only. You'd need an exact verifier in case of engineering designs, and something built with neural networks is not one.

To get a better understanding (other than reading a paper) read this excellent blog post https://openai.com/blog/generative-models/

GANs are a general tool -- they just happen to get a lot of attention for generating images of stuff. Here's an example for generating sequences [1]. The example is language oriented, but ultimately GANs are interesting because you can use them to build a generator for an arbitrary data distribution. This can have many applications in engineering (to take a random example -- generating plausible looking chemical structures under a certain set of constraints). As with any ML application, you need to quantify your tolerance for "inaccuracy" (in a generative setting, how well the generated distribution matches the true data distribution). This is simply an engineering trade-off and will vary based on the application.

[1] https://arxiv.org/abs/1609.05473

The approach was applied without any real knowledge of art, even though it has been applied to the domains you mentioned I don't see why not.

[edit]: it is a lot harder to build a NN when there are very constraint rules. But it is also a lot easier to verify and penalize it and generate synthetic data.

What's new in transfer learning?

Reply on news.ycombinator.com