Introduction to Autoencoders | Hacker News Reader

Introduction to Autoencoders

pinecone.io

104 pointsgk111 comments

The author lifted entire sentences without modification from his sources. E.g. "The parameterized distribution in the code of the autoencoder can be randomly sampled to generate discrete values for latent vectors, which can then be forwarded to the decoder, leading to the generation of new data." I thought this seemed unclear given the previous context, so I looked at the source article he linked to [1] and found that exact sentence. This kind of plagiarism is not cool. If you can't explain the concept, don't publish the article.

[1] https://blog.roboflow.com/what-is-an-autoencoder-computer-vi...

Hi, I'm responsible for our marketing which includes educational content like this. I'm looking into this urgently. Thank you for flagging it.

Update: After a closer look we determined this article is not original and decided to take it down. We're also making some internal changes to prevent this from happening again. Thanks again for flagging this!

I’m not surprised. Isn’t the whole concept of Pinecone to spam HN with valuable looking content so you can bait and switch into “our generous free tier” (so it doesn’t matter in the end that you understood nothing, because you can just abstract it away)? I’m not sure what I dislike more. This kind of “content marketing” on HN, or the fact that it apparently works.

Basic autoencoders are conceptually easy, but try to understand how the VAE KL loss was derived. I remember I spent two days reading https://arxiv.org/abs/1906.02691 and I still didn't quite get it.

Explaining Autoencoders seems to be a rite of passage for ML bloggers. It's conceptually understandable but has broad applications and is a jumping off point into other architectures.

Educational content tends to have a bias towards concepts that are easier to explain rather than what is most useful. But as far as the latter goes, Autoencoders are a good place to start.

Autoencoders are insightful because they move the concept of "intermediate representations" to the foreground. This perspective is useful for understanding the compositional architectures of NNs at any scale.

NNs can be generally described as "complicated feature engineering stacks followed by a relatively simple regression". This description captures the information processing and compression perspectives on NNs, which is realized by autoencoders in a laboratory-ideal way.

Can you explain it now? I'll give it a shot now. The minimal KL divergence should be equivalent to the maximum likelihood just another way to express/compute it.

To grok the ELBO, don't start with VAEs; start with basic variational inference. https://arxiv.org/abs/1601.00670 is good, though maybe increasingly dated...Once you're happy with VI in general, https://arxiv.org/abs/1606.05908 is a nice read on the VAE.

Thank you! Do you have any recommendations for educational resources at an even more basic level of understanding of variational inference? The first paper you mentioned is "for statisticians", and it seems pretty dense to me on the first glance. My current level is somewhere at the first undergrad pstat course.

Oh yeah that's gonna be a dense read then :)

I really like Kevin Murphy's books. I learned a lot from them when I was getting started on this stuff. He's just released a two part follow up to his 2012 book. Both parts are available online.

I also really like David Mackay's book (also free online). It's...idiosyncratic. But fantastic!

Best of luck with your learning! Start simple, go slow. Write code every step of the way!! ^_^

Reply on news.ycombinator.com