Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by hgsgm | Hacker News Reader

Parent Full thread

hgsgm·

This command is a lot of words to say "I don't like it" without giving any reason to believe you.

If it's not new or novel, why aren't people using it? If it's bad, what's wrong with it?

View on HN

It performs worse than baсkprop.

Is it slower and less accurate ? Or just slower.

It is far less accurate compared to SOTA models. The paper says it should train faster, but it doesn't provide any metrics; so it's hard to make any "pound for pound" comparisons.

So the initial “hey, this might be a good idea” implementation performs slightly worse than something that has had literally billions of dollars thrown at it?

It is slower to the same backprop that we have used for decades now.

No comparisons to AdamW were made.

In fact, this algorithm uses backprop at its core, but propagating through 0 layers.

The question was "If it's not new or novel, why aren't people using it?". For example, take a look at this paper: https://arxiv.org/abs/1905.11786. It was published 3 years ago, it also does parallel layer-wise optimization, it even talks about being inspired by biology, though the objective function is different from Hinton's. Why aren't people using it? Because it performs worse. It is that simple. Is it an interesting area to explore? Probably. There are millions of interesting areas to explore. It doesn't mean it is worth using, at least yet.

Very significantly worse.

Reply on news.ycombinator.com