Group Normalization | Hacker News Reader

Group Normalization

arxiv.org

82 pointsvictorvation12 comments

Open articleView on HN

No relation of subgroup normalisation, just a bad name.

Mostly only relevant for convnets it seems

"only"

Has anyone made or found an implementation yet?

Figure 3 in the paper.

available in PyTorch master via: https://github.com/pytorch/pytorch/pull/5968

When it comes to using super new research models or methods in deep learning, implementing it yourself instead of looking for other implementations is almost a requirement. In my experience the majority of online "implementations" of research deep learning models or methods have subtle bugs, flaws, or outright implemented something different than the paper they reference. This isn't limited to random github code: for example, I wouldn't trust anything in tensorflow.contrib unless I read the source.

Completely agree. Well, tf.contrib I am usually happy with the docs and a glance through GitHub issues, but, yes, code found elsewhere for very new, or even just recent papers can be really hit or miss, and not always obviously so. There is a lot of great stuff out there, but sometimes even popular papers over a year old can still be a challenge to find good code for. finding 3 separate implementations, all stlighty wrong or unusable is not uncommon.

Having implemented algorithms from research papers I came to the conclusion that the same is true about the papers. It’s unbelievable how bad the quality of the description of algorithms often is.

Are you generally able to achieve the results the new research claims after you implement it yourself per their description?

I would have thought if their description is shit then their results are shit too (cherry-picked from hundreds of runs, outright fabricated, misinterpreted, incorrectly graphed, whatever - just plain doesn't work, like a bad recipe.)

As a followup if this is the case then how do you know what new deep learning research is worth your time to implement?

https://github.com/ppwwyyxx/tensorpack/blob/master/examples/...

Nothing to do with database normalization or reconciliation

Reply on news.ycombinator.com