Hacker News Reader

There actually is some work (https://arxiv.org/abs/2003.13630) claiming that FLOPS are a poor measure of real-world performance - with some of the more recent FLOP-efficient models actually running slower than older models.

Forgive me if I’m being dense, but shouldn’t we expect performance to degrade if flop count per unit time is decreased, assuming performance is defined as overall runtime (FPS in this case)? It’s a trade off scenario where runtime performance is being balanced by other concerns such as power consumption.

>because you want to run on an edge device or have power constraints.

Plug for TinyML

[1] https://www.tinyml.org

We have started to hit this in the last two years. Models get deeper, more FLOPS but not practically better metrics except FPS. But for our use cases and on our hardware constraints, 30fps on a model that's a couple years old outperforms even the Yolov99 or whatever is SOTA.