Current state of the art in objects classification

Inference speed and net size is definitely considered important and some researchers are definitely tackling it (e.g https://arxiv.org/abs/1602.07360). However it's more of an engineering area (meaning lots of engineers are working on it) and less of a research area (meaning fewer papers get produced per work) because there's not much in the way of performance-per-gflop benchmarks or whatever.

CPU performance is good-ish when you're doing sequential stuff e.g it's not bad at old-school RNNs, but for image recognition using a cheap graphics unit is still going to be better, and if the question is "can I use cheaper hardware and get good performance" the answer is yes, you just have to do some tricks (e.g distillation).

Plenty of image recognition projects on the cheap raspberry pi and its graphics unit.

Inference Speed is very important. The TensorFlow object detection model zoo lists timings for their models; we picked our model to meet timing demands on a cell phone.

I don't think CPUs are going to give you decent performance unless you microoptimize the inference for the host architecture and even then the gains are marginal.

If you want to avoid expensive GPUs, take a look at things like Intel's Neural Compute Stick, Google's Vision Kit and the Edge TPU. But you can also run models on modestly priced cell phones- we use $100 cell phones to do object detection and tracking, but it's not pretty.

BTW, try SSD, it was even a bit faster than YOLO2.

Checkout MobileNetV2 as a backbone, the paper uses it for SSD and I believe weights are release.

Agreed. I wish that more effort would be made towards reducing inference time, since that is the limiting factor in many real-world applications.

Speed is considered a fundamental problem, e.g. the aforementioned YOLO and _Faster_ RCNN.

If you look for "tensorflow raspberry" on a search engine, you will find several pointers about running TF on cheaper hardware. You would probably get about 1 frame per second with it.

You may also look at recent SnapDragon-powered dev boards, which have TF core accelerators, and provide a serious boost of FPS for less than $700.

Also, using a cloud ML service is an option if you have only a few inferences/day to process

Have you read the linked paper? It finds no evidence of such overfitting happening, despite common fears of this phenomenon.

Quote: "This shows that the current research methodology of “attacking” a test set for an extended period of time is surprisingly resilient to overfitting."