Hacker News Reader

Comment by cm2187 | Hacker News Reader

Out of curiosity, what are the possible use cases for object detection at >100 fps? I assume it would have to be objects that move very fast, i.e. nothing ordinary that I can think of.

[edit] actually stupid question. I assume it's more about throughput than fps, i.e. be able to process lots of streams on the same machine, for instance for doing mass analysis of CCTV streams.

View on HN

While I'm not into object detection such as this, I can easily imagine this being part of a system where you want the rest of the system to have time to act on the information.

As such the point isn't that you can detect objects >N fps, but rather that the object detection shouldn't take more than X% of the time per cycle so that the overall cycle time can run at a given rate.

If your pipeline depends on running inference on a single frame at a time, for example some kind of control loop, then you need to be a bit careful about how you measure speed; you have to use the effective time per batch (ie batch size 1), not the amortised frames per second using as big a batch as will fit. You can still interleave processing though.

Very practical question :) Exactly as you say, multi-stream throughput. Also for faster than realtime offline processing of video. Check the caveats section at the end of the post - DeepStream is probably not well suited to high throughput single-stream inference.

For me it's only exciting because it lowers the barrier of how much I can do with a much smaller system than double 2080ti's.

Yeah. A 2080Ti doesn't fit in your pocket or in your AR glasses but the same techniques and tools scale down.

Self-driving. Ideally you want something around 1000fps and low latency, so it has time to react.

I'm sure military and sports applications are obvious too.

Doubt

Humans reaction times are much slower than that. In fact for some things it can take a whole second https://www.visualexpert.com/Resources/reactiontime.html

Maybe racing sports have shorter reaction times, but I'd be frankly surprised if it was something < 100ms

10fps for your average drive should be more than enough

Answering my own question. Possibly industrial applications like detecting objects on a fast conveyor belt. A recycling facility for instance.

100ms is generally considered to be the lower bound for a valid start, i.e. 99 would be considered a jump start in F1 iirc

But machines aren't (yet) as capable as humans at driving-situation-recognition and driving-decision-making. One way they can compensate for those shortcomings is to be superior in other ways: 100% vigilance and super-fast reaction times/decision-making.

No, that's only if you want the human to be able to make the reaction. If the application was self-driving, you'd prefer the car to react faster than a human. For a military application like projectile detection to avoid or destroy the object, you'd want something even faster.

For reference I am making a farming robot that goes at 1 meter per second and I run the main control loop at 10hz. I would absolutely run a car that goes freeway speeds at at least 100hz.

I'm not sure what your argument is. Any self-driving system should strive to be much better than human drivers. It makes complete sense to have reaction time much better than human.

I don't think you'll find a 1000 fps camera on a "standard" AV platform. And if you did, I imagine it would be too noisy to be useful without a ton of illumination.

Smartphones offer 960fps video capture since a while now.

Cloud service that needs to serve multiple requests or process many video streams in parallel. (faster performance = less hardware required, bigger scale and potentially improve end user experience - save from their data being on the cloud of course).

On device (eg mobile phone) processing with battery usage that respects the user. Older hardware/models inclusion as well.

Of course the above aren't cases were the stream itself is 100+fps, but more of broader general benefits. For a 100+ fps stream.. well there are many things that go fast, imagine you wanted a robot that tracks or catches a fly before it takes off. Flies have a reaction time of 5ms (200fps), that's why it's hard for us to catch! Expand and apply the same concept to other things (that are fast, or happen very quickly) now...

Another aspect is you might get better accuracy with larger input dimensions, but the number of pixels scales quadratically with width/height.

Food processing, recycling separation? I can imagine lots of small parts moving fast

Plenty of robotics tasks could benefit from high FPS tracking of single streams. Generally process tasks where faster=better. But yes, tracking many streams at once is useful too!

tomato / potato sorting while they are on conveyor. bigger fps, more objects to dump on said conveyor.

this would be useful for sorting cells at high speed. https://www.sinobiological.com/category/fcm-facs-facs

Smart missiles and weapons i guess

Shooting down drown swarms.

Roulette spin predictor

I guess, but that's generally illegal, and if it became common & easy then casinos would simply require bets to be placed before the ball is dropped.

Reminds me of:

http://www1.cs.columbia.edu/graphics/courses/mobwear/resourc...

Hypersonic missiles

Reply on news.ycombinator.com