Ask HN: I want to detect when arbitrary things come into a video stream | Hacker News Reader

Ask HN: I want to detect when arbitrary things come into a video stream

8 pointseof7 comments

I have seen some javascript implementations of 'object detectors' but they all seem to be limited to just faces or hands or some specific type of gesture.

Is there anyway I can just show "it" say.. some little statue I made when I was a kid and have "it" tell me if it's in an video? I want to basically: have a video stream loaded (for example, the webcam on my laptop) and then, if some _thing_ that I have predefined comes into the video, then do something with it.. basically make an HTTP call to some other service I have written.

Is there any "it" that can help me do this?

View on HN

What you are looking for is an "object recognition" algorithm. These algorithms take one or multiple images of an object (e.g. your little statue) and then try to locate it in another image.

Generally, they are computationally expensive, so performing them on a video stream, especially inside the browser, will most likely not work, even for rather low resolutions.

I have some experience with the SIFT algorithm. Takes some time to grok it as the algorithm consists of several sub-steps.

You may want to look into OpenCV, they have a couple of algorithms implemented.

I would imagine it would be pretty slow, but you could have a video stream write to a canvas element at regular intervals. You could analyze the pixels in the canvas to look for a pattern. This would be the hard part, since you'd have to write an engine to detect edges and create a hash of all detectable objects in the frame, then you'd have to match each hash to a known hash of the desired object. To get the known hash, you'd likely have to feed in many example images of your object and create hashes from these.

Well, what you describe is extremely difficult to do, and people have been trying to do with for decades now without success.

It could work if you place yourself into similar conditions over all the videos. By having a good understanding (precise model) of what you want to recognize, you could make some pattern recognition / object detection to be able to find it in other videos. But the huge problem in this field is that the conditions keep changing. Light, luminosity, color, shape, distance. All of the makes pretty much every method of recognition of "it" completely useless.

There is one thing you can currently do though. It is target tracking. Without knowing the target before, and with minimal user interaction to follow it, it is currently possible to track it quite accurately.

Hope this helps.

I've seen people do this with a Kinect I think. Obviously that's more info than just a video stream but I don't know exactly what your limits on your solution are.

Are you restricted to running in the browser?

not necessarily, but preferably

i'm no expert, but i believe machine learning / classification algorithms can help you on this. you just need to train the algorithms on what you want to find.

'finding' objects means feeding the algorithms a series of 'features' that you extract from video and normalize into data structures - color, corners, edges, velocities, relative distances, etc, which are then fed into the algorithms to produce a classification or a probability.

analyze every Nth frame and try to classify found objects. if a known object is found, do something.

Reply on news.ycombinator.com