Show HN: OpenCV based multi-object-tracker with various object detection models | Hacker News Reader

Show HN: OpenCV based multi-object-tracker with various object detection models

github.com

105 pointsadipandas21 comments

I work a lot with object detection at my current role. Would be cool to add Detectron 2, seems that this library that has some of the state of the art(SOTA) models in object detection. https://github.com/facebookresearch/detectron2

Also the TF Object Detection API has a suite of pretrained SOTA object detection models https://github.com/tensorflow/models/tree/master/research/ob...

What makes object detection really valuable is being able to finetune a model to detect objects that are relevant to your problem. However collecting, labeling, takes a lot of time since the best way is to manually label. But 30-50 hrs of manual labeling can get you some great results when you are leveraging pretrained models. Building a tool to manage data labeling and training for finetuning object detection models would be very valuable.

Also, what is a hard but an important question to figure out is how well are these models are preforming, and will they perform well to your problem. This paper gives a lot of insight into how image size and other factors really impact performance of modern object detection: https://arxiv.org/pdf/1611.10012.pdf. How large or small your object is in your dataset is also something to consider since there is a limit to how small of an object your detector can recognize.

Also, I would recommend this repo to better understand how object detection models are evaluated as well: https://github.com/rafaelpadilla/Object-Detection-Metrics

> Building a tool to manage data labeling and training for finetuning object detection models would be very valuable.

Something like recaptcha?

@mendeza, Thank you for the feedback. I am planning to add detectron models as well to this repository.

Initial intention of this project was to avoid direct dependency of any deep learning framework. I have only used pre-trained models and opencv for this project.

Making a tool for data labeling is an awesome idea. I will definitely like to work on it. We can collaborate on this if possible. You are also welcome to send in your pull requests.

This is a good idea! TF Object Detection framework takes a lot of effort to use on simple ideas, so frameworks that enable more abstraction the better!

Facebook has such awesome AI tools open sourced. I am more impressed with their code than Google

Pytorch makes it even more fun.

Check out our open-source tool called Label Studio, https://labelstud.io and we have a commercial offering that does exactly this, finetuning the models.

It would be great if your tool integrates popular object detection frameworks such as TF Object Detection API. Would save people a lot of headache installing and configuring in order to begin finetuning

I actually checked out your github repo and saw your framework that enables integration of trained models into your tool for active learning. Looks awesome! https://github.com/heartexlabs/pyheartex

I've built several object detection systems and you definitely want to fine-tune your model to get good results, even if the object class you want to track is available in a pre-trained model (often it isn't).

For example, pre-trained models don't work too well at detecting people in surveillance footage when the camera is mounted on the ceiling and people are at an angle in the footage. Off-the-shelf object detection models also don't work well in poor lighting conditions but can work well if you fine-tune one of the pre-trained models for your dataset and lighting conditions.

Here's a great and fun tutorial on how to fine-tune an object detection model using TensorFlow (it's a surprisingly fiddly process but worth the headache): https://towardsdatascience.com/how-to-train-your-own-object-...

I suspect you can use your fine-tuned model with this repo by replacing the `get_ssd_model.sh` compiled graph with your fine-tuned model graph.

It's also a fun exercise to implement the tracking yourself - here's a solid tutorial to get you started: https://www.pyimagesearch.com/2018/08/13/opencv-people-count...

The top-down view problem is what I was facing when I was trying to use a detection model for a project with quadcopters. For this I had to train my object detector with the data we collected from that view. YOLO was used for this work.

Yes, replacing the fine-tuned model with compiled graph should work. I tried to keep it simple and intuitive to change the model so as to allow the use of any other custom models.

Have you tried using object attributes/features to enable tracking instead of IOU-based tracking? I suspect this tracker falls apart when there are significant obstructions. Thoughts on a Recurrent YOLO or DeepSORT based tracker?

Yes. I am actually looking into this idea. I thought instead of recurrent yolo, Correlation Filter is much simpler.

I was referring the following papers:

1. https://ethz.ch/content/dam/ethz/special-interest/baug/igp/p...

2. https://zpascal.net/cvpr2018/He_A_Twofold_Siamese_CVPR_2018_...

3. http://openaccess.thecvf.com/content_ECCV_2018/papers/Yunhua...

I've found that IOU based tracking is surprisingly reliable (my dataset was primarily surveillance footage with a static camera) if you take measures to smooth the data and predict object motion using a kalman filter.

Kalman filter is also one good way to reliably keep track of objects under significant obstruction.

PyImageSearch is a fantastic resource for learning this stuff. Highly recommend it.

Question for HN experts: what would be the best approach for object-detection when the object I am trying to detect is a specific image (not a class of objects) printed on a 2D surface (therefore it could be partially folded / curved, but still more of a distorted 2D object than a complex 3D one)?

I would check out template matching or using OpenCV feature extractors such as SIFT, SURF, ORB

Template Matching scans an image with your template, smaller image and finds the location with highest cross correlation: https://opencv-python-tutroals.readthedocs.io/en/latest/py_t...

Template matching assumes the example image wont deform or take on different orientations.

Popular Feature Extraction finds and derive more robust features that help find objects in the scenarios of distortion, orientation, and scale.

Link for feature detection and matching: https://opencv-python-tutroals.readthedocs.io/en/latest/py_t...

Pyimagesearch does a cool method where they enable template matching to work at different scales. It ignores color information (by using edge detection), which could be a critical feature if the object has specific color:

https://www.pyimagesearch.com/2015/01/26/multi-scale-templat...

There arw thousands of these repositories all over the place...

Okay, I am not sure about others. But I was working on this as a fun project. And I wanted to share my work. Hope you find it useful and please let me know your suggestions and any additional improvements I can make.

Reply on news.ycombinator.com