Tracking.js － A modern approach for computer vision on the web | Hacker News Reader

Tracking.js － A modern approach for computer vision on the web

trackingjs.com

143 pointszenorocha19 comments

I can't get it to work(it detects my white shirt, the shoulder area as a face, but not my actual face). Is this a repetition of this: https://www.youtube.com/watch?v=t4DT3tQqgRM

I also cannot get it to work. The little bit of instructions there are make no sense so I don't know if I'm doing something wrong or if it's just broken on my computer.

Doesn't work for me either. It does nothing.

Face detection still being an open problem in computer vision area. While very good algorithms exist, they still fail on some cases or are not fast enough to run in real time. Our implementation is based on Viola-Jones algorithm, one of the most used algorithms for this problem (and also used by OpenCV). This algorithm can be used to track several types of objects just by changing the training data that is used as input. For the face detector, we used training data generated by OpenCV, but we don't know which faces were used to generate it. Perhaps this issue is a matter of changing the training data, or maybe this is a case where the algorithm is not robust enough to handle. We hope we can fix this problem soon, but we still don't know how (suggestions and contributions are appreciated ^.^)

"While very good algorithms exist ... or are not fast enough to run in real time."

As a research scientist at a company whose main product is face recognition software I have to disagree with this part of the statement.

That's really good to hear! The face tracker was one of the first modules to be implemented on tracking.js and, back then, we found Viola-Jones to be the most promising algorithm to make face detection (besides being the one used by our main reference, OpenCV). The whole library was refactored for this release, but we didn't change the algorithm and perhaps it is already out of date. We would be grateful if you could help us by sending papers about better algorithms (and reference implementations?).

Are they relevant to this implementation?

What I mean is this is a client-side javascript library, so it is meant to work in the browser on who knows how powerful hardware.

There may be facial recognition libraries that work very well in real time, but how powerful of hardware is required?

And the other big question, are they proprietary algorithms or open-source?

The Viola-Jones technique is still fairly competitive for face detection performance, IF you train it with a lot of data (and of course good data, which means with all the types of variations you expect to see). However, the pre-trained classifier provided with OpenCV is terrible (unless they've updated it recently).

Fortunately, OpenCV provides functions to train your own classifier using your own data, so if you can get good data, and are willing to wait for the week or two it'll take to process, you can get much better performance using the same technique.

Finding sufficient training data (labeled faces) is more tricky. For an automated approach, you could try bootstrapping it by simply running a more state-of-the-art detector (such as on Facebook, Google, etc.) and feedings the outputs into the training function (you'll need to do some work to actually get the outputs programmatically, perhaps through undocumented APIs).

Another approach is to use an existing face dataset. One possibility is the FDDB [1] although it's kind of small (5171 faces). Unfortunately, most other datasets I know of in the face community are either unrealistic images (e.g., only "mugshot"-style) or were detected using OpenCV (and thus won't help you get better).

The final approach is to label images yourself, which would require time and money (e.g., to do it on Amazon Mechanical Turk).

If you do decide to retrain classifiers, it's essential that you augment your data in various ways, such as by mirroring images (and labels) left/right, generating slightly rotated versions of faces, and at slightly different sizes. This augmentation can GREATLY improve the performance of your detector.

[1] http://vis-www.cs.umass.edu/fddb/index.html

Thanks for this great help, apu! A new issue was created on github (https://github.com/eduardolundgren/tracking.js/issues/88) to register the suggestions =)

This is really cool and is going to help me A LOT with a detection script I was working on for a WebRTC video website.

I was looking for something like this a couple of weeks ago and I decided to use python with Fisherfaces [0]. Now I'm really curious about Viola-Jones and its accuracy via JS...

[0] https://github.com/bytefish/facerec

It's funny, I look almost exactly like the guy on the left in the "Face (Image)" example, and I have a very good quality webcam, yet it does a really terrible job of detecting my own, live face.

Any examples of all-javascript augmented reality using this yet? It seems like this could be done since feature detection is implemented.

We are currently working on a pose-estimation algorithm (EPnP) that will use the results we get from feature detection to find a camera pose, so we can make 3D augmentation. It will be very helpful to create augmented reality applications.

I am very interested in computer vision libraries. How exactly does this stack up to OpenCV or is that even a fair comparison?

tracking.js is a new library that intends to bring the power of computer vision to the Web. OpenCV is far older and is a inspiration for our team regarding the number of available algorithms, robustness and speed. With the increasing power of JavaScript engines, we believe that we can achieve very good results using current state-of-the-art algorithms and low-level optimizations, so real-time CV applications can also be done on the Web. Currently tracking.js don't have as many algorithms as OpenCV, but we are making our best efforts to bring useful algorithms to the library while still keeping the API easy to use and to extend, so we can count on the community to help us to make this project a success.

Great. This is actually very relevant to a project I'm working on. It involves porting projecter/camera electronics from open cv over to the web with javascript. So how does this stack up vs jsfeat? Are those comparable at all? I do think your api is more web dev friendly from what it looks like, but I am curious about the capabilities.

One of the biggest problems we find on most computer vision libraries is the fact that a lot of knowledge in the area is required to use them properly. We think the complex APIs that are exposed to developers that try to make CV apps can be the reason why we still don't see many apps exploring the potential of CV. So tracking.js was created with the goal of being as powerful as the best CV libraries, but without exposing the inherent complexity of the area to the developers that wish to use it. From what I saw of jsfeat, it looks very powerful, but targeted for people that have some knowledge of CV.

This looks pretty awesome.

I know it's a bit offtopic but on this page

http://trackingjs.com/examples/face_hello_world.html

where can you find such a nice 'chrome browser' graphic? it looks really solid. I want to use it on my website.

Just google "flat browser frame" and pick the one you like...

Reply on news.ycombinator.com