BodyPix: Real-Time Person Segmentation in the Browser with Tensorflow.js
blog.tensorflow.org
blog.tensorflow.org
E.g. by making it easier to distinguish parts of the body, identification via them, or gait analysis, becomes easier.
Are there compelling use cases that make this risk worth it?
Or are we just producing tech because we can?
It's a shame there isn't a Python TF implementation of this (apparently due to a different file structure between the tf.js model and TF SavedModel?), and it doesn't sound like it's a priority for the team.
So at the moment you have a choice of working with a version that's officially EOL (1.15) or using a version that has several critical features broken or missing, and on which TF Model Zoo doesn't work. Oh, and EOL version is also busted beyond belief. Worse yet, Google does not reward maintenance work, and since this is considered "launched", you can count on the bugs being there for an extended period of time until Google itself moves to 2.0, which I'd wager is unlikely to happen anytime soon. Facepalm.
That said, I think in a few years with enough elbow grease ONNX could become pretty exciting.
As to what people should use for research: just use PyTorch. I switched to it when it first came out, and never looked back, _except_ when I need my stuff to run on e.g. a phone or something, which is something PyTorch won't help you with.
Meanwhile Julia works just fine on Windows.
I don't see Tensorflow-Swift going anywhere.
There is a large subcommunity that doesn't enjoy being forced to drop down to C for anything performance.
Julia already counts some banks on their list, that tend to use ML languages instead of Python due to performance.
Julia's value proposition is mostly geared towards people trying to do things that are just too difficult, awkward or slow in other languages. It's not just about speed, dynamism and friendly syntax. It's also about solving the expression problem and providing unprecedented composability.
So while it doesn't have great appeal to end users, it does appeal to people who make the sort of things end users want so the community is currently swelling with very talented people making state of the art research libraries. Honestly, whether these libraries attract hordes of 'scripters' or not, doesn't really matter to julia's usefulness to the people who use it currently.
But I think that as julia's package ecosystem evolves and matures (We've only been at 1.0 for a year!), it's going to get more and more attractive to end users.
I cringe a little every time I see a julia advertisement that centres it's premise around "use julia 'cause it's fast!".
I love how fast julia is, but that really isn't the important part.
1) That's not the most robust solution, especially for deep learning because auto-diff tools don't cross language barriers well
2) If you have an argument for why 1) isn't a big deal, then the next step would be demonstrating why that argument doesn't apply to julia just as well (if not better) than Swift. In fact, Julia has a great interoperability story with other languages, one good example is DiffEqPy and DiffEqR which are Python and R packages respectively for accessing julia's state of the art DifferentialEquations library.
Rules are simple: shoot at a foe until it raises its hand, that means the person was shot.
Unfortunately (?), the available software at the time was subpar, and I forgot about it because of two problems:
* How to identify a foe? It would need facial/body recognition from a few set of examples (team pictures before starting, better if it shots everything else).
* How to know when to stop shooting?
These new technologies would make almost trivial to build such an application, and make it quite reliable. I guess there is a market for this in softair... I'm even tempted to have a go at it, BUT... I don't think I will because I now have to ponder the ethics of building something that can take a gun, identify targets, aim at them accurately, and fire until they are down. I am usually all in on open source software, but that's just the kind of things that sound dangerous to share (same as defense distributed, TBH).
I hope enough persons share the same misgivings, otherwise there is no point for me to refrain from building this. (Except perhaps spending my time on project that are actually useful for humankind).
I wonder if it is possible to make an estimation per actor / producer / studio how many alcoholic, drug, cigarette they consume on average. Would be awesome to see if there is a correlation with project budget. And maybe even more important is there an increasing trend of average number of drug consumptions per hour.
Consider the defense applications!
That said, I think TensorFlow.js is awesome on so many things: examples are easy understand and the build system makes them easy to run, many possibilities for using trained models in browser based apps, on my Linux GPU deep learning box training models is fast, great documentation, etc.
In some ways I like the TensorFlow.js ecosystem better than the C++/Python ecosystem.
So far, Swift TensorFlow has been a disappointment for me, but every few months I check it out again.
It looks like the interface returns 2D coordinates for the joints, not 3D coordinates; so it can't directly create 3D animation data.
It was also very jumpy in my browser, so it would require smoothing to be usable; or maybe if it wasn't running real-time it would be more accurate?
What company do you work for? This is super cool.
Can you talk about the rest of your stack? Or other projects of a similar nature?
The setup was essentially an app (c++ engine on osx running models), ingesting camera feeds, outputting skeletons, running sound->mouth poses, to unity which puppeteered 2D sprites.
On-set we output to a monitor which mixed the real shots with our graphics.
Nothing crazy! But it was a solid setup (except for overheating DSLRs)
Only Firefox/MacOS works for me.
Is this library MIT license or something else? I would try using this to cutout people from images/gifs for my animation platform SuperAnimo:
Params I used were {architecture: 'ResNet50', outputStride: 32, quantBytes: 2}, with segmentMultiPerson function.
The same images DeepLab handles perfectly. [2] If anyone sees a problem with the params/method I used let me know.
[1] https://superanimo.com/bodypix/
[2] Deeplab: https://github.com/tensorflow/models/tree/master/research/de...