OpenPose: Real-time multi-person body, face, and hands estimation | Hacker News Reader

OpenPose: Real-time multi-person body, face, and hands estimation

github.com

150 pointssytelus35 comments

Paper https://arxiv.org/pdf/1611.08050.pdf

TLDR; convnet for joint localization, old school optimization techniques (graph coloring, etc) to put the pieces back together.

Re license, I don’t at all mind research being commercialized, but the use of “open” in the name rubs me the wrong way. Who in their right mind would work on this? At best it will become a benchmark to compare other systems against.

The FSF requires a snail mail letter for copyright assignment. I don’t know the specifics, but this suggests magical auto-assign copyright clauses may not hold up.

Personally I GPL my stuff that may be commercially valuable. It allows a community to maintain an open version and produce offshoots, while you maintain your own version with only code you wrote for licensing.

I like OpenPose. This area is progressing so quickly.

DensePose was published recently: http://densepose.org/

End-to-end Pose was published late last year: https://akanazawa.github.io/hmr/

At a very high level how do they differ?

All of them take 2d images of humans:

open pose -> 2d points of major parts. (generation of 3d points is under development)

end-to-end pose-> 3d mesh of human body

dense pose -> 2d uv maps that can be placed on a 3d mesh

I should also mention:

AutoHair -> 3d mesh of hair from 2d image

http://gaps-zju.org/autohair/

ClothCap -> 4d cloth transfer to different 3d human model

(this one doesn't do "2d image to 3d cloth" but probably something that can be done starting with ClothCap)

Face -> 3d mesh of face

https://github.com/AaronJackson/vrn

The training data comes from

>Check https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimatio... for training questions. We are using their model.

https://github.com/CMU-Perceptual-Computing-Lab/openpose/iss...

What is the difference between this one and the other one ?

This looks promising. Any thoughts on the license?

no commercial use it seems.

Not 100% related, but the copyright holder(s) are not clear. There is a list of authors so I'd bet they are the copyright holders. But given the fact their email addresses somewhat point to CMU, I wonder if CMU is behind...

The Github account is from CMU perceptional computing lab. Thus I'd consider it reasonable to assume that CMU is behind.

I don't know CMU well, is it privately hold ? I really wonder if the work we see here was paid by taxpayers' money. I guess the papers are freely available but the implementation is not.

I'd say it would cost 5% royalty on sales assuming that algorithm was key. Ten years ago I negotiated a licencing agreement with CMU for a computer vision algorithm. I was leading an unknown startup. 5% with no upfront fee is what we ended at.

The PI on this project (Yaser Sheikh) also heads up Oculus' Pittsburgh lab. Not sure if there is any overlap on this project.

The prof is also associated with oculus/fb, so the license has to be friendly to them as well.

There's actually a link somewhere to a page[0] that, all the way at the bottom states the price as USD$25,000 for a "Commercial License, yearly renewal".

[0] https://flintbox.com/public/project/47343/

When I saw this a little while ago I thought it would be great for automated video analysis of barbell lifts / weightlifting given the angles involved in such things can be a good metric for form.

However, the license specifically states that you cannot use it for sports. I wonder if they singed a high value exclusivity agreement with someone in that space, or if some such company funded the research originally...

"The non-exclusive commercial license cannot be used in the field of Sports. ("Sports" shall mean any and all athletic competition between individuals, groups of individuals or teams.)"

"You agree that all and any such derivatives and modifications will be owned by Licensor"

Assigning ownership of your own modifications to them seems significantly stronger than forbidding redistribution of your modified version. With this, just getting the examples to run on your own machine is effectively free development for the authors.

Is that legal? You can claim ownership of derivatives??

Sure, you can claim anything.

IANAL, but I don't see why this would not hold up. It is their code, its in their license agreement (which is a contract you agree to). If you do not wish to give them the rights to your modifications, the license forbids you from using it

Sure, but I was of the impression that they can only control rights to use their software, not claim your own - I'm not sure if "which is a contract you agree to" is true if nothing is signed.

Note: by reading this notice you agree that all that is yours is mine, and all that is mine is mine also. ;-)

Your note does not work because I am not agreeing to anything. You either are agreeing to the license implicitly by using the product, or you are not allowed to use it (and would therefore use code which you dont have a license to)

If you read the message, you agreed to the terms. If you didn't read it, then how can you comment on what it says? :)

if you havent read it then you are using software which you may or may not have a license to, and you're just hoping that you are allowed to use it.

The license applies, wether you read it or not. The only difference is if you are aware that you are breaking it, or not.

> If you do not wish to give them the rights to your modifications, the license forbids you from using it

Unless your use qualifies as fair use

In my observations, "fair use" is often misunderstood. People violate copyrights thinking theirs is a fair use when it's not.

Is "fair use" on software even a thing?

(I know some jurisdictions have basic exceptions like "if you own software you can do what's needed to run it", but modifications?)

Well it's an option - either you follow the terms of the license agreement and give up ownership, or you're in violation of their copyright. (Or you contact them and work out better terms.) They can put any terms they want in the license agreement, if you don't agree to them you don't get a license.

Not sure if this conditional violation stuff would hold up..

Think of it with money instead. If they charge $20 for a license, then you have the option to pay, or to be in violation of copyright. Here they're requiring a transfer of IP instead of money but it's the same deal.

Not really. It's not the same as paying. Once you pay you'd get confirmation.

What a disgrace to the "Open" prefix. Guess we need a FreeAndOpenPose instead...

How does it do on modern dance?

https://www.youtube.com/watch?v=FOZ6KnVPvIU

Given two views of the same scene, could the output of something like (NotActuallyOpen)Pose theoretically be combined to create a marker-less 3D motion tracking solution?

Couldn't this be used to automatize fire weapon aiming?

Or worse : a laser beam aiming automatically to the eyes. It'd be quite bad since it'd be cheap and unlike fire weapons, lasers are easy to build or acquire.

How long until a mass shooting uses such technology?

Realistically, if you wanted to do such a thing, you could have used a Kinect for a long time. Even face detection would work in your example, and we've had robust and cheap detection for a long time. So I assume that this is a movie-plot threat.

However, for a dramatic interpretation of your query, I suggest "Daemon" and "Freedom TM" by Daniel Suarez. The book shows groups using these technologies to great effect.

How does this compare with just calling all the OpenCV functions in parallel then merging?

It kinda seems like what they did, which if it works - then kudos, but the licensing should be easy to avoid then.

Errr, what? This project uses huge neural nets for detection, nothing like that is available in OpenCV. I think they only use OpenCV for IO (accessing webcams etc).

Reply on news.ycombinator.com