Hacker News Reader

Comment by mikeknoop | Hacker News Reader

"Eventually it [Microsoft] came around, releasing a software development kit, with the caveat that the SDK was only for non-commercial use".

So now you can use the SDK to build commercial products, but only if chosen and willing to give up 6%? Why the "walled garden"?

View on HN

I think the license now allows commercial development. Additionally Microsoft is announcing that they have partnered with TechStars giving your commercial kinect idea an optional funding route. The six percent goes to TechStars. Not Microsoft.

R&D ?

Someone should train a neural network to be able to do what the kinect does without the kinect (create 3d depth map from 2d images). Obviously microsoft won't do it since then their hardware won't be needed, but with all the millions of kinects out there collecting enough data for this shouldn;t be that difficult.

Pose estimation from just the 3D depth data took 24 hours on 1000-node cluster. I can't even begin to imagine the massive dataset + computing resources needed to pull this off.

Also, (I might be completely off here since I am not competent enough with the ML) but to take a webcam image and then append the Z coord to a pixel would require someone to sit and label their dataset before it is useful to train the classifier (in which btw, the Z-coord can take anything between 80 - 300 cm - so > 200 classes). This definitely does not look so trivial to me.

The pose estimation you're referring to is not the same a raw depth map. Also, when creating that they had limited data (they used actors and some graphics techniques). For the just the raw depth there is now practically unlimited data available (millions of kinect boxes everywhere). No hand labelling of data needs to be done, it's what the kinect outputs 30 times or more per second.

If it is so easy, then why don't you do it?

That's like an insult right? i'll let that go.

When I mean easy, I mean in the sense that a community of people can create it in an open source way. I doesn't mean that i personally can whip it up on my laptop in a few minutes.

Oh no. my 2nd paragraph was about the raw depth data. From what I see, the web-cam view doesn't seem to contribute to the depth data (firing up the streams and viewing them shows different perspectives and there isn't any way to query how far a pixel in the web-cam view is from the camera).

I remember reading somewhere that the depth sensor projects something (infra-red?) on a surface and then the transformation of the image is used to build the 3D model. So it might still need some specialized hardware (not necessarily MS hardware since Asus seems to have a similar device out in the market).

EDIT : NVM I just looked up the video of Andrew Ng's class where they discussed this. So it is possible.

There's a google tech talk video on it by the student on youtube http://www.youtube.com/watch?v=bWbEsDbfayc

I think the problem you'll face is that there are lots of ambiguous inputs: the same pixels could be an image of something small close up or something large further off, for example. That's likely to confound the training process.

The kinect dodges this problem by actively gathering extra info. It projects a rapidly changing pattern of infrared light out in front of it; based on the pattern it detects it can figure out how long the light took to reach a reflective surface at each pixel and infer the depth from that.

Of course, it will be less accurate than the kinect itself (but spreading software is cheaper than hardware so it's worth it.). One of andrew ng's students trained this knd of system (3d from images trained using 3d data) a few years ago and was accurate to automatically drive a toy car. That was using just a limited amount of data. Using data from thousands of kinect users will better results than that. It's all a matter of data, the more data you have, the more accurate you can be.

The difference between something in a lab and a commercial product -- or even something that is moderately usable in uncontrolled conditions -- is often vastly underestimated. The effort to make something real is probably several orders of magnitude harder than what's required for "I can hack something up in a couple of weeks."

I believe it has more to do with how much data you can get your hands on. With the kinect - widely available depth data, and cloud computing, I believe a lot of things that seemed difficult are now possible. Open source movement needs to start shifting to crowd sourced data and volunteered compute time movement.

A large publicly available depth image dataset could be quite valuable as a research tool.

Reply on news.ycombinator.com