Hacker News Reader

Comment by state_less | Hacker News Reader

I'd love to see AI, using games, master the art of determining a depth for objects in the scene. If you ask a person, "about how far away is that car?", they often give you an okay answer that is at least in the same magnitude as the actual distance 1 m, 10 m, 100 m, 1000 m. If AI could do that, you could then navigate an environment in the real world better using only a camera or two. So you start with a virtual world that looks real, train up the bot, then use it to navigate in the real world. Has this already been accomplished?

View on HN

That's a great (and hard) problem!

More generally, imagine AI that could learn the physics of the world. For example, if the ball is rolling away, the AI should be able to predict that the ball will look smaller on the next frame.

Going further, if the ball is about to roll under a shadow, the AI should predict that the ball will become a darker shade of green.

(After several years working in a robotics research company, these kinds of capabilities are exactly what we determined would be necessary for robot AI.)

Agree, it's not easy. Learning the basics, for example projecting a rectangle with 3d coordinates to 2d coordinates, then feeding the 2d coordinates into a NN and ask for the (depth) third dimension. Can you teach the NN a perspective transform? Can you rotate the rectangle and recognize rotation. Can you add other rectangles to the scene and detect each? Can you add color and lighting to infer more properties and get better results? Shine some more info on the problem ;)

These are like unit tests of AI (basic shapes and transforms) and I agree physical reckoning is at the top, one of the big tests that is a capstone and something beautiful to behold in nature (eg. sports). Maybe the a virtual soccer game at the end?

From my lidar experience, I wanted to reach for a model rather than deal with noisy sensor data. I want to generate the output (3d world) with my model, then the NN learns the inverse (eg. the scene graph used to generate the scene).

I enjoy thinking about this stuff, though it really makes my head spiral sometimes when I relate it to my own reality. It's easy to feel like you're losing touch.

This is a trivially easy problem if you have stereo cameras, just like humans.

Reply on news.ycombinator.com