Learning a Probabilistic Latent Space of Object Shapes via 3D GAN | Hacker News Reader

Learning a Probabilistic Latent Space of Object Shapes via 3D GAN

3dgan.csail.mit.edu

58 pointstacon8 comments

So it can map between objects and latent representations both ways, and the latent representations are disentangled (additive) - you can do vector math with them to combine concepts.

I'm wondering how far we are from using such models to boost robotics. A robot that understood the world around it would move much better and be able to perform actions.

What is the bottleneck? Could it be that GANs are too slow for robotics, or that we still can't control a humanoid to do basic tasks such as walking, grasping, pushing and other manipulations?

From watching robot videos I get that we have dexterous robot arms and legs, we just don't know how to use them to achieve useful things in unstructured environments.

I'm sure there's a bottleneck somewhere or we'd have smart dexterous robots today.

Being too slow for robotics is a extremely general statement. A robot that needs to drive a car is very different than something that has to move things.

The bottleneck is the fact that this is a ongoing research topic and integrating these things into a robotics system still requires effort.

> The bottleneck is the fact that this is a ongoing research topic

That's a general statement as well. It's like saying "the bottleneck is that we don't know how to do it" which leaves the question unanswered: what is the stumbling block, what is keeping us from being able to do it now? such as: speed / sensors / not enough robots and funds for researchers / esoteric machine learning considerations / something else.

I'm trying to understand the slow progress in robotics as contrasted to the fast progress in deep learning.

Actually, our "dexterous" robots are below human beings among many axes.

Weight being high is one of them, but also sensors for present pose, present load, anything touching the armature, flexing, .... Humans are much better at that, and it's important stuff!

Sensors can't be the only reason, since humans can tele-operate robots through vision alone (no tactile or force feedback) to do things that no autonomous system today can.

GANs are unstable and hard to train. Researchers have come up with a series of hacks to make it work in some cases, but it's not trivially applicable to arbitrary datasets.

It's exciting to see deep learning gaining ground in 3D. I'm wondering how quickly until deep learning beats 3D reconstruction. State of the art algorithm used in practical applications still does not use deep learning: http://www.gcc.tu-darmstadt.de/media/gcc/papers/Waechter-201....

The exiting thing for me is at 1:30 in the video. Smooth changes in the latent space lead to smooth changes in the shape (eg. the chair arms gradually recede along their longitudinal axis). This means you have good generalisability (the network is robust to input noise) and composablility (the network can produce novel results through transformations in latent space).

Reply on news.ycombinator.com