There's probably a project to make here. Take those pixel2pixel models that are able to generate images from drawings, generalize them to 3D, say a pixel2vertex. Would be pretty neat.

The key would be to find a way to constrain the output space so that it's not so vast that it won't be possible to converge to something useful.