3D Face Reconstruction with Position Map Regression Networks

To my human eye the facial landmark placement in this paper looks much better than the 3D reconstructions (which look pretty poor compared to the facial shapes a human eye or human artist would interpret from the photos). The authors argue that their approach isn’t limited by the explicit finite number of facial basis functions problem encountered in most prior 3D reconstruction work, but they seem to neglect that because of the finite size of the ground truth 3D scan dataset their approach is still limited in exactly the same way, it’s just limited implicitly rather than explicitly. If you don’t have enough ground truth scan data for black or Asian or Slavic or etc. faces, the network can’t learn to represent them correctly and you end up with the same low-dimensional results as the prior work, you’ve just hidden the low dimensionality from view by burying it in your neural layers rather than making it explicit in a set of easily identifiable basis functions. Sure, if these authors had a larger ground truth dataset they could do better, but so could the explicit basis function methods.

Again, the facial landmark detection looks great, my concerns are purely with the inaccuracy of the 3D model reconstructions.

Next - select your own cast for the movie you're about to watch.