Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

I thought the cats looked like abominations.

The interpolating animals look so much like my dreams it's unsettling. I often wake up when things get too "unrealistic" and people or animals are changing shape too rapidly. Whenever I see something come out of a NN that looks like something that came out of my brain I get a little future shock.

> I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once.

It's not like they trained this on a GPU farm. According to the paper [1], they "trained the network on a single NVIDIA Tesla P100 GPU for 20 days".

[1] http://research.nvidia.com/sites/default/files/pubs/2017-10_...

That is true, but if you don't look at them as a machine learning method, but rather as a computer graphics method, then it is quite impressive. It has the added benefit of being allowed to overfit as long as the average human does not find out. If you optimize for psychovisual metrics, GANs are fine.

Seems like you could validate using random points in the latent space, and crudely verifying that the nearest neighbors in image space aren't similar.

This is also relevant to your comment: https://arxiv.org/abs/1705.07663

My interpretation of this "problem" is that it's a radically efficient solution to the problem of generating content in absence of outside reinforcement.

Said another way: If we gave a human the task of: [make a painting of a bridge] using a handful of examples of bridges as inspiration, and they did a 1-1 copy of one of them, it would be the most efficient result. However there is generally a culturally implied task of [the new painting should not be a direct replication of one of the examples].

So this "problem" with GAN's is a novelty requirement which is not explicitly built in to the generation chain.

An GAN that just made verbatim copies from the training set would not be able to smoothly interpolate in latent space. Also, see Fig. 10 in the paper at http://research.nvidia.com/publication/2017-10_Progressive-G....

This is a general problem for generative models, regardless of whether they are explicit latent density (variational autoencoders) or implicit density (GANs) or fully-observed (PixelCNN). To date no consensus has been reached in the research community on what the "right" way to evaluate a generative model is: e.g. Parzen Windows estimators, log-probability, ELBO, etc.

Yann Lecun himself (as of NIPS2016) was pretty critical of probability-based metrics, as those have strong dependence to the choice of model (e.g. if the model is poor the log-probability is meaningless).

In GANs, the critic and the generator are trained w.r.t. each other, reaching some kind of equilibrium. A recent proposal that seems to be "ok" for evaluating GANs was proposed by https://arxiv.org/abs/1705.05263, which is to train a separate critic on the generator, for use in evaluation (the generator never sees gradient information from this critic). This evaluation critic approximates the Wasserstein distance. One could imagine actually training the independent critic on a validation set of images not seen by the training set.

Yeah. There are a lot of really sketchy transitions that don't look human in that video. Also, when the picture looks fully realistic, you can Google search it and find that there are lots of very similar images.

Now, the fact that they usually aren't identical to Google's finds is moderately impressive. So is the fact that some transitions are "smooth" - being able to move from one head/eye position to another. But the difference between drawing a face and copy-pasting someone's face onto a different hair+background is very significant, and quite often the algorithm seems to be doing the latter. (And in any case, GANs are clearly not the way humans draw faces.)

It would be interesting to see someone try to do the same thing without a neural network. How far would they get on the same training set? The dataset is 30,000 pre-aligned and cropped images. (Would be nice if there was a searchable version to make sure generated versions are not identical to something in that set.)

I bet you could get pretty far with just matching and region replacement, plus some color corrections. But not one would pay you for that.

There are a few things to say about this:

1. If you train a conditional GAN to do image inpainting (for example, left to right), it should be quite apparent the degree to which the model is copying and pasting the training set - by running the model with "given" parts from the test set.

2. I disagree that an ideal GAN could just output the training set. I think the right conceptual framework is that any generative model is trying to produce a distribution similar to the data distribution, and we try to accomplish this by using samples from the data distribution. So if the model memorizes the training set, then it isn't actually that close to the true underlying data distribution. In likelihood-based models (for example the usual generative RNN) you can test this by evaluating likelihood on a validation set.

Exactly as the actual celebrities then... I think i recognized one of them.

Oh, damn. I've already had enough of fake personas, but now there is virtually no limit.

Had the same thought! Imagine how much time they could have saved drawing it all!

Yea, but this looks much better. http://blendberg.com/pl/aktualnosci

Neural Networks are really good at identifying patterns in data. As a classic example, if you wanted to predict housing prices, you could build a data set that maps features about houses (square feet, location, proximity to Caltrain, etc) onto their actual price, and then train a network to recognize the complex relationship between features and pricing. Training happens by feeding the network features, letting it make a guess about the price, and then correcting the guess (backpropagation).

Convolutional Neural Networks work similarly, but with images. Instead of giving a CNN discrete features, you'll usually just use the pixels of the image itself. Through a series of layers, the CNN is able to build features itself (traditionally things like edges, corners) and learn patterns in image data. For example, a CNN might be trained on a dataset that maps images onto labels, and learn how to label new images on its own.

This video uses Generative Adversarial Networks (GANs) to actually generate new images. In this case, you have two networks "competing" against each other. One network is a traditional CNN trying to identify is an image is "real" or computer generated, and the second network tries to generate new images to trick the first network.

We've been able to generative fairly realistic small images before (usually 64x64), but doing it this well on high-resolution (1024x1024) images is unprecedented.

http://kvfrans.com/generative-adversial-networks-explained/

A neural network is fed hundreds of thousands of images of celebrity faces. From this it learns the typical characteristics of human faces, and can then be used to draw new faces with random combinations of those characteristics.

This work is interesting because it is the first one that has been this successful at high resolution, so the output images are large and detailed rather than postage stamp sized.

Typically in a neural network, you train a single network against a single loss function that is known in advance. For example, an autoencoder is (usually) a neural network that has a chokepoint somewhere, and is trained to reconstruct the input image. Since there is a chokepoint (a layer that is significantly smaller than the input), it learns to compress the input and reconstruct it. Sort of like a lossy image compression. To train it, and tell how well it does, we can just measure the output against the input (difference between then reconstructed image and original input image). This tells us how well the network does, and gives us a well known loss function we can use in advance.

But what if we don't have a loss function? Or we don't know it? (for example, how do we even measure "what makes a face a celebrity-like face?") In that case we can train it against another network that is itself trained to differentiate between a "real" input and a "fake" input. The new network takes an image as input, and outputs a probability that the input is real or fake. We don't know the loss function, but by alternating which batch of images this network gets (fake or real), we can tell how well it does (it should estimate the real oens are real, and the fake ones fake). By training these two networks in tandem, we can use the information from the new network (the discriminator) to tell the old network (the generator) how to generate new, better images. This way we don't really need to know the loss function in advance, between the discriminator serves as our loss function.

That is the general idea. In practice, it's fairly non-trivial to get these two networks to work together nicely... often one will get much better than the other, which prevents the other from learning.

In this particular paper, they are using a technique to expand the size of the images to much larger than you would normally be able to.

Nice idea.

In the us, child pornography with digital actors has already been ruled to be legal.

Note to all: Please don't do this. Lots of missing kids out there and the agents working in this area don't need anymore distractions.

I would say that it probably is. If you trained a network using illegal data (e.g. cp images) then not only did you have to have that data once, which is of course illegal, but the data itself is at least partially encoded in the network weights which I think should make it illegal.

Written in Theano, of course! R.I.P.

Interesting thought.

apropos: https://en.wikipedia.org/wiki/The_Thirteenth_Floor

> I am not sure if I am a toy simulation of the past from the future, right now.

Think about this: what is the only generation that lives in times when massive recording, storage and communication is possible, but was not yet influenced by AI? Ours. When AI will want to simulate a "natural" human society, it has only us as templates to train its humanGAN on. Even our past comments on reddit and Twitter have been used many times to train dialogue systems - we're being uploaded with each post we make.

> In another 300-500 years

instead of "In the future", triggers me. Do you know something we do not? I am resigned to the notion that everything is up for grabs.

> I am not sure if I am a toy simulation

We need "reality discriminants". The trippy thing is if they could exist and their output is not necessarily boolean. There would be a threshold point at which beings can exist along the simulated-real spectrum, were by they can understand the output of the "reality discriminants", yet they are not real.