Nvidia releases Alias-Free GAN code and pre-trained models, naming it StyleGAN3

Looks like lizard people...

I wonder if you can't just put a bunch of results with and without artifacts into two different bins, and do another round of training on them. But I don't know enough about how style transfer and retraining of these nets and all that modern stuff works to tell if that is feasible.

I suspect because there's a lot of entropy in hair and because of the shape of the optimization function, (which might even have a spatial term) a regular pattern in such a noisy and hard to learn region falls into a local minimum while the rest of the image converges to the true minimum. There's a little meat left to optimize here, but you need to do it cleverly because there's no reason for a neural network to learn all the many combinations of hair pixels in this application. That could require as many parameters all the neurons involved in generating the faces, I'd bet.

Someone on Twitter pointed out that it has larger repeating patterns too https://twitter.com/Zergfriend/status/1408184663420510209?s=...

Also the some teeth are getting drawn in front of the lips and the pupils are not round. I think the not-round pupils make the eyes point two different directions, kinda unsettling.

https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-...

https://nvlabs-fi-cdn.nvidia.com/stylegan3/images/stylegan3-...

> It's important to see this sort of thing happening more and more.

Why? If we insist on the authenticity of images, this is holding on to the old status quo in the same way we apply book and record copyright to digital content. We don't allow what the tech enables to the fullest, but we restrict it by pressing it into the old mold (e.g. by using DRM to make music a commodity).

I think "photographic proof" is a historical accident of the 20st century (and it was never perfect, those with resources could always manipulate pictures to some extent).

As a thought experiment, it might be interesting to imagine what happens when you "open up the dams" and are able to synthesize any image you can imagine! In the beginning, this will cause a lot of trouble (say with harrasment and fake news), but I believe society will adapt quickly. I think right now there is a real problem with the internet remembering too much (pervasive surveillance on the one hand, and constant risk of moral outrages for stupid things you did in your past). It would be an antidote if nobody could believe in any picture anymore.

Just run the whole codebase through OpenAI Codex, then regenerate the source code.

Can't copyright an inferred artifact ;)

You are talking about the models, right? If you train your own model on your own data without transfer learning (or with transfer learning from a liberally licensed third party model once those exist) then you can do whatever you want to with your model, no?

Nobody will be able to tell if you stick it in a tiny thumbnail and give it enough jpeg artifacts.

I was joking, but it appears that StyleGAN3's new approach allowed it to develop unsupervised 3D maps of faces within the deeper layers, which might result in interesting things when hacked by researchers.

And it fits pretty good too. GAN Nvidia Alias-free Models Style

As defined in the license, the capitalized term "Work" means only the StyleGAN3 software and derivatives. So it means you can't use StyleGAN3 while simultaneously claiming it infringes one of your patents, but it doesn't mean Nvidia can use StyleGAN3 against you as leverage in an unrelated patent suit.

I'm not a lawyer, and I won't comment on whether this is legal, but I'll note that it's quite similar to the patent clause in section 3 of the Apache Public License.

https://www.apache.org/licenses/LICENSE-2.0

Apache 2.0 has a very similar clause. But there might be some subtle differences in the wording that makes this broader or stricter.

Yes they are legal, and I'm not sure I follow the argument that they shouldn't be

It is legal, yes. That license is awful and proprietary (3.3), but it's most definitely legal.

It's not on purpose but a direct consequence of the computational requirements of the model.

I say this often and I cannot stress this enough: this is not magic! High quality results require a lot of computation and with it a lot of other resources like VRAM.

This area of research has always been resource intensive and no one was surprised by this back in the 1990s when some research required hardware that wasn't available to mere mortals (looking at you, SGI Indigo and Octane).

You should look at it be the other way around: it's astonishing that you can use off-the-shelf consumer hardware (here: mid-range gfx cards) to participate in and benefit from cutting edge research.

Even better yet, services like Google CoLab even allow you to meddle with this for free if you don't own the required hardware.

BTW, it's not the researchers' fault that the market is f'ed up again due to shortages and crypto mining. Otherwise you'd be able to buy a 12 GB card for around $330 (e.g. the RTX 3060), which doesn't sound particularly outrageous to me.

We need FOSS GPUs + compute!

Nvidia makes most of their money selling GPUs as computational accelerators-- both generic CUDA and neural network applications.

They don't _need_ to profit off their ML models. It's a value added service. Ensuring their hardware has marketshare at the leading edge of the ecosystem is the main point.