Hacker News Reader

Comment by in3d | Hacker News Reader

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat.

1. https://twitter.com/emostaque/status/1595731407095140352?s=4...

View on HN

I predicted back when they started backpedaling that there's a chance that sd1.4 or 1.5 will be the best available model to the general public, for a very long duration, because the backlash will force them to self-castrate themselves.

You can see nobody likes this new model in any of the stable diffusion communities. It's a big flop and for a good reason. The reason it was so successful in the first place was because you could combine artist names to get the model to the outcome you want.

I'll again remind anyone who thinks they might want to use this to download a working version of SD now. They might break their own libraries in the future, and getting SD1.4 could be a real hassle in a year or so. Getting the right .ckpt file, which can have pickled python malware, is not so trivial, and this will get worse in time.

It's going to diverge into castrated official model that intentionally breaks the older models and older models from unofficial shady sources that might contain malware.

That's like saying that obtaining the On the Origin of Species or Linux kernel will be harder in future. If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics.

>If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics.

I suspect for similar liability issues as SD 2.0, that they will not strt embedding sub-2.0 weights into consumer electronics.

These models will always work best with open datasets and open platforms for this reason.

Social media/"AI ethics" pressure groups will eventually come from these organizations (see Meta's recent debacle with Galactica). Being an unknown org without these pressures was a big reason Stable Diffusion got so popular in the first place.

As someone completely unfamiliar with SD but interested in playing around with it in the future, what exactly should I download, to have a fully local instance of 1.4 or 1.5?

For Macs there's Diffusionbee with a no-brainer setup.

AUTOMATIC1111's web UI, with the models among dependencies.

https://github.com/AUTOMATIC1111/stable-diffusion-webui

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki...

I'm not comparing with the others because I don't have experience with them, but https://invoke-ai.github.io/InvokeAI/ is great, with an easy install and active development.

Mixing artist names was by far the most effective way to create aesthetically pleasing images, this is a huge change. DreamBooth can only fine-tune on a couple dozen images, and you can't train multiple new concepts in one model, but maybe someone will do a regular fine-tune or train a new model.

I'd be curious how well the model still performs given such prompts. Disparate concepts, interpolation, n' all that. Surely it performs worse - but I bet it gets closer than you might think.

Here’s a comparison study on Reddit: https://www.reddit.com/r/StableDiffusion/comments/z3ferx/xy_...

It does look like artist names have a significantly reduced effect.

Oh very cool! Indeed filtering the data results in outputs closer to DALLE-2.

That really depends on whether you mean 'like artist X' as 'aesthetically pleasing'. I was fooling around with furry diffusion and got to try a few different models. Yiffy understood artist names, and furry did not: it had further training but stripped of artist tags.

All these models are pretty good as that community is strong on art, styles, art skill, and tagging, causing the models to be a serious test case for what's possible. The model with artist names was indeed capable of invoking their styles (for instance, an artist with exceptional anatomy rendering had it translate into the AI version). The more-trained model without the artist names was much more intelligent. It was simply more capable of quality output, so long as your intention wasn't 'remind me of this artist'.

I think that's likely to be true in the general case, too. This tech is destined for artist/writer/creator enhancement, so it needs to get smarter at divining INTENT, not just blindly generating 'knock-offs' with little guidance.

What you want is better tagging in the dataset, and more personalized. If I have a particular notion of an 'angry sky', this tech should be able to deliver that unfailingly, in any context I like. Greg Rutkowski not required or invoked :)

This is extremely misleading and you seem to have confused all the other replies.

StableDiffusion 1.0 used CLIP released by OpenAI. 2.0 uses a CLIP retrained from scratch by Stability.

We don’t know OpenAI’s dataset so don’t know what was in it or how to recreate it. Nothing was “removed”.

Removing NSFW content is fine, people who care about that can work around it easily. Removing celebrities and commercial artists was a mistake though and I expect this will need to be really impressive in other ways or people aren't going to bother using it.

It's remarkable, this sense of entitlement people have. You literally have a computer program here that can make photorealistic imagery of almost ANYTHING you ask it to, which was impossible even half a year ago, and here you are complaining that people won't use it unless it incorporates all of the protected imagery of famous artists and celebrities. Amazing.

Is it entitled to think that a 2.0 will not have regressions on useful functionality?

yes

Loss aversion is strong in humans.

If version 2 is worse than version then obviously a lot of people will use version 1. That doesn't make them entitled.

I don't think you're using the principle of charity here ("make the best interpretation of a post"). The person isn't complain, he/she is just saying that people will probably return to v1 unless v2 has something impressive to compensate.

does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change

Based on sample images I’ve seen, “Greg Rutkowski” doesn’t work anymore for example.

Ran the model locally. Neither "trending on artstation" nor "Greg Rutowski" make any differences to the image anymore.[0]

I suspect that people will find keywords that would improve the aesthetics further again, or that fine-tuning will also take place.

[0] https://imgsli.com/MTM1ODQ5

“Prompting” and “keywords” are not an essential part of this technology. If you like tokens, make your own tokens with textual-inversion or image inputs.

this is really disappointing

Reply on news.ycombinator.com