Hacker News Reader

Comment by Closi | Hacker News Reader

I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon.

They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.

Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?

There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).

View on HN

Not true. Stable Diffusion locally on my 3080 is already faster than Adobe’s offering, not to mention after all my model tweaking and fine tuning much more useful and accurate. Is it overall easier to use or setup? No, but I can run it for free over Adobe.

It's not free for you.

Energy, partial hardware cost, setup time, fine-tuning time.

That's good for you but not 99.99% of the population

I honestly don't think that only 1 in every 10000 Adobe users own a modern-ish gaming-capable PC (and that's what's required to run SD as of now). Point being, the fact that you can already run these models on off-the-shelf consumer hardware is very promising, and as time goes on, more and more new hardware will be capable of running this.

I assume that they actually use the hardware /GPU they have and might not be able to run SD next to it constantly without upgrading.

But someone needs to make this possible and maintain such a solution which would cost also money.either you pay adobe what you already do or pay someone else who maintains the model, the infrastructure etc.

Sure some will run it themselves but my guess is that this is a niche group of people as most don't care .

OP will have been exaggerating with 99.99%, but I suspect 90% of people don't have a PC with something equivalent to a 3080.

And designing your software to a minimum-spec of a 3080 would be pretty wild.

But a 3080 is by no means a minimum requirement - not even close. A 3080 will give you a very comfortable experience, sure, but so will a midrange card from the same generation. And people have been squeezing SD on way less powerful hardware - even most cards from the GTX 10 series (that came out in 2016) can be made to run it, although some sacrifices have to be made below 6GB of VRAM.

even consumer GPUs have massive generational speedup. In the mid term, cloud will be hard pressed unless Nvidia aims at depressing consumers (leaving themselves open to attacks from everyone else on that market segment)

yes, but they could just simply add more and more memory to consumer and cloud versions of cards. So, nvidia is safe. More memory, bigger models.

memory is the non linear lever Nvidia uses to charge more for non gaming card.

People using Adobe products tend to have better hardware than the average

I also have a 3080, and there’s no way SD is faster. The best comparison as far as resolution is SDXL and in the time it takes me to generate one image at a reasonable amount of steps Adobe generates 3 options.

This is referring to the Photoshop stuff, which is way better than any type of SD inpainting for removing things from images. Firefly might be slower? I haven’t used it since it first came out.

I agree for just general image generation SD or Midjourney are better options in their own way.

SDXL, 3090, 22its/s 4090, 34its/s

xformers, 1024x1024x diffuser pipeline.

I kind of doubt those numbers, considering my 12GB 3080 gets something like 3 it/s in ComfyUI - though I suppose that's usually with a LoRA. Even if they're accurate, very few people have a 3090/4090. Keep in mind the Photoshop version would have something akin to Controlnet going on as well, which would slow things down.

You can run this on an M2 Mac Studio. That’ll be consumer level hardware in a few years.

Run what? Adobe's offering is cloud-only, and I don't think the hardware requirements are disclosed.

The models obviously. Thsse are for the most part available open source on Github

Firefly models are not publicly available. The integration with creative cloud apps is not customizable. You can't point it to a local stablediffusion setup. Open-source competitors will flop just like GIMP.

Fyi, there are Adobe plugins that do allow y you to use a local SD install.

I mean that sounds good and cool and worth trying for the average hackernews, but not for the average adobe user. Like people are not paying for AI tech to get the 80% mostly-there version, they're paying for the 100% spit-and-polished end product that has the 80% of the work needed to achieve the last 20% of productizing. Consumers have demonstrated over and over they will pay a lot more to avoid dicking around with stuff and professionals can have an even stronger tendency to do this bc their time has a cost.

I have an M1 but I first remember thinking it was very impressive that my Mac could run these AI models…

…until I tried the same on my RTX 4070 and it made my Mac look like a joke.

For the 30 seconds my Mac would have taken for 1 result, which will probably need revising, the RTX would give me 30 results.

However the RTX was half the cost of my Mac, so it’s not a good investment if I just want to generate some images. I’d rather pay for the cloud if I didn’t have the RTX already.

Which M1? That makes a big difference. And the Mac Studio runs the M2 Pro/Max/Ultra.

Yeah my M1 Max is about as fast as my old 1070, at 24 steps per image it comes out to little over 2 images per minute. My new 4090 does an image per second.

It should be about 6 images per minutes (for SD v1 at 512x512) and 1 image per minute on SDXL for the said hardware (M1 Max): https://engineering.drawthings.ai/integrating-metal-flashatt...

M2 will be a joke for the AI models of the future.

The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM). That's why their quality is leaps and bounds above stable diffusion XL, its because the model size simply allows for it.

Model sizes continuously grow to exploit the available hardware to the limit. Midjourney and GPT-4 have both proven that model quality is decisive to success and paying customers, so consumer hardware can never catchup to whatever Nvidia sells to the cloud.

How much vram does an a100 have? The M2 ultra has more than 100 GB.

I don’t think I’d say Midjourney is “leaps and bounds” above SDXL. It’s better overall, but SDXL closed the gap a significant amount.

> The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run

Do you have any source for this speculation? In my experience image models are always much smaller than language and even the largest llama will fit in a smaller GPU machine than that.

>The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM)

Unet are really expensive to run compare to a regular GPT model and they are compute-bound thx to convolution, a reason why no one has trained a unet that comes close to consume 100GB of VRAM during inference. I doubt that MJ is much bigger than SD XL, it's good but not revolutionary.

Have you tried actually downloading SD and all the various custom models and tools?

For me it kicks the shit out of Midjourney in flexibility and quality. I can make more images of higher quality, faster and cheaper.

Midjourney trades flexibility for consistent quality and consumer accessibility.

An optimized XL model in the hands of an expert beats it, handily.

And Adobe sells to experts, not consumers for the most part.

I suspect that the energy cost will be the limiting factor here. Which will constrain models eventually.

At 5usd for 100 operations, that thousand dollar gpu will break even very swiftly. I used 5000 or more runs when I was learning to use stable diffusion. In a commercial context, staff training alone will pay for the cards

My thoughts exactly. I did some guesstimates last month, the experimentation and playing I've done with my 3090 so far would have cost somewhere around $7k on the metered OPC options.

Reply on news.ycombinator.com