Charl-e: “Stable Diffusion on your Mac in 1 click”

Thanks for the shout, I've made something similar to yours (https://phantasmagoria.stavros.io/) and I needed a GPU backend. Trying out their sample script, it seems to take a minute or so to just error out with "taskID doesn't exist" or similar. Have you hit that issue too?

Can you give an estimate of what the cost is per run?

>> Back then, as in the early years ot the PC, software was mainly self-contained, single-executable extract-and-run.

Software in general? Yes. But of you tried to leave the path the manufacture prepared, then you entered a world of pain. I remember how difficult it was to connect my first smartphone around 20 years ago to my Windows PC using ActiveSync to achieve a synced calendar. Just one example of: there was no download and run solution for processes that seem simple today.

Broadly speaking the reason it's the whole "making sure you are using the correct version of python that uses the correct NVIDIA drivers via the correct Pytorch installation" that causes the issues.

This is actually a solved problem. But it's been solved lots of different times in different incompatible ways which tend to clash on any individual computer.

When you say, "Back then," do you mean 30 years ago, 25 years ago, 40 years ago, 20 years ago, or 50 years ago?

The largeness of the model does add complexity, you can't really package it as part of the binary for example.

It's not the case that useful software was ever self contained. If you recall trying to do anything online in the early to mid nineties, you'll remember how complicated it was to use almost any website and how much manual fiddling and configuration was involved to get online.

Ignoring the internet, early games and graphical applications were a mess of settings and configuration. Even today you often have to tune graphics settings to get playable game performance on anything but top line hardware.

nope, (well yes it was a reference) but was a strategy in the early '00s:

https://en.wikipedia.org/wiki/Comet_(programming)

Really? I find the difference between the Stable Diffusion code and the code you find in a typical Python package to be night and day.

Why is there precisely one test (that has nothing to do with the core functionality)? Why is the Git history full of things like “finish”, “correct merg”, “fix more”, “add code”? Where is the linting config? Why are there print statements everywhere? Why does non-UI code have UI code embedded in it? Why is there random code commented out? Why is there no consistency across the codebase? Why is everything written as if they’ve never seen a Python project before (comments that should be docstrings, docstrings that should be comments, print("WARNING: ") instead of using logging or warnings, underscores in CLI flags, no shebangs in scripts intended to be run from the command line…)

Not all Python software gets all of this right, but it’s incredibly rare to have so many misses, even for hobbyist developers. Unless it’s code released by researchers, etc. It’s pretty typical in that context.

> This is not a project made by people without dev skills.

No, but it is a project made by people lacking software engineering skills, which is a distinction I drew in my earlier comment. Like I said, they can write clever code, but there’s a difference between bashing on code until it works and building it properly. This is the kind of codebase you get when you have people who have been writing code for a long time, but never in the context of a software engineering project alongside experienced software engineers they can learn from. Put them on a team like that and they’ll be forced to unlearn most of these bad habits fast because they’d never get their pull requests approved otherwise.

I’m trying not to be harsh – I understand that this code is more of a code dump from researchers than a real software project, and they’ve done some incredibly clever things here – but if somebody is suggesting that Python projects in general are like this, it really should be pointed out that this is not in this slightest bit representative of a typical Python project.

Most deep learning these days requires CUDA acceleration to enable GPU / TPU for the libraries (i.e. PyTorch, Tensorflow), which is an absolute nightmare to set-up.

If used to work but somehow was broken on M1Mac a few days ago. Not sure if fixed. Get an older branch if it doesn’t work for you.

In my experience, they’re simultaneously important for data science and very annoying to install and manage.

Just to be clear, pandas and numpy are not the "wonky" libraries. They are, in my experience, basically two of the most easily installed and dependency managed libraries in python, given their ubiquity and maturity. Maybe there are machine configurations I'm not familiar with that they are not easily compatible, but I've never seen them cause issues. Usually it's cuda or other gpu stuff, or conflicts in less regularly maintained packages

2be honest numpy is ez to install on all major platforms. In deep learning I'm almost never saw usage of pandas but deep learning models have problems with PyTorch as some projects just lock old PyTorch version that just dosent work on new/old python version.

FWIW I don't I've ever gotten a satisfactory result from anything less than 50 steps

Wow! What's your motivation to provide access to such powerful hardware for free?

8 seconds here, awesome!

It is slower than an NVIDIA you though. Maybe 30s per image on my M1 Max with 32gb

I guess it depends on how you do it. Depending on how I've set things up, I've found that more samples isn't necessarily better (usually just different). I suppose it's an optimization problem. I have found that I can pretty reliably look at a 1 sample image and kinda guess where the earliest iterations are going to go, and that might be the most appropriate workflow, actually, but beyond that it seems a couple of samples can drastically alter outputs, and likewise with prompt editing. Whereas with img2img there's a lot more control, I pick an input, I can force it to strictly abide to the image and the parameters I want, and as someone else said in- and outpainting are nice as well.

I guess I'm just manipulating probabilities in my favor?

You draw over the part of the image that is not ideal and get it to infill it

Because it depends on your online tool, Midjourney for example adds things to your prompts to make sure your images are better, OpenAI adds things to your prompts too.

SD is much more raw.

Disclaimer: I’m not well versed in this field, but my basic understanding is that Diffusion can and is “fuzzy”.

An image without “crisp” pixels that create lines is acceptable. A basketball with an extra/missing blue pixel is acceptable.

With words, they either haven’t invented Diffusion yet, or the nature of the problem is too hard for Diffusion.

With text you can’t have the model Diffuse to “teli ne abouf a fdog?” “The dod is graen and has stix leg5”. It’s just too obviously wrong.

Meanwhile, the “fuzziness” allows Diffusion models to be smaller, compared with models that need precision.

That's good to know as I just got a good deal on one and was wondering if the AMD GPU would be useful or if I needed to start planning for an eGPU with some NVidia silicon. Thanks!