Stable Diffusion macOS native app | Hacker News Reader

Stable Diffusion macOS native app

github.com

383 pointsthrowme_12387 comments

Interesting UI, and great that's it's open-sourced!

M1 Mac owners should also check out the iPad version of Draw Things, which has a large number of features and is in very active development:

https://apps.apple.com/us/app/draw-things-ai-generation/id64...

Be aware that there is a bug in macOS 13.2b1 that makes Draw Things unusable on that version of macOS. Hopefully Apple will fix that soon.

In my opinion Draw Things is the best stable diffusion app currently available for macOS/iOS. It's free, it is available in the App Store, it manages downloading models and weights, it supports a wide selection of popular models as well as custom weights, and it supports inpainting, text2image, and image2image.

It has some oddities: it uses a sqlite database to store generated images rather than the more traditional files, and the UI is idiosyncratic. You have to follow the author's twitter feed to learn how to use the advanced features.

Appears to be https://twitter.com/drawthingsapp

Storing images in the database is not a bad thing https://www.sqlite.org/fasterthanfs.html

Your link suggests this is the case for small images, thumbnails, not large images such as that typically produced by SD

That article also points to the paper https://www.microsoft.com/en-us/research/publication/to-blob..., the images produced by stable diffusion are about 500-600Kb in size. At this point it could be a similar performance or could be a little better or a little worse. I am not saying that it is better to store files in SQLite, just saying that nothing wrong with it, if those files are below 1Mb.

> I am not saying that it is better to store files in SQLite, just saying that nothing wrong with it, if those files are below 1Mb.

There is one very obvious thing wrong with it, the fact that those resulting images aren't files that are easily accessible to other things.

Unless storing in SQLite actually offers a real advantage in some other way, it's a disadvantage when the output is something you'll want to use with other tools.

SQLite has also measured the performance impact for storing larger BLOBs:

https://www.sqlite.org/intern-v-extern-blob.html

The verdict: storing blobs in the database is totally fine. Stable diffusion is totally compute bound, the overhead of storing a 1mb file in a sqlite db is miniscule.

Will storing thumbnails in a postgres database offer performance improvements as well?

It’s likely a function of building for iOS first and then porting to MacOS.

be advised that it’s quite slow on an ipad if it doesn’t have an m1. it was faster on my iphone 14pro than on my ipad mini :|

The iPad Mini is quite old in terms of the SoC it shipped with.

Any recent iPad or iPhone has much better silicon , even if it’s not an M1.

mine has the A15 which was launched last year. and it still feels slow :(

Ah I forgot about the refresh for the 6th gen.

The difference in speed with your phone might be down to RAM? 6GB vs 4GB? I forget if the latest mini is LPDDR4 or 5 but it might be 5

Otherwise the A16 and the A15 aren’t that different, though the A16 does have a newer ANE ( https://en.wikipedia.org/wiki/Apple_A16?wprov=sfti1 ).

It is unfortunate that the UI is not even attempting to be platform native, though.

The box vs. the chocolate

I use iOS instead of Android because of the box. Apps that don’t look like the box art stick out like a sore thumb on iOS.

It crashed for me on my Macbook M1

Here's the MacOS list I know as of now: (none support fine tuning)

Use CoreML: Mochi Diffusion - https://github.com/godly-devotion/mochi-diffusion Amazing AI (closed source) - https://sindresorhus.com/amazing-ai Gauss - https://github.com/justjake/Gauss

Use CoreML but via Swift (not as optimized?) Draw Things (full featured, closed source)- https://draw.nnc.ai/ AI Photo (closed source) - https://apps.apple.com/gb/app/ai-photo/id6444911377?mt=12

Use vanilla GPU: Diffusion Bee (not updated for a month) https://github.com/divamgupta/diffusionbee-stable-diffusion-...

TI and hypernetworks (and possibly LoRA) will be doable.

Draw Things is only for iOS. You can try to run it on a Mac, but it crashes for me.

Also CoreML on a Mac seems to require macOS 13, which is reportedly buggy. I was going to update and changed my mind after reading people's experiences online. :(

Draw Things works for me on an M1 Pro on 13.0.1.

Yeah, the macOS "native" build was added for "Draw Things", it should support macOS 12.4 and all the way above now.

What's the difference between this and the Diffusion bee? The DB project has not been updated in a while.

Same, would love to see a comparison. I've been a mostly-happy user of DB for a while but always open to competition

This uses Apple's CoreML port and should be much faster if you have an ARM-powered Mac.

As far as I can tell, the advantages of Gauss are that it runs Apple's more efficient ml-stable-diffusion engine, and that the GUI is better. The disadvantage is that you (currently) have to build it yourself in a multi-step compiling process that involves "goo".

https://apps.apple.com/gb/app/ai-photo/id6444911377?mt=12

i’m using this one. pretty good so far. same use of efficient apple stable diff engine.

as well as this monster on mobile:

https://apps.apple.com/gb/app/draw-things-ai-generation/id64...

I appreciate the user-focus move of displaying the CFG as a slider, but its a fine-grained value. That makes the app unable to replicate a specific render shared online. I suppose this is a tradeoff :) I just wish there was a common UI element that combined a slider and value display...

Also, would be nice to share some binaries on GitHub!

Have you tried diffusionbee? It's not fully native but it does let you replicate prompts.

Mhm. DiffusionBee is Electron, so its UI is really just some CSS on HTML elements that looks vaguely Mac-ish.

What did you try to replicate? You will need the exact prompt (possibly with negative prompts), seed, CFG, sampler and model to do it.

Wonder how it compares to ‘Amazing AI;’ a native Mac app for Stable Diffusion I stumbled into earlier today.

Link: https://sindresorhus.com/amazing-ai

For anyone interested (and using an M1/M2 Mac), whilst Amazing AI offers limited twiddling (just the step count and seed) it is native Apple Silicon, comparatively light on resources, and on my base model 2021 Air (8GB RAM, 7 GPU) it produces a 2048x2048 pixel 100 iteration image in under 5 minutes.

I know that headline number is slow compared to machines with beefy specs, but remember that this is 100 iterations on the lowest power M1 you can get, with 16 times the pixel count of the online 512x512 ones. And even with 40 iterations you can generate some stunning images with the right prompts.

Draw Things has a massive amount of configuration, is equally fast (it's also native) and can do a variety of output sizes.

Both are free from the Mac App Store.

What about the swap use? I gather that high-resolution images with your limited RAM must produce a devastating amount of writes.

Haven't noticed anything, but TBH I also haven't looked as I've had no need to question things - the rest of the OS runs fine alongside, with dozens of browser tabs, plus both Amazing AI and Draw Things running (though only one actually generating at once). And with about 50 iterations I'm losing around 1% battery every couple of images so it doesn't seem particularly taxing, whilst running cold.

Edit: It isn't obvious with Amazing AI but it may in fact be doing what Draw Things makes explicit in the options, which is generate at 512x512 and use your choice of upscaler to get to 2048x2048.

Does this have any sort of ability to feed it training data? I know I can do that with some SD frontends on Windows - ex: give it 5 pictures of my face, and have it generate a 3D avatar of me.

I doubt it, but even if it did you probably don't want to be re-training the set on your mac. There is plenty of instructions on how to train your face into the dataset using AWS. IIRC it costs around $20 of compute time.

No it costs far less.

you can use runpod.io it costs around $1 to retrain, ofc depends on the number of images and steps

Run it on google colab with TheLastBen fast-dreambooth for free in an hour.

What tool do you use to train on Windows locally?

Dreambooth is the most popular. Need a 12gb card for windows though.

Thanks, will check if my 8G rtx3070 can run it :)

If you are on linux, you can do it with 8gb I hear. I have the 3060 8gb and had issues on win10, so I can train on google colab using the TheLastBen fast-dreambooth for free.

How is this better than stable diffusion webui from automatic11111? I see severe lack of features in this one.

Shouldn't the horses wear spacesuits?

They are natives and breath the local atmosphere.

Cannot find the info of what mac, especially for this wave of ai. One did mention in the comment of air but that is so far I can get hold of.

I only have a m1 mini in lieu of my developer unit and it is out of the country even. I tried to get back in but at least one app required 32gb to build, which stopped my 2023 jan plan of getting a MacBook Air 24gb. A Mac mini studio seems the minimum.

Anyone has a low end mac which feel the pain of being in the low end. Or must I get a studio mini.

Is this available in the form of a library I can use?

No, it's not even documented how to install it. Not sure if it's trivial on macOS 13, but it definitely is not on Monterey.

The build instructions are right there in the README

Not really. It links to another repo which must be installed, and that doesn't have complete instructions.

Would this work M1 iPads?

I love the interface.

Too bad there's no complete instructions for installing this, and no issue tracker to ask in.

This might be nice, but I really don't get projects that have no releases and not provide build instructions.

what do you mean? you can open an issue on github and there are build instructions in the readme

Looks like they opened the Issues tracker after I posted, good. Someone else already inquired about the installation procedure.

I think this got shared on hackernews too early for this to be ready, the author says somewhere else that packaging and publishing to app store is in the pipeline if I recall right.

Does SwiftUI really count as native on the Mac? My experience is that Apple has not invested enough for that to be true. Apple is Apple, so when they say jump, developers jump. But if any other company tried pulling what they’re pulling with SwiftUI, they’d be getting a swift kick in the behind by their dev community. It feels like their UWP moment; nice idea, bad execution.

This isn’t to criticize the project; I like and want to see more native Mac software, I just feel very disappointed with Apple’s leadership in this area. A native Mac app should almost automatically be without a doubt better than the Electron version, but at least with SwiftUI that isn’t happening.

Regardless, inconsistencies between AppKit on Mac and SwiftUI on Mac are enough to make it not native, IMO.

Kudos to the author for taking advantage of CoreML, one of the areas where native software shines on Apple platforms.

That is the point of frameworks. Apple can keep improving SwiftUI and everyone using it will benefit with minimal effort, if not automatically.

It does not have to be perfect, there just has to be constant progress.

That statement applies equally to any web-based framework, any native framework, Java, appkit, UWP, Silverlight…

Apple could have invested in SwiftUI on Mac to the same extent they did on iOS, and things would look very different.

SwiftUI as currently implemented on Mac breaks enough platform conventions and is, relative to existing native UI, slow and buggy enough to IMO not more native than any other cross platform UI solution.

Which UI conventions are broken? SwiftUI mostly uses the standard AppKit widgets, then it's up to the developer to use it or to customise everything and make a monster out of it. The main "issue" is that it's much easier to customise things in SwiftUI.

> That statement applies equally to any web-based framework, any native framework, Java, appkit, UWP, Silverlight…

Except for the part where Apple doesn't hype and then quickly abandon developer APIs the way some other companies do. AppKit, for instance, has been continually iterated on for three decades now.

Since Apple has been very clear that SwiftUI is "where the puck is going", I doubt it will be abandoned as quickly as Google or Microsoft's "future of platform development strategy" of the month.

> Does SwiftUI really count as native on the Mac? My experience is that Apple has not invested enough for that to be true

https://blog.timac.org/2022/1005-state-of-swift-and-swiftui-...

iOS 16.0 included 226 SwiftUI apps. Several highly promoted features (such as complications on the Lock Screen and widgets) are now only possible using SwiftUI.

Some new SwiftUI only frameworks (like Swift Charts) developed directly from internal Apple teams creating new SwiftUI apps, and factoring out useful components.

> Regardless, inconsistencies between AppKit on Mac and SwiftUI on Mac are enough to make it not native, IMO.

SwiftUI is not a wrapper around AppKit controls, although that is/was how some SwiftUI views are authored. Likewise, some recently added AppKit controls are actually wrapped SwiftUI views.

There is an ongoing effort to unify macOS and iPadOS which became much more noticeable at the UX level in macOS 11. It is unfortunately difficult to determine which inconsistencies are deficiencies in SwiftUI and which are the "legacy" behavior of AppKit.

Since SwiftUI is a system framework, we only see its evolution in spurts on an annual schedule.

UWP stands for Universal Windows Platform. https://en.wikipedia.org/wiki/Universal_Windows_Platform

UWP failed and continues to fail because Microsoft themselves are not invested into it.

Most of the apps that come out of Microsoft use something else. Eg: Vscode, Teams etc, Office etc.

It’s not the case with Apple at least from what I can see publicly. Not an Apple fanboy by any stretch.

But I’ve been burned by Microsoft when they introduced Silverlight as the platform for developing Windows Phone 7 apps. Only to deprecate it in Windows Phone 8. A lot of developers were not happy at that time.

UWP version of VSCode would have failed in the market. Office, too.

That seems like a failure of UWP, not the applications or the market..

You are right.

Reply on news.ycombinator.com