Show HN: Run Stable Diffusion Directly on iPhone | Hacker News Reader

Show HN: Run Stable Diffusion Directly on iPhone

apps.apple.com

162 pointsmofle42 comments

Apple released CoreML Stable Diffusion library a little over a year ago [1]. Hugging Face released their version of the example app for the CoreML Stable Diffusion library [2].

The app should be able to run on iPhone 14 Pro, I believe the requirements is about 6-8Gb of RAM. And I was not able to run it on iPhone 13 Mini, because it has only 4Gb of RAM.

- [1] https://github.com/apple/ml-stable-diffusion

- [2] https://github.com/huggingface/swift-coreml-diffusers

> The app should be able to run on iPhone 14 Pro

Can’t install it, says device is not supported.

Yeah, the Compatibility section states 15 Pro and 15 Pro Max only.

How does this compare to Draw Things?

Comparing the App Store listings, it looks like this app has a much simpler interface and far fewer features than Draw Things. Some users might prefer the simplicity of this app.

(Draw Things is by far the most advanced app that supports on-device Stable Diffusion on iOS devices and Apple Silicon Macs. It had a non-standard UI, but otherwise is really good.)

https://apps.apple.com/us/app/draw-things-ai-generation/id64...

Thanks for the link, it seems fascinating!

Yea I had no idea this existed! Totally going to play with this now. Surprised this is free tbh

Hell, i'd pay for something like this on Desktop (tho i'm on Linux (NixOS)).

I clicked the link on my Macbook and it apparently also runs on Apple Silicon Macs. I’m guessing it relies on the Apple Silicon Neural Engine.

I enjoy its UI personally. It’s sad to say it, but non standard on a mac means it’s actually kind of usable for a ‘pro’ app.

I have to disagree w.r.t. UI and non-standard being preferable for a pro app.

Having used Drawing Things regularly for the last few weeks, I still get confused by certain interactions and UI elements, leading to mistakes, and 'lost productivity'. It would greatly benefit from a UX pass, as more standard UX improves expectations of what will happen upon performing an action.

Don't get me wrong: I appreciate that it was released –for free– and that its capabilities are what they are. I'm merely arguing that more cohesive UX and pro functionality are not mutually exclusive.

As an example of a 'pro' app, there's Pixelmator Pro, which is a very Mac-assed app. I was able to pick it and start using it immediately without tutorials as its typical UX is intuitive (to me, as a macOS user), even when it came to more complicated operations.

Some more examples that I can think of off the top of my head: Proxyman, TablePlus, Kaleidoscope, Tower. The only exception to my observation, based on tools in my daily arsenal: VSCode. Non-standard UX, yet still intuitive.

Everything else that's non-standard feels like I'm battling with the UI daily, even after years of use: Android Studio, Slack, and most of the complicated Electron apps.

It’s limited to iPhone 15

To drawing things (yourself) it compares by not stealing. Yes, I know that’s not what you meant but with that name, I couldn’t not write it

Testing on my iPhone 15 pro - I couldn't find it in the app store with a search but I looked up the developer and was able to download it there. Working so far, first image took a while (a few minutes, as the app warned me), but subsequent images were a bit faster (~1.5 minutes). Phone does get pretty warm though.

iPhone 15 Pro with 30 steps at 512x512 resolution (SD v1.5) should take around 35 seconds on iPhone 15 Pro with Draw Things. 1.5 mins are too slow. (I am the author of DT).

It could indeed be faster. The app does not currently use the neural engine (ANE) because it has a tendency to crash the app, so it uses only CPU and GPU. The app also does upscaling, which adds ~10 seconds.

I am going to put model related code we use in a public repo soon (it is very similar to https://github.com/liuliu/swift-diffusion but in NHWC format). ANE will be around 25s if it runs. DT's default only uses GPUs and 35s is on GPU (yes, like you said, upscaling would take extra 10s).

Great work, Sindre! Didn't realize you were in the AI and app space. You've got quite a prolific OSS portfolio spanning decades at this point.

Runs pretty slow and hot on my 15 Pro. Seems like a pretty old model as well.

That's your 15 Pro at 100% compute, anything that makes it go to 100% compute for the same amount of time will make it as hot.

The storage capacity of Stable Diffusion on my computer has exceeded 100 GB, so I don't consider it a productive option.

It really would not work with 14 plus?

Nice, good job

Stable Diffusion runs on iPhone without any issues on DiffusionHub.io

Not locally, though.

The author of this app has also made a bunch of nice Mac apps, mostly menubar apps. I use several in my daily work.

And they created about 50% of all useful npm packages too

https://www.npmjs.com/~sindresorhus

Requires iPhone 15 pro

"Developed exclusively for Apple silicon (M1/M2) - The app is NOT compatible with devices running on Intel chips."

I guess the app store redirected me to the desktop store?? Does not indicate an iphone 15 being required at all

The description is written by the software developer. The "Information" section near the end lists requirements for supported devices. iPhone and iPad say "Requires iOS 17.1 or later and a device with the A17 Pro chip or later", and the first iPhones with A17 chips or later are the iPhone 15 Pro and iPhone 15 Pro Max.

I like how when I first opened the link, App Store said I need iOS 17.1 or greater.

So I updated to 17.2.

And only then does App Store let me know that it’s not compatible with this device anyway.

Thanks, Apple!

(I have an iPhone 14 Pro.)

See: https://sindresorhus.com/amazing-ai#why-does-the-app-require...

I tried using it to generate some sprites for a game I've been thinking about. Kept telling me it couldn't show me the image because it wasn't safe (I asked for robot pirates). Couldn't see a way to turn off the nsfw protection. Uninstalled it :(

Slightly OT but is there a decent setup for sprite generation out there? Non phone, I mean. It certainly seems like there's been some work in maintaining consistent style and even subjects across runs, does that work with 'character A walking frame 1, character A walking frame 2' etc anywhere yet?

This is a surprising thing when first working with these models (especially ones implementing NSFW filters, which are noisy). If you go check civitai.com you'll see that there's a lot of... well... porn. There's many LORAs to download but a very useful one ends up being the clothing slider. While I think the intention is to remove clothing, it is helpful in adding clothing. Unfortunately this doesn't look to support LORAs which are essential to getting many of those high quality images you see floating around.

My guess here is that the model is just trained on too many sexy pirates (it also has a propensity for producing asian women, which this model seems to do too). It does look like they support negative prompts but it requires you using "##" to separate positive and negative. Interesting design choice. You'll find these negative prompts helpful: disfigured, low quality, child, sexy, nude, extra limbs, ugly hands; and anything in the same vein. What works best is dependent on the base model and there is variance between different positive prompts. You may also have more success with something like automatic1111 which as long as you feel comfortable doing a git clone (which you're on HN, so I assume you are) then it'll be a better interface, but I don't know if there's a apple arm model or if baremetal has improved since last I checked.

Reply on news.ycombinator.com