Comment by liuliu | Hacker News Reader

Comment by liuliu | Hacker News Reader

Parent Full thread

What you get with Draw Things: 1. Download the app from Mac AppStore; 2. Download the model; 3. Tap "Try recommended settings", have the guarantee that for whatever model it supports, it is the fastest in Mac ecosystem, no need to fiddle.

What you get with local LLM options: 1. Download the inference engine app from the web; 2. Download the model; 3. Configure MTP / DFlash / DSpark whatever; 4. Configure your Pi / OpenCode harness to point to this local LLM inference engine. 5. Configure tools for these harness to be effective. 6. Switching between Ollama, LM Studio, llama.cpp (DwarfStar4), oMLX, MTPLX, to see which one is fastest for your workload. 7. Again switching between different quants of the same model to see which one is less dumb.

To be honest, llama.cpp probably the closest to deliver on "just use it, don't worry about speed" if your focus is about a pure LLM inference engine.

I’ve casually used Draw Things a couple of times (great tool!), and I did not find it beginner friendly. I remember feeling a lot of doubt, confusion, and frustration.

Am I getting these results because I picked the wrong model? Or I need to improve my prompt? Or the tool just can’t do what I’m trying to do? How current are these model recommendations? Have they been superseded by something newer?

As a true beginner to AI at the time, even the sizes and bits were meaningless to me. And I don’t remember having any context as to what I should be attempting to run on my mac.

So I think you need to include steps 4, 5, and 6 of swapping between different models, quants, and prompts. And step 7 is probably wading through the complicated UI, full of jargon that most people don’t know.

Don’t get me wrong, I recognize that it’s a powerful tool, and the steep learning curve exists because it exposes quite a few power-user features. But for someone graduating from commercial AI image generators that take a text prompt and maybe a choice of couple models, it’s not easy.

Yes, I heard you! I think one of the issue Draw Things inherited is the baggage of supporting too much models. Once you settled on a model, then it is just "try recommended settings", and prompt.

The model part is unfortunate, but luckily converging now.

On the LLM side, hopefully it is not an issue too as long as getting aggressive at pruning models.

Or you can just ask Sol to do all that for you, and do split testing, to find best performance.

I would like to see "guarantee" for that app. You get full refound and it is free, right?

Ollama was/is this but they pulled a heel turn...

Reply on news.ycombinator.com