Hacker News Reader

Comment by vunderba | Hacker News Reader

One of the things they seem to be emphasizing here is the UX around being able to place specific elements where you want them in an image. If the positions of the components in the overall composition are very important, this seems to make that a lot easier and kind of reminds me of InvokeAI.

Ideogram V4, an open-weight model released back in June can also do this [1], but you have to use a relatively cumbersome JSON structure to describe all the different bounding boxes. So it’s definitely a bit of a hassle.

I'll probably be waiting until it goes open-weight (hopefully soon) like they did with Flux.2 / Klein.

[1] - https://docs.ideogram.ai/using-ideogram/getting-started/prom...

View on HN

You just get an LLM to do the bounding box stuff or use the ComfyUI node that provides a GUI for bounding box generation

I always hated Comfy's node based UI, but agents make it tolerable. Now I just have them set up a workflow and I go in and tweak it manually if the results aren't where I want them. I even have agents cherry doing multiple runs and cherry picking the best outputs, models have gotten good enough that it's a real time saver, assuming you have references they can and a rubric to check against.

How do you get agents to set up a workflow? You just get them to modify the JSON directly and then import it, or do you have a tighter integration (e.g. in the UI)?

The agents can interact with Comfy via API pretty well, which afaik ends up being directly with JSON.

> I always hated Comfy's node based UI

It's one of the most uniquely hostile user experiences I've ever had the (dis)pleasure of working with

Awhile back I built a project which reads comfy's API and builds a typescript sdk from it. Much nicer to work with and still benefits from node caching and the comfy ecosystem. Perhaps I should clean it up and open source it, though I haven't looked around and there may already be other projects doing this out there.

Using AI for the ComfyUI API or workflows worked pretty well for me even early this year.

However, today I don't see a reason to use ComfyUI at all.

For Qwen Image 2.1, I had Opus 5.5 create a backend outside of ComfyUI and it was able to make generation take 20% less time with some optimizations.

The optimizations it implemented were caching the text computation in Qwen Image 2.1 rather than including it in every step, fusing projections into a larger matrix multiplication, and decoding the VAE in horizontal bands or something like that.

If there was anything interesting in ComfyUI nodes, I imagine I could just have the AI adopt the relevant code instead of dealing with ComfyUI or custom nodes.

I remember precisely how ComfyUI started, and the motivation was something along the lines of "look at the cool guys working in Houdini and Resolve, they must know something" (I wish I was kidding). Of course nobody thought that DAGs are just poorly suited for the task, even when plugin authors started trying to slap loops on top of them shortly after that.

It's definitely not for me.

From where I'm sitting, it's just turning python functions into boxes and instead of write the function yourself, you drag from the output of one box to the input of another. For 2 or 3 boxes, this is cool, but I opened up a professional workflow and was taken into a view with 100s of boxes and wires all over the place. Uhh, ok?

For myself, I'd rather just create my own python environment, write some quick pytorch or mlx calls, wire up some cli to it and share that in a GitHub.

I recently found a project[0] that takes a ComfyUI workflow and turns it into a simple UI. I have no affiliation with it and haven't tested it myself, but it looks like it might be handy once you have a finished workflow you want to use.

[0] https://github.com/saintbrodie/Orange

ComfyUI has a feature called “App” which is basically this. You pick the inputs and outputs in a workflow and then they become a kind of a subgraph.

Disclaimer: never used it for actual generation so I don’t know if it’s doing anything special other than being a subgraph with a different name.

As someone with general experience with some generative AI tools I started exploring Comfy last week and it was the easiest to pick up by far. This is not meant in a troll way, it just reflects how you do it programmatically the most, which I've seen the most before. The other UIs are always opinionated layouts on top of the actual logic where you constantly have to look up/sift through menus for what you want to do. In Comfy, you just use the searchbar and get the right node.

That being said: defining composition made me immediately think that someone probably made a gui like this with easy to move bounding boxes, and I'm happy they did.

Yeah, given how much better Ideogram v4 outputs are when you use the proper structured JSON (background, elements, etc.) I think most users probably have stuck some kind of Qwen/Gemma-based LLM between their raw prompt and the CLIP encoder.

Ideogram 4.5 released 2 days ago with more features along these lines

https://www.youtube.com/watch?v=2mecWZgbaEg

Not open weights yet.

true but they're announced https://x.com/ideogram_ai/status/2105327223431737780

Adobe has had something like this for a while with Generative Fill. Drag a box, specify contents.

Isn't that just inpainting?

Bingo. With slightly more control and metadata.

Reply on news.ycombinator.com