Hacker News Reader

Comment by BoppreH | Hacker News Reader

It's interesting that these models can generate seemingly anything, but the prompt is taken only as a vague suggestion.

From the first 15 examples shown to me, only one contained all elements of the prompt, and it was one of the simplest ("an astronaut riding a horse", versus e.g. "a glass ball falling in water" where it's clear it was a water droplet falling and not a glass ball).

We're seeing leaps in random capabilities (motion! 3D! inpainting! voice editing!), so I wonder if complete prompt accuracy is 3 months or 3 years away. But I wouldn't bet on any longer than that.

View on HN

In my experience with stable diffusion tools, there is some parameter that specifies how closely you would like it to follow the prompt, which is balanced with giving the AI more freedom to be creative and make the output look better.

Yes, that might be the case. Though the prompts don't seem to try showcasing model creativity, so I'd be surprised if Google picked a temperature so high that it significantly deviated from the prompt so often.

The way I see it, input being confined to a "text description" is the next immediate problem that needs to be solved. I don't think we can rely on textual inputs for much longer as the human language is too imprecise and/or verbose. It's hard to imagine what exactly the optimal interface would be, but I'm thinking we'll need ways to dictate attributes for each entity being represented, the backdrop, and the view composition all as separate individual components. Ideally all these components can also be reusable and provide reproducibility guarantees without having to share a global "seed" as well.

A "mood board" of inspiration images could be an interesting input method. With deep vision models, we can already separate different "levels" of concepts: a high level subject ("person riding a horse"), textures (the horse hair), medium (painting vs 3d rendering vs photograph), etc. It'd be interesting to have a "smart mood board" that goes from text prompt, to visualizing that hierarchy with different options. Then the user could interactively increase or decrease different parameters, ultimately iterating alongside the computer to realize their creative vision.

That one person already used the other half of the auto-encoder to go from an image of "ugly Sonic" from the Sonic The Hedgehog movie, to a special "{sonic}" token, and then they could put that into prompts.

It is not far off.

Like an AI-assisted photoshop, but that's not restricted by language, only interactivity. Down the line you'll need a direct mind meld because some ideas don't have words to describe them, but that's not the "next immediate" problem.

People quickly created photoshop plugins to integrate stable diffusion when SD came out

https://twitter.com/wbuchw/status/1563162131024920576

Reply on news.ycombinator.com