Hacker News Reader

Comment by Scene_Cast2 | Hacker News Reader

Something I keep seeing is that modern ML makes for some really cool and impressive tech demos in the creative field, but is not productionizable due to a lack of creative control.

Namely, anything generating music / video / images - tweaking the output is not workable.

Some notable exceptions are when you need stock art for a blog post (no need for creative control), Adobe's recolorization tool (lots of control built in), and a couple more things here and there.

I don't know how it is for 3D assets or rigged model animation (as per the article), never worked with them. I'd be curious to hear about successful applications, maybe there's a pattern.

View on HN

Something I realized about AI is that an AI that generates "art" be it text, image, animation, video, photography, etc., is cool. The product it generates, however, is not.

It's very cool that we have a technology that can generate video, but what's cool is the tech, not the video. It doesn't matter if it's a man eating spaghetti or a woman walking in front of dozens of reflections. The tech is cool, the video is not. It could be ANY video and just the fact AI can generate is cool. But nobody likes a video that is generated by AI.

A very cool technology to produce products that nobody wants.

That's an over simplification I think. If you're only generating a video because 'I can oooh AI' - then of course no one wants it. If you treat the tools as what they are, Tools - then people may want it.

No one really cares about a tech demo, but if generative tools help you make a cool music video to an awesome song? People will want it.

Well, as long as they aren't put off by a regressive stigma against new tool at least.

Are there any valid reasons people might not like this or is it only "regressive stigma?"

Humans find lots of value in human effort towards culturally important things.

See: a grandmother’s food vs. the industrial equivalent

well we had an entire article explaining parts of that. You can skimp out on some areas and fool a human, but human proportions and environmental weight is hard. Then you get to motion and it's extremely hard to fool the human eye.

One end of art is spending millions of man hours to polish this effect to fool the eye. the other side simplifies the environment and focuses more on making this new environment cohesive, which relaxes our expectations. Take your favorite 90's/early 00's 3d game and compare it to Mass Effect: Andromeda to get a feel of this.

AI is promising to do the former with the costs of the latter. And so far it's maybe halfway to Andromeda in its infancy of videos.

If you used AI to make something awesome, even if I liked it, I'd feel scammed if it wasn't clearly labelled as AI, and if it was clearly labelled as AI I wouldn't even look at it.

> if it was clearly labelled as AI I wouldn't even look at it.

If you dislike it without even seeing it, that would indicate the problem isn't with the video...

Yes, the problem is with AI. I'm tired of trying to find X and finding "AI X" instead. I google "pixel art" I get "AI pixel art." I google clipart I get "AI clipart." I go to /r/logodesign to see some cool logo designs, it's 50% people who used ChatGPT asking if it looks good enough.

The only good AI is AI out of my sight.

> I'd feel scammed if it wasn't clearly labelled as AI

TBF - have you looked at a digital photo made in the last decade? Likely had significant 'AI' processing applied to it. That's why I call it a regressive pattern to dislike anything with a new label attached - it minimizes at best and often flat out ignores the very real work very real artists put in to leverage the new tools.

Face it. People are okay with super resolution efforts, including most deep learning-based methods. But not "AI". You can run video through i2i as a cleanup tool and upload it on the Internet, some tried and quit. YouTubers and TikTokers aren't doing it and they're all for attention.

Output of current image generators are trash. It's unsalvageable. That's the problem, not "regressive pattern".

You still have to take the photo. That's a billion times more effort than typing a prompt in ChatGPT.

honestly, that's the same argument people made against photographs when the technology became available. Same argument made against the printing press.

New tools aren't inherently inferior, they open up new opportunities.

I've never seen a photograph pretending to be an illustration, or vice-versa. It's only AI that pretends to be a genre it isn't.

If anything photography mostly just replaced painted portraits, hence why high art went super abstract in response.

> A very cool technology to produce products that nobody wants.

creative power without control is like a rocket with no navigation—sure, you'll launch, but who knows where you'll crash!

Yes, it turns out there's more to creating good art than simulating the mechanics and technique of good artists. The human factor actually matters, and that factor can't be extrapolated from the data in the model itself. In essence it's a lossy compression problem.

It is technically interesting, and a lot of what it creates does have its own aesthetic appeal just because of how uncanny it can get, particularly in a photorealistic format. It's like looking at the product of an alien mind, or an alternate reality. But as an expression of actual human creative potential and directed intent I think it will always fall short of the tools we already have. They require skilled human beings who require paychecks and sustenance and sleep and toilets, and sometimes form unions, and unfortunately that's the problem AI is being deployed to solve in the hope that "extruded AI art product" is good enough to make a profit from.

The problem in your example is that you wouldn’t think a picture of a man eating spaghetti taken by a real person would be cool.

You may feel different if it’s, say, art assets in your new favorite video game, frames of a show, or supplementary art assets in some sort of media.

> or a woman walking in front of dozens of reflections

A lot of people will not notice the missing reflections and because of this our gatekeepers to quality will disappear.

While I am in the same camp as you, there is one exception: Music. Especially music with lyrics (like suno.com) - Although I know that it's not created by humans, the music created by Suno is still very listenable and it evokes feelings just like any other piece of music does. Especially if I am on a playlist and doing something else and the songs just progress into the unknown. Even when I am in a more conscious state - i.e. creating my own songs in Suno, the end result is so good that I can listen to it over and over again. Especially those ones that I create for special events (like mocking a friend's passing phase of communism and reverting back to capitalism).

In my opinion, Suno is good for making really funny songs, but not for making really moving songs. Examples of songs that make me chuckle that I've had it do:

A Bluegrass song about how much fun it is to punch holes in drywall like a karate master.

A post-punk/hardcore song about the taste of the mud and rocks at the bottom of a mountain stream in the newly formed mountains of Oklahoma.

A hair band power ballad about white dad sneakers.

But for "serious" songs, the end result sounds like generic muzak you might hear in the background at Wal-Mart.

appreciate your position but mine is that everything out of suno sounds like copycat dog water.

Makes sense that GP appreciates the taste of dog water when they’re mocking their friends for having had values (friends whom likely gave up their values to stop being mocked)

My generation do not give up on their values because they are being mocked - they mock back even harder until somebody ends up dying from laughter.

Reminds me of Cohen as covered by the Doug Anthony All Stars

    I got my shit together meeting Christ and reading Marx
    It failed my little fire but it spread a dying spark

https://youtu.be/elr0JmB7Ac8?t=42

Probably accurate for videos and music. Videos because there’s going to be just too many things to correct to make it time efficient. Music because music just needs to be excellent or it’s trash. That is for high quality art of course. You can ship filler garbage for lots of things.

2D art has a lot of strong tooling though. If you’re actually trying to use AI art tooling, you won’t be just dropping a prompt and hoping for the best. You will be using a workflow graph and carefully iterating on the same image with controlled seeds and then specific areas for inpainting.

We are at an awkward inflection point where we have great tooling for the last generation of models like SDXL, but haven’t really made them ready for the current gen of models (Flux) which are substantially better. But it’s basically an inevitability on the order of months.

Even with the relatively strong tooling for 2D art it's still very difficult to push the generated image in novel directions though, hence the heavy reliance on LoRAs trained on prior examples. There doesn't seem to be an answer to "how would you create [artists] style with AI" that doesn't require [artist] to already exist so you can throw their life's work into a blender and make a model that copies it.

I've found this to be observable in practice - I follow hundreds of artists who I could reliably name by seeing a new example of their work, even if they're only amateurs, but I find that AI art just blurs together into a samey mush with nothing to distinguish the person at the wheel from anyone else using the same models. The tool speaks much louder than the person supposedly directing it, which isn't the case with say Photoshop, Clip Studio or Blender.

Shrug. That’s a very different goal. Yes, if you want to leverage a different style your best bet is to train a Lora off a dozen images in that style.

Art made by unskilled randos is always going to blur together. But the question I feel we’re discussing here is whether a dedicated artist can use them for production grade content. And the answer is yes.

https://www.kiplingsociety.co.uk/poem/poems_conundrum.htm

> but is not productionizable due to a lack of creative control.

It's just a matter of time until some big IP holder makes "productionizable" generative art, no? "Tweaking the output" is just an opinion, and people already ship tons of AAA art with flaws that lacked budget to tweak. How is this going to be any different?

No, it's not "just a matter of time." It's an open question whether it's even possible with anything resembling current techniques.

I don't think it is a question at all. It is not just possible, it's implemented in reality. Compositing is a thing in imagen space, and source adjustments in this scheme are trivial. I'm talking about controlnets, style transfer adapters, straight up neural rendering of simplified 3D scenes, training on custom references, and a ton of other methods to establish control. Temporal stability is also a solved issue.

What it really lacks is domain knowledge. Current imagen is done by ML nerds, not artists, and they are simply unaware of what needs to be done to make it useful in the industry, and what to optimize for. I expected big animation studios to pick up the tech like they did with 3D CGI in the 90s, but they seem to be pretty stagnant nowadays, even besides the animosity and the weird culture war surrounding this space.

In other words, it's not productized because nobody productized it, not because it's impossible.

The generated artwork will initially displace clipart/stock footage and then illustrators and graphic designers.

The last 2 can have tremendous talent but the society at large isn’t that sensitive to the higher quality output.

Reply on news.ycombinator.com