Hacker News Reader

Comment by bloaf | Hacker News Reader

They know they are going to be the next target in the war on general purpose computing. They're trying to stave it off for as long as possible by signalling to the authorities that they are the good guys.

A confrontation is inevitable, though. Right now it costs moderate sums of money to do this level of training. Not always will this be so. If I were an AI-centric organization, I would be racing to position myself as a trustworthy actor in my particular corner of the AI space so that when legislators start asking questions about the explosion of bad actors, I can engage in a little bit of regulatory capture, and have the legislators legislate whatever regulations I've already implemented, to the disadvantage of my competitors.

For people who say "people can make whatever images they like in photoshop," I will remind you of this: https://i.imgur.com/5DJrd.jpg

View on HN

appeasement never works. those who wage that war should be directly confronted.

and they will lose it, just like they've lost the war on encryption.

I doubt they care if people make porn with diffusion models. They just don’t want to be the ones providing the model to do it.

This is business, not ethics, though. They just don't want the negative attention, that's it. And because this is a business, time matters, same as DRMs. Almost all DRMs get defeated, yet the work, because they hinder the crackers, even if for some time. Same here. While Stability is not under attack yet, they can establish themselves as the household name for AI in a safer context.

Banknote printing is primarily protected against on the hardware level of printers, no? With the nigh-invisible unique watermark left by every printer, there’s virtually no way you’d get away with it. My guess is that the Photoshop filter exists mostly as a barrier against the crime of convenience.

My point is that there is precedent for governments requiring companies to implement restrictions on what images can be handled by their software.

As I explained: This kind of mandated restriction is looming over AI. Companies are trying to get out in front of these restrictions so they can implement them on their own terms.

>My point is that there is precedent for governments requiring companies to implement restrictions on what images can be handled by their software

But images of boobs are still legal. So this NSFW filter seems to be much more above then the law asks. Is the issue is that even if you do not train with CP you might get the model so output something that some random person will get offended and label it as CP? I assume that other companies can focus on NSFW and have their lawyers figure this out, IMo would be cool that someone sues the governments and make them reveal facts about their concern that CP of fake or cartoon people is dangerous" , I think they could focus on saving real children then cartoon ones.

It’s possible that the end game is hardware in GPUs, to detect whatever they want to prevent, before it’s displayed.

You kill way too many birds with such a stone. Of course you could never do any kind of photorealistic game in real time if you had to pre-screen everything with an actually effective censor.

Indeed, what they're already doing is already hobbling the models.

Emad is right that we learn new things from the creativity unleashed by accessible models that can be run (and even fine tuned) or consumer hardware.

But judging from what people post, one thing we learn is that it seems models fine tuned on porn (such as the notorious f222 and its derivative Hassan's blend) can be quite a bit better at non-porn generation of diverse, photorealistic faces and hands too.

> Of course you could never do any kind of photorealistic game in real time if you had to pre-screen everything with an actually effective censor.

I'm not sure I understand this. A possible implementation could be a neural net that blanked the screen with a frown face any time it detected something it thinks was "bad". What purpose/need would pre-screening serve?

That you describe IS pre-screening. And it's not workable, because it would take a ton of dedicated resources to make it work in real time, and even then it would be disastrous for latency, unworkable for most games and even desktop applications.

> That you describe IS pre-screening.

I think this is making the assumption that all frames are blocked.

> then it would be disastrous for latency

We're talking about the future here. I'm not sure it makes sense to use current tech to say it's not going to happen, or come up with latency numbers. But, "real time" inference is definitely a possibility, and is in active use for video moderation (Youtube, etc) and object detection (Tesla, etc). Nobody will notice a system running at 2000fps.

You can typically work around that by modding the printer firmware, if needed. It's not baked into the hardware.

Some AI startups backed by the biggest players are already hitting legal/regulatory issues.

Very niche example here, that can be easily circumvented

Also seems problematic to approach this from a purely capitalistic and consumerist angle. There is a lot of opportunity here besides just launching the next AI unicorn.

I am not clicking that link because no one should take the risk of you proving your point of what horrors could pop out of one of these models.

I will say that while the government backlash is inevitable just like it was with encryption, these image generation models are so easy to train on consumer hardware that the cat is hopelessly out of the bag. It might as well be thoughtcrime.

Link doesn't show any model output - it's an screenshot of photoshop refusing to edit a banknote.

Or it's an output of "blank adobe photoshop with dialog refusing to edit bank note, full screen, windows vista, 4k, artstation, greg rutkowski, dramatic lighting".

I agree, that was the riskiest click I've made in a while.

Reply on news.ycombinator.com