Washington's Lottery forced to pull site after creating AI porn of lotto user

Well, it does highlight the fact that AI will make this sort of manipulation so much easier and faster.

Imagine a world where there the only vehicles are bicycles and people crash frequently. Now, the first car is introduced and someone crashes at 100MPH -- would you really say that the car is irrelevant to the situation because people have been crashing for decades?

Yes, you are right: the root problem is the way we view people's minds as a resource to be exploited through psychological manipulation, but that does not make irrelevant the very real new danger of AI.

State lotteries are a compromise: They inhibit underground lotteries. The goal of the state lotteries is to divert people away from the danger of the underground while keeping itself from being overly attractive.

It seems that society can't delineate between accepting a practice and promoting a practice.

Is it advertising lotteries that is immoral, or the lotteries themselves?

"It is very immoral to stab someone with a knife."

"It is very immoral to kill a billion people with a bomb."

And we should treat these situations with the same attention. Whatever happened to considering magnitude?

This reeks of someone doing absolutely no market research

I would even say it's only negatives.

But unfortunately, it will have to stay legal, otherwise people would still seek it through illegal means, possibly exposing themselves to much worse dangers.

It's the same problem with drugs (including Alcohol) and tobacco, they need to be legal in order to he regulated.

I'm reminded of the film They Live (1988). The protagonist, Nada, seeks a job at a construction site, as that is his trade. Due to the union, the foreman can only hire Nada as a day laborer, as Nada is not in the union. His way of life is gate-kept by powerful interests controlling the hiring process, and access to capital is constrained by these interests. By the time Nada obtains the sunglasses, he has already lost that job and is well down the path of becoming radicalized. Ironically perhaps, he first heard about the group producing the sunglasses when they hijacked the TV signals to essentially advertise their ideology.

Once Nada acquires the glasses, he could be said to be enlightened, but I also view it as a sort of cautionary tale, considering how the film ends for Nada. Along the way, he sees through all of the ads, and in fact much of the surrounding content itself rings hollow for him. His actions are reminiscent of Ted Kaczynski in this context. Why didn’t he instead seek to join the union? Were the glasses actually what they purported to be, or were they a kind of rose-colored glasses that were designed to manipulate the wearer just like the advertisements did, but while making them believe they were seeing things more truly? Nada thought he was acting under his own agency before putting on the glasses, and yet he so readily trusted that he was still after putting them on, while adopting an anti-capitalist, anti-authoritarian ideology as part of an underground fringe group.

Perhaps Nada was just tired of being part of the system, but by rage quitting as he did, we as the viewer are led to believe he goes out in a blaze of glory, but I view his exit to stage left thusly as “full of sound and fury, signifying nothing,” to quote the Bard of Avon. There is no exit from capitalism that way, if there is any alternative at all if capitalist realism holds true, and Nada’s final line is perhaps an admission that he was himself used all along, and he was simply tired of scraping by and being in scrapes. He had gone down the path he was on as long as he was willing to go, and there was no way back, as he was already too far gone to come back.

I’m reminded of the is/ought distinction, and I’d make a similar distinction between advertisements/content. Content is itself a similar kind of appeal to believe or think about concepts and ideals and ideas that come from outside ourselves. Due to the power of belief, capital seeks to gate-keep access to content, and advertising acts as a sort of escape hatch, blow-off valve, or side door through which access to those seeking content are able to be reached, accessible for a fee. It’s not exactly ideal, and not without its own hidden costs to the buyer or seller.

Lack of capital is the conflict that leads to the crisis in Nada’s life, but I wish that his story had a happy ending. I feel that Nada was perhaps less free after putting on the sunglasses than he was before, as strange as that may sound. He fell for a classic blunder, believing that he actually was immune to advertising, rather than being sold an even bigger lie: that he could (only?) effect change through violence, that his life was no longer worth living.

Someone has to make value judgments on the underlying data and algorithms, if only to build the system in the first place. If users can't (or don't care to) run these models locally, who else will decide other than the service operator?

‘Therefore AI vendors should release uncensored models and let everyone else figure it out’ is not a logical consequent from it.

‘Therefore don’t use current AI tech to build apps that photomanipulate user submitted content and echo the results back to them with your logo on it’ would be a more logical conclusion to draw.

In other categories of AI generated content though, such as speech and music, the open source alternatives are either lacking or completely absent. And with the company behind Stable Diffusion being on the verge of financial collapse, we may see Stable Diffusion fall far behind it's closed source counterparts.

> Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers

Open models == free == no customers. For open foundational models, yours sounds like a demand that someone else invests millions in training a foundation model of your liking, but you're not motivated enough to fine-tune your copy of the model for hundreds or thousand of dollars?

If you're not paying for the "censored" proprietary models, you are not a customer either. If you are a paying customer of "censored" models, and would like them to change, remember businesses have the right to choose who they want as customers - and they can decide they don't want the "uncensored" market segment. I'd love a $25,000 electric Ferrari too, but Ferrari is not about that.

What makes you their customer? Are you offering them money for access to the uncensored model you say you want?

Do you have some business that you’re going to be able to build that tests on access to an uncensored model, and their failure to offer that to you is preventing them from getting paid by you to provide the underlying technology for your venture?

Because on the other hand there is a state lottery over here who absolutely is willing to pay someone money for a safe, censored model. They are an actual paying customer with a real use case and cash to back it up.

Of course, it turns out the AI they ended up using didn’t (if this story holds up) actually meet their expectations with respect to safety and alignment - but let’s be clear: they wanted a safe, censored model.

I find it hard to believe that the market for uncensored models - LLMs that occasionally output 4chan style racist tirades, diffusion models that occasionally generate pornography - is actually worth addressing, compared to offering models which don’t do those things.

What if someone else's idea of a fun pic at the beach is a nude photo? Is it only censorship when the worldview being enforced is not aligned with yours? A truly uncensored model should accommodate everyone, but society will (and should) absolutely burn down any AI service that accidentally generates nudes from a beach photo of a family that has minors. "Intent" is a human attribute, AI is probabilistic.

This abundance of caution by the companies avoiding nudity is unfortunately read as promoting a worldview or pushing an agenda, while it's just risk mitigation.

This suggests there is a large, and perfectly reasonable demand for censored models that adhere to broadly ‘SFW’ cultural norms.

So why are people upset that commercially available models generally try to cater to that obvious need?

And, to be fair, its doubtful it would get very far; there's no evidence the aggrieved party wants to sue, she doesn't have the photo it generated, etc. I'm not standing on a soapbox arguing that the secret service should get involved or something. I'm just asserting that, it seems to me, we the people should care a bit more about this than we seem to be. Other people saying "its just a bug!" are incomprehensible to me; there's a spectrum between "your signin button is using the wrong hex code" and "your chemotherapy dispenser is outputting the wrong volume of medicine", and it is not obvious to me that this bug isn't closer to the right side of that spectrum than left.

An engineer in the room might even know that these generative systems are capable of hallucinogenic, and sometimes even explicit, output; but they don't have the political capital to stop decision makers from deploying these systems, or underinvesting in safeguards, etc. That's why we need pushback, ideally lawsuits but at least public outcry; so decision makers treat the decision to use generative AI with the care and consideration it deserves.

Yes.

> every piece of hardware between you and the end user

Nah. See how you said "where do you draw the line" and then proceed to just ignore drawing the line? I'm ok with drawing a line. There doesn't have to be logic or reason behind it. That's where I draw it; the repositories and websites and apps that I distribute.

> Unless they specifically trained their own model to do this - which they obviously didn't - this is just a bug.

Go try to make Midjourney generate a nude image of someone. Its extremely difficult. You'd have to be extremely intentional and, frankly, malicious. I've never seen it happen.

The way this article is written, it sounds like the user just uploaded a photo and it happened. So, what kind of prompting are they using which led to this? Also: They said they don't have a copy of the image, but that's extremely hard to believe just assuming how systems like this would be designed; the image has to be served from somewhere, are they seriously purging the images from e.g. their storage buckets that quickly? Possible, but hard to believe; of course, they'd say "oh we don't have a copy" if they investigated, found it doing this, and just wanted the story to die.

Or, we could do as you want, and just sweep this under the rug. Nothing to learn. No one to blame. Just a quirky little thing that happened! So quirky! A state gambling office manufacturing pornography of random people to sell lottery tickets! Quirky and fun!

You should have a responsibility to put the necessary safeguards on a potentially dangerous tool that you're using and to do your due-diligence if you're using that tool. It would have been impossible to accidentally produce porn of a non-consenting individual if they had not been using an AI image manipulation pipeline at all. If you do choose to use an AI image manipulation pipeline, you should take on the responsibility to make sure that it's not going to do illegal things you didn't tell it to do. If there's a risk of causing harm with it, just don't use it! You don't need to be producing cute images of people in Vegas or whatever, and if doing so necessarily implies a risk of accidentally creating porn of them instead, you could (and in fact, have a responsibility to) just choose to do neither.

How about the meaningful thing, that why is tax payer money being used to create a generative AI site at all? This should be a non-problem because this is in no way the role of government.

Plus most image generation pipelines have an optional secondary step that checks for nsfw images and blocks the output before it reaches the user. They may have neglected that step.

Something nsfw could still have gotten through, but it’s unlikely given the many safeguards that most pipelines have built in.