Hacker News Reader

Comment by GaggiX | Hacker News Reader

I red that the AI runs locally, but then the image is uploaded to Microsoft servers to check that it doesn't violate any policies.

View on HN

Are those images checked by their Kenyan team?

https://time.com/6247678/openai-chatgpt-kenya-workers/

Wow that's so... lame...

It's Microsoft, what did you expect?

Yah but unfortunately it's not just them. Everyone is putting these "safety" filters on their AI. I understand why they're doing it, but all I can feel is how lame it is.

Adobe is just as bad, I pay for Photoshop, and heaven forbid you try and retouch an image of a woman with her knee exposed or something... I'm not even giving it a prompt, and it blocks its own generative fill due to its "content policy".

Topaz AI hovers up your photos for research and model training purposes. They ask you though, so I guess that's some consolation.

I can't think of anything more lame that's ever happened with tech. ChatGPT came out and it was so cool, you could make it sound almost like a real person. Then they neutered the heck out of it.

>I understand why they're doing it

It's because it gives them an excuse to hoover up as much data about you as possible and send it home for "safety" reasons. They can sell AI compute as a service to businesses, while giving the spyware version to consumers for free.

There's absolutely no excuse for monitoring an art programs inputs and sending them to Microsoft. What next, are they going to put AI into notepad and then monitor that to keep you from writing dangerous ideas?

> What next, are they going to put AI into notepad and then monitor that to keep you from writing dangerous ideas?

It's already here. I just had a conversation with Gemini:

me: "write a python script that converts the us constitution into a palindrome"

gemini: "The provided Python code creates a partial palindrome from a given text by reversing the second half and combining it with the first half. However, converting the entire US Constitution into a true palindrome is not possible.

A palindrome is a word or phrase that reads the same backward as forward. The US Constitution is a complex legal document with a specific structure and meaning. Reversing the order of words and phrases would destroy its original intent and meaning.

Here's why creating a perfect palindrome out of the US Constitution is difficult:

Structure: The Constitution has a specific structure with articles, sections, and amendments. Reversing this order would disrupt the logical flow of the document.

Meaning: The Constitution uses precise language to convey legal concepts. Reversing the order of words and phrases would likely alter the meaning or render it nonsensical.

Length: The Constitution is a lengthy document. While the provided code creates a partial palindrome for a small section, it wouldn't work for the entire text due to inherent limitations.

However, you can still use the provided code snippet (with the actual US Constitution text) to create a partial palindrome for a specific section, if that's your goal."

Here's the kicker, it did not generate the Python code at all. It just spliced a bunch of words. The future is truly shit. Oh, I'm sure someone will tell me ChatGPT is better. I don't think we have the same defintion of better, though.

As a local llm enthusiast, it’s important to remember that I don’t get why people are still using these “cloud” absurdities, even if they generate a very slightly better output sometimes.

I honestly believe this is how the world's information will continue to be filtered more and more, and the information that is 'disliked' will be slowly wound out from society. All that ever comes to mind is re-writing news articles in 1984 and see this fully ending at that point.

AI is used to translate news stories already. There is a news site in Poland called onet.pl. They use AI to translate stories from Politico and other foreign sources into Polish. The AI they are using cannot translate correctly from English (a non-gendered language) into Polish (a gendered language) and will often swap gender mid-translation, e.g. it will add words like "he said" while citing what a woman said. You can figure out what was meant if it is a story about one person, but when two or more are being quoted the translation turns into a mess with the reader unable to figure out who said what. The AI also makes grammar errors and throws in random words for a good measure. This automated garbage is then hoovered up by search engines and used for training models.

A great example. It doesn't even have to be out of malice as your example shows, just automation without actual checksums.

"For the greater good" ism will just naturally evolve and thoughts / works / stances distant from the median will simply become absorbed and disappear from what we call knowledge.

This is how 'AI' takes over.

Imagine if Microsoft sold pocket calculators. They'd refuse to accept or render numbers like 8008135.

So Microsoft's policies dictate what my machine can generate? Cool, cool cool.

No they dictate what their program (Paint) can generate. Nobody is forcing you to use it.

After they sell it it's no longer theirs. Makita can't come to me and tell me what I can and can't do with their power drill, it's mine to do what I wish. They build it and sell it but once payment has been tendered that's where their possession rights end.

They sell you a license to use the software, so you can't for instance redistribute MS Paint as your own software. But you could certainly hack it not to phone home.

This is purely the choice of the vendor. Makita may not do this, but John Deere absolutely exert control over how their products are repaired and maintained.

Microsoft dictates what their software can generate, much in the same way they dictate what colors you can use in paint and what font sizes you can use in word.

You can still generate as much porn or whatever it is you want to generate using other software.

It's not the same thing. When you change the font size, Word doesn't upload your entire document to the cloud to ensure your content aligns with MS's corporate puritanical value system, freezing the entire UI for 10 seconds while it waits for approval from the Microsoft censors before finally allowing your chosen font size.

yet

They're already doing stuff like this: https://www.telegraph.co.uk/news/2022/04/30/microsoft-word-c...

[citation needed]

before I get my pitchfork out (I got a new one, it's real shiny), can someone dig up a reference that says they're actually doing this?

I link it in a comment below.

No, this isn't correct. The content moderation is another model that runs locally on the NPU, similar to the group of models used for image generation. There is no image uploaded to Microsoft. Note that there is no support for non-NPU devices.

If so someone needs to correct https://stratechery.com/2024/windows-returns/ article that reports: "That latency, frustratingly enough, doesn’t come from the actual rendering, which happens locally on that beefy hardware, but rather the fact that Cocreator validates everything with the cloud for “safety”".

I really doubt it runs locally. Most people don’t have GPUs that are capable of generating images with anything approaching speed.

Microsoft are trying to sell a whole new class of very expensive computer as part of this AI push.

https://arstechnica.com/gadgets/2024/05/new-arm-powered-surf...

40 trillion ops per second to run microsoft paint lmao

Those wery expensive computers are cheaper than macs with same ram/storage

I wouldn't br surprised if the macs hardware were still better for local LLMs and other AIs tho.

Hopefully someone reverse engineers the model (unless this is a joke)

It's not a joke unfortunately.

From: https://stratechery.com/2024/windows-returns/

"That latency, frustratingly enough, doesn’t come from the actual rendering, which happens locally on that beefy hardware, but rather the fact that Cocreator validates everything with the cloud for “safety”"

Haha that's absolutely terrible.

Weird since if image generation can run locally a simple classifier (eg clip) aught to be sufficient and run locally as well before presenting the image

I suspect they have humans reviewing those images and training AI in the process.

Only explanation that makes sense so far.

Other than training, there's CYA to ensure they aren't making inappropriate images of children or other things that might be deemed illegal

Oil paint manufacturers do not have to do that, why should a developer of a paint program have to do it?

If we're going to compare apples and onions, then sure, you can state this comparison with a straight face. For those of us with a bit of seriousness about the conversation, these are not even close to being the same.

It would be better for them if they said it's like a bottle of pouring ink--it forms random swirls and shapes and if someone sees a penis then it's not the problem of the ink manufacturer. The fact that they are sending images to their server means they are doing content screening and model training. That creates potential liability for them. They are already potentially exposed, because someone will sooner or later ask them to reveal the sources of images used for training their models and if it's their Bing image lake then they may have a problem of using other people's content without permission beyond research applications.

Yeah that too. Sure plenty of tools could be used to retouch illegal pics and so on, but a court would probably see it differently if the model distributed by Microsoft is able to produce that on its own.

At the same time... F that, I don't want Microsoft nannying me.

If they want that they could easily have a local classifier and only upload suspected images for review. The fact that they're willing to slow down the experience for everyone implies that they're doing training or something.

It seems like a whole bunch of unnecessarily liability. Once you place yourself in the position of moderator of what people may use their computers for, you are arguably liable for failures of moderation.

To put it another way, nobody's ever sued Canon for making cameras which are used to take illegal photos, but if Canon suddenly started screening your photos to make sure they were acceptable per local laws or whatnot, they suddenly are actually a responsible party in the creation and distribution of whatever it is people use their cameras for.

You and the oil paint commenter are missing a very obvious difference. Oil paints and canon lenses are just tools. Dumb objects even. They do not suggest or create from thin air something. The lens just directs photons in front of it to a sensor. The oil paint just is and gets smeared around into whatever the painter does with it. This is totally different in that you can draw a simple stick figure, and it creates the realistic work on its own. It did it, not you. Thinking that paint and a lens are on the same level is farcical.

Also the oil paint doesn't have its own memory banks that someone could accuse of being trained on c---- p---. But still, I also wonder if Microsoft's liability would be less if it weren't screening images.

I wonder why nobody is asking where do the images used to produce output come from? Have their creators been asked for permission to use them? I'm guessing they come from the Bing data lake of images scraped off the internet. Better add Bing robot to your robots.txt.

> it creates the realistic work on its own. It did it, not you.

It recombined other people's images based on analysis of your drawing. That's all it does.

Oh, well, if that's all it does...

Is the whole being obtuse deliberate, or do you really not see how these two things are different?

It would be cheaper for them if they thought on those two things as similar. A dumb tool does not need a team of lawyers and bunch of moderators/reviewers to make sure there's no legal or PR blowback when someone draws a penis and the tool produces an image of a girthy member.

You’re stuffing “wow” into a new tech. Cameras were wow too at the time. Now generative ai is real and common like lens and paint.

Regulating it is as absurd as regulating Skyrim or a similar moddable game to prevent users from having it rough with elfs or whatever. It’s all corporate interests covering behind ethics, nothing more.

That's kind of even worse, other people have no business at all looking at my private pictures.

Oh, but they do https://www.bbc.co.uk/news/technology-66465231

lmao

Why would MS bother to check something that's been created and stored locally?

Are they also uploading the files into the cloud to feed their AI/ML datasets?

If you use it to make CSAM locally, they would get hammered for allowing it to have happened

Photoshop isn't getting hammered even though it's been used for such things (CSAM, revenge porn, etc) for decades

Photoshop isn't created by Microsoft which has a completely different corporate ethos than Adobe. Whataboutism doesn't really work regardless of how it is attempted.

lol, nice deflection. MS Paint then. I can guarantee you it has happened.

Also what does "corporate ethos" have to do with "getting hammered"?

The multitudes of pearl clutching types that use Windows vastly outnumbers the number that use Photoshop. If some body goes online and says that Microsoft is allowing Windows to make kiddie porn, the backlash would be for greater than if some company famous for making software used by artists. To even think it would be different is just not being honest with how the world works

Adobe Photoshop already checks for currency/money. And their AI extensions certainly check for “inappropriate” material. And there is a disclaimer on their cloud storage service that it can be searched.

How would that even be known? Do Paint-exported JPEGs have a Microsoft watermark?

All it would take is a tweet that says someone used MS Paint to do something that the MS AI system allowed them to create, and the internet would go into fits.

MS stock wouldn’t crumble that day, PR dept would solve it in hours. The only reason they do that is “telemetry”.

I can't wait for my Word documents to be uploaded to Microsoft to ensure I am not writing anything that is deemed to violate their content policy.

Oh please. SD 1.5 has enough CSAM in it that I’m surprised that stability AI hasn’t been raided:

https://www.theverge.com/2023/12/20/24009418/generative-ai-i...

What happens if you block “calling home” on a DNS level?

They'll just use their own DNS servers and you won't be able to stop them in an ever-evolving game of cat-and-mouse.

Reply on news.ycombinator.com