Hacker News Reader

Comment by weberer | Hacker News Reader

The AI aspect of this is a red herring. The real problem is that they're secretly adding in a unique identifier into every image you create. If somebody does not like your meme, they can just send a copyright subpoena to Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account. Just like age verification, this is another weapon in the war against internet anonymity.

View on HN

> a unique identifier into every image you create

Printing (even text) is also a risk: It's very likely your printer is secretly adding marks to the page that contain its serial number and the current timestamp. [0]

Meanwhile Microsoft (and Apple) have "telemetry" harvesting those serial numbers of all internal and external devices you've ever had connected or reachable. Then they link them to your MS/Apple account, IP addresses, and the extended social-graph of all computers that were ever in the same room or shared the same bluetooth speaker.

In short, your "anonymous" flyer critiquing The Regime and depicting Dear Leader as a clown could lead thugs straight to your door. Or to the door of whomever you're staying with.

[0] https://www.eff.org/issues/printers

and then in few years a new mono-mustachioed or mono-browed leader will emerge who declares memes a blasphemy punishable by death and will call up all the telemetry gathered to punish all involved.

"I don't worry because I have nothing to hide, based on my perfectly accurate perception of today's political and legal forces in my state and nation which will stay safely static forever."

You know a certain dictator denies access to the country they control based on whether visitors "like" them? They use a private militia, funded by stealing from taxpayers, in contravention to that country's laws, to target citizens who are not supporters of them. Some of those citizens get shipped abroad to foreign prisons by these militia. Others get shot in summary executions in the streets - the controlling regime covers it up with false media reports, and uses corrupt judges to evade scrutiny ... a certain software company support that regime, even acting against foreign scrutiny on their behalf.

Essentially all of that country's businesses that are close to the regime need to be considered hostile.

We don't need to wait a few years, the revolution already happened; the insurrectionists were freed. These companies paid their tributes, in dollars, to the regime.

Maybe next ML will be integrated into software suites to enforce that regime's lies? Most Western governments use such software, the distributers of which have already shown they'll act in the regimes interests against supposed allies...

I had assumed parent-poster was making a sincere but humorous point, as opposed to sarcastically dismissing the risks of a change in management, but I suppose it's ambiguous.

Good satire is often indistinguishable from fundamentalism.

Also known as Poe's Law.

https://en.wikipedia.org/wiki/Poe%27s_law

> It's very likely your printer is secretly adding marks to the page

It's most likely how the FBI caught NSA leaker Reality Winner:

> Both journalists and security experts have suggested that The Intercept's handling of the documents, which included publishing the documents unredacted and including the printer tracking dots, was used to identify Winner as the leaker.

https://en.wikipedia.org/wiki/Reality_Winner

Yeah - these criminal agencies use this to protect their billionaire buddies on top. The Epstein network must be really huge. I always wondered how one or two person can meta-coordinate +5000 underage girl and their +10000 customers. That never made any sense to me, yet suspiciously enough only Ghislaine is in prison. That does not add up.

There isn't just one Epstein network. There are many similar networks.

This seems like something that could be confirmed by reverse engineering a printer’s firmware? Surely someone has done that?

It is confirmed and the algorithm has been reverse engineered, though not by looking at firmware

https://w2.eff.org/Privacy/printers/docucolor/

Seems like the algorithm is simple enough that they did it just by looking at some prints

No need to reverse engineer, printers are required to do this for decades.

You'd still need to reverse engineer it to build a better firmware without this anti-feature.

And the reason why it refuses to print this B&W document when yellow is empty.

You don't need reverse engineering if it's publicly known fact for 20+ years, telemetry just identifies you. 20 years ago, you would need to find the printer.

So what you're saying is we should use linux?

Until it gets outlawed for not telling websites what age you are.

>Then they link them to your MS/Apple account, IP addresses, and the extended social-graph of all computers that were ever in the same room or shared the same bluetooth speaker.

Did it get revealed that Apple did that? This is news to me

Exactly. Forget the AI aspect, this is entirely to identify users for any purposes they deem necessary. People are ignoring the surveillance state aspect of this. Reminds me of the device id debacle they have attached to their outbounds Windows network calls

Can this also be weaponized? Get an innocuous image from someone you don't like, grab their GUID, add it to another image, sent it to the thin skinned politician in power.

No, because the GU in GUID stands for globally unique. Adding it to a second image would cause a detectable collision.

Also, C2PA, another technology mentioned in the article, means tampering is easily detected.

> Adding it to a second image would cause a detectable collision

So? Still the weapon worked.

Collision only detectable when you have both messages. Detecting a forgery seems more likely.

It depends on which version of GUID it is. Only version 3 and 5 are hash based. The other versions are random, time, and/or MAC address based. Version 3 uses MD5 and its cheap to get the same hash for different images: https://crypto.stackexchange.com/questions/1434/are-there-tw...

It's like Snowden said (paraphrasing): "If privacy comes from policy, the policy can be changed on a whim, and in effect you have no privacy at all"

Does it trigger on non AI images? The post doesn't say so at least.

It does kind of say: The GUID is coming from the moderation endpoint, which is hit when you generate a local or cloud AI image based on your prompt. If there is no prompt, there is no endpoint, and likely no GUID.

Obviously Paint could have been watermarking prior to AI though, but this specific AI watermarking appears to be only that.

The fact that local ai image generation uses an online moderation API is a bit worrying too....

Why not just mod the app to not call this API?

Since the watermarker runs locally, you could also do the reverse: generate some porn with a different open model, then run that code to tag it as "Content watermarked by Microsoft Responsible AI"

Probably both so they can change what they censor without waiting for you to run windows update and also so they can collect your information (IP, timestamp, etc) to associate with whatever you did

I don't think they use a model that would run smoothly on most hardware that normal people use.

Besides, you need to sign-in and pay to use that feature. It's very obvious that's not local.

did you even read the article? you're wrong on all points.

Frankly, I only skimmed it. Now I see that "On Copilot+ PCs, image generation is local but prompt moderation remains remote".

I don't have a Copilot PC, but if you still need to pay MS to run the model on your own hardware it's laughable.

No, you don’t need to pay for using on-device models.

> Microsoft to instantly get your full name, address, email, phone number, and any other data associated with your Microsoft account.

Provided you bent the knee and created a Microsoft account.

I was forced to create multiple as part of the mojang migration - truly a nightmarish experience to use their AuthN, haven’t seen something quite as bad since

Maybe you were "tricked into", but probably not forced. We are free to refuse, at cost, but free.

The cost is that you can never play Minecraft again

It might be small cost in retrospect.

Nah I sunk many thousands of hours into these accounts; definitely not a small cost. Sure I could get a new account and pay twice for the game I already purchased multiple times - that would’ve ground my gears too

I think the new account would have to be a Microsoft account also, so you wouldn't have even gained anything by not migrating.

Some servers allow joining without a Microsoft account even these days, but probably can't play on any of the servers you regularly play on, if you don't have a Microsoft account.

Including old pre-enshittified versions.

Never play online

That's half the game.

They used dark patterns to trick me into signing up. I do not know how anyone can trust them, at all, ever.

Assuming you are on windows, you probably have

Not even. They log the IP and your ISP (or VPN provider!) knows the name/address of the person using that ip at that time.

> every image you create

*with the help of AI*. Does in fact make a difference.

While it does make a difference, their willingness to quietly integrate watermarking features into what people saw as simple apps for doing simple tasks is unnerving. It only takes a small change for them to start baking your identifiable information into all images they edit, or some government politely asking them to do that.

I wonder if in ten years we'll have a horrifying world where everything that leaves a machine is imprinted with its permanent identifier. Every file comes with a verifiable history of who created it, what computers it passed through, who made edits. We're closer to that world than we think.

There is a certain convergence of wills here: watermarking AI products on the one hand, and infusing AI into everything on the other hand. When you layer generative AI into Notepad, once just a raw text buffer, then you’re setting the stage for watermarking everything because AI touches everything.

AI watermarking is now the law.

Imagine the horrifying world where nothing is - and a so-called AI is thereby free to impersonate anyone.

We already live in that world, and the sky hasn't fallen so far. Even in the worst case scenario, the world isn't going to be as horrifying as the alternative, because people react to the proliferation of scams, they will not keep falling for fake impersonation for the 10000th time a row. In this scenario, we simply will stop taking factual-looking images at their word in the same way how you don't take any piece of writing on the internet as factual, verifiable truth. Journalism will rely on a person vouching for their images with their reputation, and images with unknown sources will be looked at with suspicion.

I am not willing to pay the price of sending every bit I create to some megacorporation's server or stamp it with my only identity so the computer can tell me what's AI and what isn't. Especially not while the average quality settles into an uncanny valley that's often discernible with the naked eye, both for text and images.

> Journalism will rely on a person vouching for their images with their reputation

Their images? I don't know how they'll be able to prove that.

Why would they need to subpoena Microsoft and why will Microsoft hand them the data if someone doesn't like my meme? I understand what you are trying to say but it does not make a whole lot of sense even from Microsoft pov. They are not bound by law to hand over personal data to anyone without a warrant issued. If a warrant is issued then it is their lawful duty to give that data, the same would be done by you if you were in their shoes or your office or your family. Law is not optional to follow. But Microsoft doesn't need to do watermarking on memes to track you or to do this as a war against "internet anonymity".

Watermarking is just one puzzle piece in the surveillance state; the legislation around it another.

I would say the watermarking is not really a puzzle piece at all when it comes to the surveillance state, which was my point. They don't need watermarking to know who you are or to get information about you. So why would they work on this really visibile method if their intention was to progress the survelliance state? Microsoft with all their resources (and assuming it is an evil corporation purely acting from the state's interest to spy on you) will use this to track you?

You can incentivize cooperation without having to compel action with a warrant. This sort of corrupt quid pro quo is quite common, in other countries at least...

How can they send a copyright subpoena for work which cannot be copyrighted? There is a contradiction here between "no it's FAIR USE" and copyright.

> address

Do not tell Microsoft where you sleep. IANAL but they are not a government or financial institution and do not have a right to that information.

Make sure you register mailing addresses with your credit card institutions in addition to your residential addresses, and make your mailing address your billing address so that you aren't forced to tell a thousand businesses like Microsoft where you sleep.

Windows 11 effectively forces users to register with a Microsoft account. Once that's established, all it takes is for an unaware user to fill out an e-commerce form and save an address for auto-fill.

> Windows 11 effectively forces users to register with a Microsoft account.

It takes zero effort to bypass that with Rufus, if you set up your own pc.

My comment was in regard to the average PC user. The kinds of folks who would never install an operating system and would likely agree to use Edge while signed in to their Microsoft account.

Microsoft has also been pushing hard for Recall and recording all local activity forever. Not impossible to imagine that anything looking like a home address “somehow” gets ingested in the telemetry.

A bypass existing is good, needing a bypass in the first place is still a problem.

A bypass existing is actually not good because that's how the frog is boiled. Ignorant users get an account while power user are placated for now. Then at some point the bypass is removed and when power users speak up they will be told shut down by all the drones with "you have needed an account for years, why are you throwing a fuss now".

> Windows 11 effectively forces users to register with a Microsoft account.

While it's definitely a dark pattern that I 100% do not agree with, Pro editions still allow you to do a local account. No need for the oobe /bypass command, still can be done through the OOBE GUI setup by selecting a Work/School account option then selecting Sign in options to then specify a local account to create.

This is broadly impractical for the average citizen. A scalable solution would be to make this sort of thing illegal.

> This is broadly impractical for the average citizen.

No it's not. Sign up for a virtual mailbox for $15-$25/month.

> A scalable solution would be to make this sort of thing illegal.

I'm posting this in the genuine interest of people being able to maintain anonymity from data leaks, privacy leaks, and in general not needing to tell businesses more personal information than is necessary to render services. This is in a country that has no protection of personal safety whatsoever, and any business data leak could mean life or death to average citizens who are being threatened by criminals, stalkers, and more.

It seems every time I post something of this flavor the same handful of you come out of the woods and want to make privacy illegal, and I'm not sure who you are trying to support.

I think they meant make what Microsoft is doing illegal?

or make it illegal to ask for address, etc. definitely a little more effective than mailboxes

Ahhh okay if that's what they meant, then yes, I 100% agree and apologize in advance.

I fully agree that businesses should not be asking for addresses. Non-financial businesses don't need to KYC in the first place, and financial institutions can KYC without needing to know where you sleep.

I got triggered because people seem to always want to come out of the woods and say "addresses should be public record" or things of that sort and I vehemently disagree in the interest of privacy, in a country where a stalker can just look you up, terrorize you, and the police will do nothing about it.

No, it was my brusque wording that maximized ambiguity. I am with you about the importance of privacy—we otherwise risk immeasurable horrors.

They (the ones wanting to make privacy illegal) are just the ones that want to develop the tools and consume the data for it because it'll be good for their resume or some bullshit, or they're just intrinsically broken humans.

> No it's not. Sign up for a virtual mailbox for $15-$25/month.

Did you notice that it's an affordability crisis out there? An absolutely enormous number of people are skipping bills, taking on credit, and using predatory lenders to make end meets, and your recommendation is to add another fee on top of things.

It's not practical or useful guidance for the vast majority of people. I strongly agree with supporting privacy, but this sort of behaviour (adding trackers, etc) to normal functions without a full disclosure and opt-out mechanism must be made illegal, otherwise we are just creating markets for "privacy preserving" technologies that are increasingly less likely to actually be effective for that purpose but sure do put on a good theatre of seeming that way.

At least in the US it's not even a good method either. Own a house? Boom, your information is publically available and you won't be able to remove it without a court order, which in most states is impossible to get. In most states, you don't even need to own a house, decide to vote? You information again is publically available. Tie your real name to a single account to purchase something, and you are trackable. Want privacy? You'd need to remove yourself from the internet, dump your phone, destroy your ID and work underground for cash. We can push against iot, and people have been forever, but at the end of the day the government will always they have a vested interest in being able to identify it's citizens.

As a state employee, even more of my info is online. One of my coworkers pointed this page out to me:

https://transparency.ky.gov/search/Pages/SalarySearch.aspx#/...

> and work underground for cash

My sister was a member of Scientology for 2 decades and she was encouraged to do this. As a result, she has almost no Social Security earnings and her Social Security check is about 1/4 of what she would be getting if she reported (and paid taxes to the government instead of to her "church") her income.

> decide to vote? Your information again is publicly available

I ran for elected office in the past. When I asked the voter registration office for a "walking list", they only checked that I did get on the ballot and I was provided with a list of every registered voter in the district I was running for. It had names, addresses, phone numbers, political party and a list of what elections you voted in (I think it went back 8 years). Some people were rather upset that I had access to that information when I knocked on their door and asked them to vote for me.

In many states voter lists are publically available online. And those same databases will tell you what party the person voted in for primary elections. The amount of information publicly available about everyone in the US is astounding.

> No it's not. Sign up for a virtual mailbox for $15-$25/month.

You are out of touch.

We stopped paying for a USPS PO Box when the price of the smallest one passed $150/year.

> Sign up for a virtual mailbox for $15-$25/month.

That isn't going to stop Microsoft from collecting your address by collecting your wifi info, or from the data you enter into websites or documents. When the maker of your OS is the enemy you will always lose.

Technically true but I don't think their engineering is advanced enough that it actually happens that way.

Why not? Collecting Wifi access point SSIDs to enhance locating has been bog standard in the industry.

Yet another subscription plus the hassle of managing that mailbox is impractical for most people.

> IANAL but they are not a government or financial institution and do not have a right to that information.

As soon as you purchase something from Microsoft - e.g. your Office 365 subscription - they have at the very least your billing address on file for the credit card.

Seems to me that it's a challenge to make an apple account without adding a credit card to your account as well. A carfully chosen Linux distro (so many options!) is the way to avoid the brunt of these privacy issues, lots of open software calls home in some way or another so user beware!

We gotta stop normalizing that. Why do you need my address to receive my money? My birth date?

Even in places with strong privacy regulations requiring businesses not to collect data they don’t need, businesses apparently get away with asking this.

The problem is CC processors. You can get away as a merchant with just a number and expiration date, that's the minimum you need to process a CC payment, but you pay exorbitantly higher fees. The more details you collect and pass on, the lower your processor's risk fee will be because the more info, the more likely it is that the customer is actually the cardholder and not using fraudulently obtained data.

And as soon as you connect with telephony systems (e.g. VoIP numbers) or rent out servers, some countries' telecom KYC laws apply that also force MS to collect validated address data.

Even without the CC processor requirements they would want to be able to identify you in case of a payment dispute. Digital payments are more like an IOU than like cash.

Right, but they’re avoiding that risk by passing on a risk to the customer, that their PII will be mishandled.

Because handling that PII is not only cheap, it’s profitable!

In the rest of the world CC processing uses a PIN code + online revocation check (when the card is present) or a redirect through your bank (which hopefully uses some type of 2FA). Works a lot better & safer, and you should wonder why they pretend to need to violate your privacy in the USA…

> Why do you need my address to receive my money?

Because of the volume of stolen credit card numbers. If you have the address, you are much less likely to be someone using stolen CCs. Seriously, most of the crooks are that lazy. So the real reason is the credit card processing companies.

Also, post 911, there is this thing called "know your customer". It was set up to fight money laundering and financing terrorists.

https://en.wikipedia.org/wiki/Know_your_customer

Indeed. This also means I can no longer use Microsoft apps, because they spy on me.

Well yeah they know my John Doe info

add your IP, location, provider, computer specs, dimensions, screen info, nearby devices, etc etc etc

Ain't nobody anon anymore thanks to the image recording GPS radio in the pocket.

That was largely my read of the situation too - it would be a colossal GDPR breech against every EU user, surely? That's got to be a €Billion fine??

Presumably USA are complicit in this spying on allied countries - did the countries know, is it a Five Eyes thing?

This really needs to be hit with the GDPR hammer. Microsoft have not obtained consent for this.

> This really needs to be hit with the GDPR hammer. Microsoft have not obtained consent for this.

At best they'd just disable it for EU... assuming they didn't successfully argue "it was in the ToS ..."

But the EU mandates watermarking of AI content.

As the article explained, the EU does not mandate a prompt-specific GUID, only the ability to identify the content as AI-generated. The highly privacy-invasive level of provenance tracking which Microsoft has added goes beyond the EU’s new mandates.

Why would the EU possibly object to this? It is exactly what they want.

Ummmmmmmm. No.

Inserting a tracker that can personally identify me without my explicit opt-in consent is a GDPR breach. I'm surprised you do not not this.

So you will defeat chat control by sitting there going “but you don’t have muh gdpr consent for that!”?

The whole EU vision is they know what the little people do all the time, think all the time, and spend their money on all the time. For the children, obviously.

If you think "chat control" is going to pass, I've a bridge to sell you.

In the meantime, the GDPR is your friend. The amount of FUD spread about it on here by those working in Adtech (and whose very salaries are dependent on invading peoples privacy) is insane.

> If you think "chat control" is going to pass, I've a bridge to sell you.

I genuinely wish you were right, but you are so naive it is frightening.

> The amount of FUD spread about it on here by those working in Adtech

You are confused. You are arguing with people that want to protect privacy. The EU demonstrably is not doing that, as WhatsApp gets ever more de facto mandated by the day.

Watermarking ai generated stuff is mandated by EU

Do they require the watermark to be a unique token that can be associated with PII?

It doesn’t need to be a personally identifiable watermark.

No - there is a huge difference between "this AI created this image" and "this user did it" - the first we need more of, and the second is a big privacy concern. The article does not say anything about identifying a user.

Article literally says it adds a guid, not a binary field indicating that the image is ai generated.

A guid isn’t a user id, if it is in this case, it’s an abuse of how guid is used, at least colloquially. I haven’t read enough to understand if it is user / machine id, I.e. only globally unique in the sense it’s a globally unique entity identifier.

Microsoft Can Track Users via a Windows Device ID https://news.ycombinator.com/item?id=48815196

Microsoft GDID telemetry includes full browsing and gaming history https://news.ycombinator.com/item?id=48787239

They stated the GUID is used to identify the prompt used to generate the image. How are you confused as to not being able to identify the user?

"I haven’t read enough to understand". Oh, now I know the answer to my question

a GUID isn't an indicator, it's a fingerprint.

so unless you want to draw a distinction between 'user' and 'machine' , yeah it is for identifying users.

to believe otherwise, especially with Microsoft involved, would be incredibly naive to their history.

This a variant of the provenance scheme: C2PA its implemented by all major Camera maker, and Google it seems, Apple support it with 3rd party apps on iPhone, but they have something called "Apple Reference Image" brewing.

Personally I think there is a good argument for being able to distinguish AI generated image and video...

> I think there is a good argument for being able to distinguish AI generated image and video...

Neat but that's not what's being built here. What's being built is "we can trace back this content to who made it" which is bad. Doesn't matter if today that it's limited to AI generated content. Won't be tomorrow. Your devices should not act against your best interests. No cop in my pocket please.

Reply on news.ycombinator.com