Accounts with GAN Faces Attack Belgium over 5G Restrictions | Hacker News Reader

Accounts with GAN Faces Attack Belgium over 5G Restrictions

public-assets.graphika.com

213 pointskbumsik86 comments

The endgame here is bizarre.

I can see a future in which there is no online trust at all. Not news, not proclamations, not even previously trusted sources. This trust erosion threatens the very fabric of government.

If I stop trusting remote sources, who can I actually trust? City government with a physical presence, well known physical police and government officials known personally to me, and no others.

This troubling trend does not end well.

Yes. It’s our own Tower of Babel. All built in the name of 3rd party advertising.

> Yes. It’s our own Tower of Babel. All built in the name of 3rd party advertising.

Do you mean Library of Babel [1]?

[1] https://en.wikipedia.org/wiki/The_Library_of_Babel: "In any case, a library containing all possible books, arranged at random, might as well be a library containing zero books, as any true information would be buried in, and rendered indistinguishable from, all possible forms of false information..."

I suspect they meant what they wrote.

The Tower of Babel refers to the story from the Bible's first book, Genesis, in which the hubris of people leads to the building of a gigantic tower, a technological marvel for its time but one that ultimately contains the seeds of its own undoing. People build the tower as a symbol of greatness and unity but in the end are scattered by God and thrown into disunity and disarray. It's an enduring story precisely because it seems to so well encapsulate our relationship with technology and the law of unintended consequences.

Borges' story on the Library of Babel is in conversation with this much older story.

This is not a call for fatalism. This is a call for trust infrastructure, trust anchors, and proven underlying identity and trust mechanisms.

What happens when you try to fake being an Estonian who holds a national ID card that utilizes cryptographic primitives? Along those same lines, this infrastructure is the very same needed for business to transact (think document/agreement execution, bank accounts, brokerage accounts, payment processing, real estate ownership record systems, etc).

If I can't prove who I am, that's a gap to be solved for by government (and many have already solved for this, it is a well worn path [1]). If you require me to attest to my identity, that's a regulatory, governance, and oversight issue. As glib as is sounds, fight misinformation/disinformation with trust (infra). Make trust the default, not the exception.

EDIT: Login.gov [2] provides authentication services for DHS' Global Entry. Why can we not use that to attest identity facts elsewhere? Why can't any citizen get a CAC [3] to use with this system? (I frequently see Login.gov is hiring for SREs, but no internal advocates/champions; why?) Why can I pay with Apple Pay but can't prove my identity without a paper birth certificate and social security card (or a passport if you're among the well heeled)?

</soapbox>

[1] https://en.wikipedia.org/wiki/List_of_national_identity_card... [2] https://login.gov [3] https://www.cac.mil/common-access-card/

Why should I trust the US government to assert who you are? Surely they make false passports for their spies all the time.

Why should I trust your Keybase proof? Because "they [Zoom now] say so"?

Those seeking to verify identities and personas are free to ignore whatever roots or trust chains they choose. Edge cases aside, the US government (in the case of your example) still holds and projects trust value (not sure how many US passports per day are used to validate citizenship, employment eligibility, and entry requirements at nation state borders, I assume it's quite a bit).

Trust is hard (there are entire industries around it), but the notion that it's impossible should not be entertained. It's a core component of modern civilization.

> Make trust the default, not the exception.

So the death of anonymity. I already hear the defenders of such a proposal: "If you don't want to give Facebook your federally issued ID then you still have the darkweb!" Followed by an endless stream of hit pieces equating everyone fleeing to non-privacy invasive platforms as nazis/pedos, and mobs of idiots demanding that infrastructure providers null-route wrong-thinkers.

Are you sure that really want a CAC? The OPM gave my biometrics and SSB to China, as well as every other military/gov employee. Except the CIA - the only one who managed to fight off the administrative record merge. If only they didn't draw so heavily from the veteran pool...

I'm not here to advocate either way for anonymity (two cents: providing privacy requires strong legislation and rigorous enforcement, Germany does well in this regard I think), simply improved trust infrastructure the country badly needs (which would be an efficient medium by which to improve trust online). No problems with my CAC, despite OPM's failure. Elect better legislators and improve working conditions for technologists in government if you want better security posture (which we should). There is a reason USDS has to hack the GS pay scale to get good people into positions of leverage.

Facebook already requires you to use a government issued ID to identify yourself if they question your profile [1]. Not a legal requirement, Facebook's requirement. Twitter also requires government ID to get a blue verified checkbox [2], or to report fraud.

TLDR: I will take a somewhat ineffective government, warts and all, with the understanding work is necessary to improve it over fatalism and apathy that brings about total dysfunction.

[1] https://www.facebook.com/help/159096464162185

[2] https://help.twitter.com/en/managing-your-account/twitter-ve...

> > Make trust the default, not the exception.

> I'm not here to advocate either way for anonymity

Is trust and anonymity somehow disconnected in your mind? Before you answer that, I'll point out that I didn't say pseudo-anonymity.

> No problems with my CAC, despite OPM's failure.

So you're either a post breach boot or you haven't yet noticed a personal impact. While I've never had the desire, due to the contents of my OPM file, I could never do any business in China under my own name without drawing a disruptive amount of attention. That may or may not be a problem in the future, nobody can say. But it can be said it should have never happened in the first place, as there was ample well reasoned warning and precedent. Anybody else remember the clipper chip? What about that "golden key" stupidity?

Whatever policy Facebook has at the moment is completely beside the point. What you are talking about would require the force of law. This proposal has been floated numerous times, tying online activity to a federally issued identifier.

> Elect better legislators...

lol

I hold contractor status, and I would never step foot in China (for obvious reasons). I think we see things fundamentally differently, and I wish you well.

I don't wish you ill, but I wish you and your ideas had no effect on me. Which is more to the point - you are focusing on the wrong part of the problem. Trust and disinformation isn't the problem, the problem is the second order effects. It would be easier to reduce the potential damage that useful idiots and victims of propaganda can do to everyone else, than it would be to pull off the impossible trust+anonymity+benevolentFed scheme.

Trust is a difficult problem.

Geopolitical entities have been using the Internet to undermine trust in "the West," broadly speaking, for at least half a decade. Warfare has always included disinformation campaigns (they called it "propaganda"), but never has it been so viral.

I could at least envision a world where cryptographic identity is taken for granted. Think Keybase for publications. Sign articles to prove authenticity.

The cryptography involved is largely a solved problem. Software engineers did it with Keybase, and it's easy enough to use that people with very little cryptographic ability can prove who they are to the Internet (assuming they're an established identity with social accounts, or they have trustworthy people willing to vouch for their identity). But we'd need browsers/clients that have the ability to display proof status on messages, and it would have to be nearly as easy to use as existing clients like Twitter.

I hope we start building businesses that repair trust, rather than harm it. Making everyone distrust everything might benefit a few opportunistic parties in the short term, but in the long run everyone loses.

>Geopolitical entities have been using the Internet to undermine trust in "the West," broadly speaking, for at least half a decade. Warfare has always included disinformation campaigns (they called it "propaganda"), but never has it been so viral.

I mean, you have to understand that the actions of the globally dominant "West" (North America, EU, Australia, and their allies) in the past 20 (and further beyond that) years have done a lot to undermine trust in them also.

Fake news to lead people into the Iraq War (a 20 year quagmire which only cost millions of lives and trillions of dollars for minute changes on the ground), an inability to defeat ISIS which meant the US had to rely on Iranian militias to beat them. On that note, I think the whole history of US-Iran relations is enough to undermine faith in the "West" as it stands.

I'm not saying other axes of power are better, but trust isn't a competition, it's very possible for people to trust no one outside their few close acquaintances. Trust has to be earned, and the "West" doesn't do a good job of it.

Interesting, just looked through notsureaboutpg's reply to your comment and it seems like someone has gone through and systematically downvoted(?) all of their comments (now marked as dead). Why?

No, trust is a very easy problem. Online trust with people that you have not had any previous real life contact with is a hard problem. Which is why I still put a premium on meeting people irl.

I suppose it should be clear from context that I meant online, but I'd like to offer some food for thought around offline trust as well.

In 2009, a man arrived at the world-famous Sun Studio in Memphis, TN for a private tour. He told David Brookings, the young, aspiring musician giving the private tour, that his name was Steve Eason. David had been briefed that Eason was a big figure in the music industry, very famous, and so Sun would have to be very careful to preserve his privacy. Eason was also darn ill; he'd been hooked up to some gear that followed him around the studio.

The kid had never heard of Steve Eason, but he gave a passionate tour. At the end, David handed Steve a CD with music on it, hoping to land a record deal. A month later, Brookings got a message from Apple, asking him to come join iTunes to curate Rock & Roll playlists.

And Steve Eason? He wasn't a musician, or even a record producer. The man receiving the Sun Studio tour was Steve Jobs, who had been in Memphis to receive a kidney transplant for pancreatic cancer. He was staying in a house bought by a man named Eason, but Eason was - in fact - the doctor Jobs had scheduled to perform the transplant surgery.

Brookings has worked at Apple to this day.

In truth, more people go by alias names than you'd expect. Without checking a driver's license (and, depending on the sensitivity, maybe some utility bills), you may never know if someone is who they say they are.

Also, I have friends I've met offline, who have moved around to different countries and swapped devices. It's always a big pain trying to establish trust once your physical relationships go digital. There are only so many challenge-response questions to ask them (i.e., things only the two of you know). In reality, our digital and offline lives are intermingled; they each inseparably affect the other, and sometimes it feels like identity problems are turtles all the way down.

Identity and trust are not necessarily 100% congruent. There are plenty of people who I trust whose identity I would not vouch for and there are many more whose identity I am sure of but who I would not trust.

Trust is the main problem people are trying to solve when hiring. Before 2020 most of my interviews were done in person, so even with IRL meetings it's a hard problem.

This usually boils down to references and then by extension the trust in those references. It is definitely tricky, before COVID we would fly in new hires to meet them and talk to them, since then we have only hired one new person and only because one of our team has a prior relationship with that person so we can extend our trust.

Why? This was the situation in the 90s. No one trusted the internet. It's hard to imagine unless you lived it, but buying something online was once seen as completely stupid, and people were universally skeptical of information (and people) online. We should be more concerned about the level of trust built up since then, not the other way around.

Trust no one? Have people historically trusted anyone other than their families?

That might bring it back to actual people. If you can't even tell which pop artist is real, or which talking head is legitimate, maybe you trust the people you know. We go back to trusting that newspaper that you pay, who in turn pays someone to actually be on the ground somewhere. And trusting people you personally know IRL. No more of this reporting on things the journalist read about on twitter and read a press release about.

Implying you trust government and police, because they never done anything malicious before...

At least with online sources it's up to you to decide.

Not really trust, but it's the local gang with the physical power and presence to operate conspicuously in the area.

Ehh.. ive thought about it, either for fake news or fake identities. Maybe also people will start spending less time online and care more about local stuff that overall, will impact their lives more directly.

Maybe not that bad of a thing.

This is basically how I already treat the Internet. I take a defensive stance and just assume everyone is a foreign bad actor or shill.

I am 100% convinced that if Twitter expanded their verified account program then these types of attacks would be rendered obsolete very quickly.

It just doesn't make sense that there is no way for an average real person to get verified and display an instant signal that differentiates them from a fake bot. It's absurd and it causes real harm.

Why stop at Twitter? I would expect some service to exist (any maybe it currently does) that can easily verify whether or not any account is driven by an actual human.

But forced verification = people complaining about the lack of anonimity (which I would support too). Whistleblowing would be more restricted (or they could trust Twitter to keep their identity a secret, until a Twitter employee leaks their info for some money, e.g. Saudi money: https://www.buzzfeednews.com/article/alexkantrowitz/how-saud... )

Why not make it pseudonymous, and then federate identity verification by having neutral third-parties attest to who you are?

Because approximately zero non-crypto-geek humans would participate, as the mechanism is inscrutable and of dubious value.

The complete failure of the PGP “web of trust” is a precedent.

If verification is easy and accessible, what’s stopping a real person from getting a simple verification and then loaning out their account for bot actions (for a price)?

When you get caught you get booted off the platform.

As we can see with all those Twitter accounts that got bought for their usernames, which is AFAIK not allowed, but here we are, hey? /scnr

>> If verification is easy and accessible, what’s stopping a real person from getting a simple verification and then loaning out their account for bot actions (for a price)?

> When you get caught you get booted off the platform.

Which may not actually be a problem for most people. The thing that's stopping me from selling my Twitter account to a disinformation network is not my fear of losing access to Twitter, it's that I care about the problem of disinformation and don't want to see myself in the news for something like that.

I'm sure there are thousands of people who'd sell a simply-verified Twitter account, and they probably wouldn't even demand that much. People are already spending hours a day trying to sell their nudes, and still only making a few hundred dollars total (https://www.nytimes.com/2021/01/13/business/onlyfans-pandemi...).

if verified accounts would be guaranteed to be able to post anything and hold accountable in court(not by twitter) that would be a completely different situation.

in this case, an average user selling verified account would risk legal and financial consequences. It's unlikely spammers and other bots would be able to afford buying such accounts in mass.

And additionally verified accounts would be out of touch for any censorship.

This nefarious social credit scheme operated by tech oligarchs who will subject everyone else to it unwillingly brought to you by seniorvn as an attempt to help us 'trust'.

How about no.

Buying verified twitter accounts is also a non-scalable solution. Sure in the short run you could probably get quite a few, but in the long run there are only so many people.

There's also a built-in compensating factor: To the extent that twitter accounts are worthless, it's easy to buy them. To the extent that twitter accounts are an important part of your identity online, people will tend to protect them. Try getting people to sell you their social security numbers.

Right now they're closer to the "worthless" end of that spectrum. But maybe verification would change that?

> Buying verified twitter accounts is also a non-scalable solution. Sure in the short run you could probably get quite a few, but in the long run there are only so many people.

I don't think that's a problem if your goal is disinformation or manipulation: the report this network details consisted of only 14 accounts.

> There's also a built-in compensating factor: To the extent that twitter accounts are worthless, it's easy to buy them. To the extent that twitter accounts are an important part of your identity online, people will tend to protect them. Try getting people to sell you their social security numbers.

> Right now they're closer to the "worthless" end of that spectrum. But maybe verification would change that?

I guess I'm disputing the presumption that Twitter accounts will ever be that valuable across all the members of society that the risk of loosing access to Twitter will be enough of a deterrent to any particular rando out there. Twitter's appeal seems to be mainly limited to certain slices of society (e.g. politicians, political pundits, and wannabes), and there are probably far more people outside those slices than inside them. If a rando waitress can get a verified Twitter account, and such accounts are useful for spreading disinformation, the GRU and black-hat PR agencies will probably be able to get all the accounts they'll ever need for something on the order of ~$100 a pop.

So I as someone that has never used Twitter could make a few bucks if I joined - worst case I’m back where I started?

It will ding your social credit score. As in other people will trust you less, and not just on twitter.

Why would someone trust me less not just on Twitter?

I think they would make less money if they stopped receiving ad dollars due to fake views.

Of course. Which is why they don’t do it :)

Maybe they could charge people to become verified?

We debated doing this as an outside service on any platform then creating filters but it’s dangerous as a business because it’s so trivial for Twitter to decide to do it. It is remarkable they haven’t done it. Facebook too.

Online verification .. what private data do you wanna know. With the actual rate and size of personal data leaks, we are close to a point, where it is pointless to do any online verification.

It just doesn't make sense that there is no way for an average real person to get verified and display an instant signal that differentiates them from a fake bot

It's the old tech cliché: "Because it doesn't scale." Which is just an excuse for "We're lazy and don't want to spend money on things that don't directly benefit our cafeteria and office toys."

After quitting Facebook a couple of years ago, I tried to log in to my Facebook account back in December to say Merry Christmas to some people, but I am locked out. Facebook asked me to send in a government photo ID, which I did. Nothing has happened since.

Responsibility doesn't scale. Accountability doesn't scale. Service doesn't scale. Doing the right thing doesn't scale.

On the plus side, I'm still not using Facebook.

>After quitting Facebook a couple of years ago

I read that as: After quitting Facebook as an employee a couple of years ago

Which made this:

> I tried to log in to my Facebook account back in December to say Merry Christmas to some people, but I am locked out. Facebook asked me to send in a government photo ID, which I did. Nothing has happened since.

Sound so very petty and cold blooded. You know, the Facebook we all know and love ;)

Three days ago there was a hacked verified account that had done a name change posing as one of the Winklevoss twins trying to get people to send BTC to a random address. That is to say trust is a reoccurring cost and not as trivial as “Has checkmark” at T=0.

If we should have learned something from Facebook than that using a real name won't stop people from spamming and agitating.

Twitter has conflated their verification process with "we support what these people are saying". Since 2016, there has been a growing number of people "de-checked".

Twitter could certainly make a new check system, but it would likely be (ab)used the sameway, eventually.

Im not going to verify my identity on social media for fear or being canceled.

"most of these images can still be identified by a range of features, notably asymmetries on both sides of the faces and a lack of detail in the background"

Mismatched, or a missing left or right earring is a pretty strong tell for the GAN Faces I've seen. Mismatched ear shapes as well.

Anywhere with high gradient variability is where to look for irregularities. Ears... because of earrings. Around the eyes... because of glasses. Hair accessories, necklaces, locks of hair, borders with clothing, they're all giving away most GAN faces.

So you just need a GAN trained with a dataset that excludes earrings and glasses. Then maybe another GAN trained to add earrings and glasses onto faces that don't have any :)

radar guns and radar detectors

The StyleGAN2 paper[0] actually addresses some of the biggest clues. They didn't completely solve everything but symmetry is one of the biggest indicators. Specifically ears and eyes. But also really pay attention to the teeth. If you go to ThisPersonDoesNotExist[1] you'll see that there's still phase issues with teeth and eyes. It's an improvement on the original paper but it is still weird. For eyes you'll notice things like them not pointing in the same direction (this isn't always super obvious). Another big thing to look at is the neck. Sometimes women will get Adam's Apples and men will lack them. Collars won't be symmetric and necks can have weird wrinkles. Lastly pay close attention to the background because that's where you'll likely find monsters. Lots of portrait photos have blurred background so "lack of detail" isn't a great metric. But backgrounds that are a bit disjoint are great indicators. None of these metrics alone is a great tool and you gotta use them together (unless someone messed up big time when selecting the photo). Poke around through [1] a bit and you'll start seeing some of the features I'm discussing. But I should also note that these flaws are known and being worked on so this information's usefulness will degrade with time (it's still good information because it helps us know how to look and what's difficult about faces).

These of course go along with standard identification techniques like lighting, shadows, etc.

[0] https://arxiv.org/abs/1912.04958

[1] https://thispersondoesnotexist.com/

Just saw ads for Huawei 5G on Twitter, which brings an interesting twist to this, as it is essentially Huawei paying Twitter to look the other way.

And here I sit, unable to have more than 1 bot on Twitter because it asks for more phone numbers than I have.

Do they just buy phone numbers for verification?

yes

https://www.google.com/search?q=site%3Ablackhatworld.com+sms...

I wonder if there's a means to verify how long a phone number has been in service and track history and association with spam in the past. Not a really great signal to develop though I suppose, could cause a lot of trouble with people inheriting a new phone number. One time use numbers are probably not going to happen either just by breaking the complexity barrier.

As nefarious as it is, that was a really fun read and fun idea to think about. But I can't help but think that something around this scale probably wouldn't work without some sort of state sponsorship/assistance.

You have to wonder if Twitter has any incentive to actually block these sorts of attacks. Especially as the attacks get better and harder to detect, the less it impacts Twitter's bottom line.

It probably counts as more active users and more ad views so I guess it's fine by them?

Facebook already established it's fine to fake audience data to advertisers.

Exactly. Twitter wants to increase active users and stats around posts and other interactions. Those are used to then drive sales of ads. Active users doesn't tie cleanly to the effectiveness of ads, but it looks like a nice top of line number.

These bots are the perfect targets for showing ads about 5G given how interested they are in it! Maybe someday we can just do away with human run accounts.

WasteNet: A botnet designed to waste as much advertising cash as possible as quickly as possible.

Download the Chrome extension and you can "adopt" bots which will be used to destroy more and more ad value.

Move over white males ages 18 - 34, the new most valuable demographic will be 1 - 2 week olds who are digital bots.

yeah I can identify people of the GAN race too, but only if it is a GAN that I know about and have played around with

there is more out there than thispersondoesnotexist

What we're seeing here is, in my humble opinion, one of the least effective fake news campaign, almost by an amateur agency.

Just imagine what real sophisticated, skilled and state-backed campaigns can do, undetected.

Nice article and great research. Nowadays it’s getting harder and harder to distinguish “real” reality online :-)

Is there any evidence any of these 'bad' actors are running their own GANs yet? I've yet to see an analysis of where the pictures come from.

This is possibly a way to attack future tracking, thousands of GAN generated pictures of oneself to pollute databases.

It's hard to know if it will work since the databases can use the same reputation systems your friends will to keep their info clean. I guess in the coming war we will see.

These will only get more sophisticated and targeted ... I wonder how many were not catched yet

The real bad actor here is Huawei. Does anyone still think Huawei is just a simple tech company trying to do business?

I mean, the purpose of this attack was to get more business...

That said, no big tech company is a simple tech company anymore. They've all become (arguably they always were) political, geopolitical, and adversarial entities.

No, the purpose is to subvert the truth. What happened to do no evil? I dont accept that is how business is done and why should you?

Yes, the purpose is to subvert the truth so that Huawei can get more contracts and more money and more power.

Do no evil has never been the modus operandi of large corporations. They have always been amoral.

I'm not being facetious when I say that maybe only the bots think that anymore.

The bots are on this platform as well.

>The real bad actor here is Huawei.

Anyone could do this, and there are plenty that would like Huawei to look bad.

The report indicated that official Huawei accounts retweeted these bots, so...

It's likely that this is indeed done by Huawei, but if it wasn't it would be very likely for Huawei to inadvertently retweet some of them.

Great research!

Reply on news.ycombinator.com