Facebook Discovers Fake AI-Generated Profiles | Hacker News Reader

Facebook Discovers Fake AI-Generated Profiles

nytimes.com

293 pointsdpflan200 comments

It's ironic that as we move into a world where anyone can be a "journalist" and anyone can have a voice, we need professional journalists more than ever.

Now a journalist will not only need training in writing well, conveying a story with facts, and investigating, but they will also need to become experts in identifying fake source material.

In a world where anyone can make a video or photo of anyone doing anything, we'll need to rely on reputable journalists to vet that material for us.

Saw a video on YouTube of the President saying something bad? No idea if it's real anymore.

Sure, a lot of this technology has existed for a while, but it was usually in the fringes (see The National Enquirer). Now it's going mainstream.

And the worst part is, there are a whole group of people who will not trust a professional journalist (sometimes with good reason) but will trust anyone online who happens to provide evidence that boosters their existing opinions.

In part, imho, the distrust comes from the the standards of the highest tiers of professional journalists being on a low right now too.

Journalists are following clicks instead of the other way around

I'm doubtful that most journalists actually want it to be that way. Those are just the economic and reader incentives in journalism today.

There are some great stats about just how much the number of professional journalists has shrunk in the social media era.

The very premise of for-profit journalism seems questionable at best.

>in the social media era.

In this context I think it should be termed "now that there are no classified ads."

I know it's easy (and tempting) to put the blame on social media alone, but there's more than that. The rise of social media certainly accelerated the evisceration, gutting, and - as some might say, annihilation - of journalism but the tailspin trajectory has been in the making for quite a bit longer.

Here's a thread from January this year, where a journalism professor lays bare the facts about the decline of his domain: https://twitter.com/jeremylittau/status/1088503510184927233?...

If my memory serves me right, that thread was on the HN front page at the time. Essentially the expense of quality journalism was seen as frivolous in the 80's, got nixed, and by the mid-90's newspapers relied on their established captive audience to churn out reliable profits with little appetite for innovation. Who was going to compete with them, anyway?

Internet changed the landscape, and social media has been instrumental in hammering nails in the coffin. But the decline of journalism and destruction of quality newsrooms has been going on since before there was public internet.

People don’t want to pay for good journalism, nevermind spending time to read and evaluate if it is or isn’t good journalism. Maybe it’s not even realistic for everyone to do that.

People also don't want to pay for good government.

As a rule, people don't want to pay for anything that benefits them in a second-order way with non-immediate consequences - even if objectively the benefits are real.

It's a classic economic monkey trap. Holding onto something now is "better" than letting go and getting a bigger benefit later.

Which suggests that perhaps we should be doing more to eliminate monkey traps from our economic and political systems.

>People also don't want to pay for good government.

Maybe because it’s already a paid service and its quality still has to be observed. But if you take their government from them, returning that substantial income cut, they instantly want it back.

This doesn’t have to invalidate your point, but this particular comparison seems pretty shaky. (Or it’s not and I’m missing something.)

My stance is that it's anti-your_country not to want to pay for good government. All of it, tax breaks for high earners, shelters, financial engineering to reduce taxes paid...all of it is treasonous to your fellow citizens.

It's not the journalists but their bosses that are the problem.

It's not the bosses but their customers that are the problem.

Given the bundled nature of mass news media, it's impossible for customers to send a signal for example that CNN news specifically isn't up to their standard. They can cut the cord on a whole set of products or keep their cable bundle neither of which sends signal to CNN about the quality of their journalism.

More niche publications don't have that problem, but I think a big question is how we systemically increase the quality of mass journalism.

At the end of the day, they are all individuals with their own biases, ambitions or plain irrationality. You just have to take that into account in all you read and hear.

One of the scariest quotes for me was Facebook's Little Red Book in 2012 (a book handed out to every employee) that had this quote:

> "When everyone has a printing press [i.e., Facebook], the ones with the best ideas are the ones people listen to."

In retrospect, it gives a ton of insight into the flawed beliefs within the company.

https://twitter.com/nemild/status/1006533287378968576

https://www.businessinsider.com/inside-facebooks-little-red-...

I can see this being a naïve but realistic take given what we knew in ~2008. In 2012 it should have been clear that this model could be gamed. And it's on Facebook that their culture wouldn't allow them to see that.

You’re assuming they didn’t see that. Given it’s now nearly 2020 and we’ve had years of reports of election meddling yet Facebook are only calming down on suggestive emoji - stating that they have no interest in policing “free speech” when it comes to political ads - I’m inclined to believe they’re not unaware their network is being games but rather don’t care because it’s still money in their pocket.

I don't think you're right @nemild, it's not a scary quote and it's also true. Sure, people can be duped temporarily but the best quality ideas last. Information is best distributed when it can be distributed freely. What alternative would you propose, censorship, regulation?

Facebook solved a problem (connecting) but it also created one (fake content, bot farms).

Either Facebook will fix it, or if they don't, new products will pop up to solve the problem to try to provide more reliable content.

Recently, might have seen it on here (I forget where) someone trained an AI to try to detect fake news. Sure the first attempts will be crappy prototypes, but the point is that people are trying to figure the problem out.

> Either Facebook will fix it, or if they don't, new products will pop up to solve the problem to try to provide more reliable content.

While this may be inevitable on a long timescale, the cost in human lives and dignity may be monstrous before this naturally occurs.

You’re downvoting and replying to me but your comment is directed at someone else.

> Either Facebook will fix it, or if they don't, new products will pop up to solve the problem to try to provide more reliable content.

Do you believe that “more reliable content” will have a financial edge over “more popular content”, and if so, why?

It's sophomoric at best even in 2008. The quote itself and the fact that the book itself is called the "Little Red Book". It's almost a straight up comic irony.

Spam and Sybil attacks were well known problems a decade earlier. It is basically what killed the vision of the semantic web in its infancy, several years before Facebook took off. It's certain everybody knew gaming data was a thing in 2008.

It might not be quite that simple - people can game the system of course - but it has been my experience that the Internet is generally very good at highlighting the best ideas and best content to the fore. The most upvoted comments and most upvoted posts are usually those which are thought out and reasonable, interesting, etc. In other words, of high quality. The best podcasts and the best music, along with the most entertaining YouTube personalities are generally those which get the most attention. At least, I have almost never "discovered" content which was really good without an already sizeable following.

> "When everyone has a printing press [i.e., Facebook], the ones with the best ideas are the ones people listen to."

Should be:

"When everyone has a printing press, the most persuasive ones are the ones people listen to"

That's a tautology!

A good tautology can be helpful in revealing errors in reasoning. Like here, the sentence "ones with the best ideas are the ones people listen to" sounds reasonable; the equivalent statement "best reasoning is the most persuasive" is clearly wrong.

“Stating tautologies is sometimes necessary in the presence of those who believe self-contradictions” - paraphrased from weird sun twitter

True.

And a good tautology like this one can at least help people realize the difficulties in distinguishing merit from success.

Which puts it up a step from overconfident generalizations.

Which is funny, because that's also a decent lay-explanation of how PageRank works.

The ideas that survive are the ones that reproduce themselves by being copied from one mind or other storage facility to another.

Where it all goes wrong is that it is very expensive to test an idea and determine whether it is good, whereas an idea that is very attractive to repeat and copy without testing is much fitter and the more it resists testing the better it is.

This is why the whole framework for protecting IP is broken as well - we incentivize coming up with ideas, but ideas are worthless - it's evaluating and testing ideas that is valuable.

I have seen the theory that the most optimally emotional ideas are the ones that go viral: https://m.youtube.com/watch?v=rE3j_RHkqJc

Emotion is adaptive, but only if the emotion is revulsion at the idea of skepticism.

It’s more like “When everyone has a printing press, the ideas people listen to are the ones that have been pushed relentlessly by a millions-strong AI generated bot army.”

Product success via pure advertising volume, is not an effective strategy. Maybe that oversimplification needs work.

The issue isn't who is the most persuasive. The problem is confirm bias and echo chambers. People aren't persuaded any more. Now they make emotional trigger snap assumptions and then only consume that which fits the narrative of the assumption.

I think almost everyone agrees that echo chambers at massive scale are really bad for society... and yet it's still as, if not more, prevalent as ever.

We're letting the world burn for the sake of online advertising revenue and broken business models.

"A lie can make its way around the world while the truth is still getting its shoes on."

In the 1850's it was widely believed that the invention of the telegraph would bring about world peace.

The problem is language itself. We trust it way too much to convey meaning, and truth. But it's just a delusion, or a mirage, in much the same way that the search for artificial intelligence is. In fact, I would say that language itself is the first AI invented.

Some of the ancient Greeks were aware of the limitations and drawbacks of language. Rumor is some of the Spartans took it so far as to refuse to read or write, and even speak as spartan (heh) as possible.

> Rumor is some of the Spartans took it so far as to refuse to read or write, and even speak as spartan (heh) as possible.

Do you have any source or pointer to learn more about this ?

Lookup “laconic”.

Not only some Spartans. Socrates did not write anything down (Plato did that for him), thinking that writing was a poor way at finding out about the world e.g.

https://newlearningonline.com/literacies/chapter-1/socrates-...

Is it your "thinking" Plato's ? Or Socartes? It's probably is better to QUESTION rather than form your own INTERPRETATION. At least a dialog is formed. Maybe the acients were more informed then we are.

Interestingly some people see novels and plays as vital tools for creating better decision makers. By entering into another persons imagination we can experience events and lessons that we would never have access to in real life or by observing real events in our communities. So - yes, Artificial Intelligence on the page.

The problem is that language is polymorphic.

I think in part Zuckerberg thinks of his own personal success this way. In his mind, he did well because He was Him, and right all the time.

A more humble view might include that he was raised in relative privilege, a lot of doors were open to him, and there was a lot of good luck involved.

He was also more willing than most to abuse the trust of his fellow students at Harvard.

I imagine it is only a small fraction of people who would be so brash as to ask fellow students for their email passwords, but with the benefit of hindsight, it seems clear that someone who asks for that information probably wants to do something nasty with it.

In the century that the printing press was invented, one of the most popular books to be printed was the Malleus Maleficarum, which systematized the practice of witch burning in the following centuries. Better dissemination of ideas does not necessarily mean better ideas. https://en.m.wikipedia.org/wiki/Malleus_Maleficarum

I think that's still true, it's just that the internet + social media let a lot of people become publishers at once, and we haven't yet decided to whom we should listen.

There are a lot of mistakes being made for which consequences have not yet occurred. Journalists of previous generations on whom we could rely had built up lifetimes-worth of reputation. On whom can you rely today? It's an open problem which I think is solvable in spite of today's difficulties.

That's nothing new. Journalists have always needed to identify fake source material. Dan Rather wrecked his career by reporting a story about George W. Bush based on fake documents.

It's interesting in the current world, prior in our "move into the world.. where anyone can make a video or photo of anyone doing anything" we're already struggling pretty badly with identifying fake news and what is considered fact vs fiction.

I don't see this as a move into a new world, but rather the increase level of hopelessness we're dealing with when it comes to good journalism. It may be time to pause and rethink our current approach to journalism since the current model is not working very well now and certainly won't be in the future.

This only happened because a team of professional journalists worked on the story for months, despite Facebook denials and stonewalling:

https://www.snopes.com/news/2019/10/11/pro-trump-outlet-link...

https://www.snopes.com/news/2019/11/12/bl-fake-profiles/

https://www.snopes.com/news/2019/12/13/facebook-bl-cib/

https://www.snopes.com/news/2019/12/20/facebook-removes-the-...

Arguably, in a long term, it might be better that the appearance of the person would not matter as much. It actually might be harder to establish a cult of personality and really bad dictatorships, if there are humorous fakes and parodies flying around. But it also might make it more difficult to keep track of real events.

Ironic that all journalists got the Iraq war wrong without the need for fake news.

We don't need real journalists, we need a way to pay for news that makes accurate reporting profitable.

Good journalism is still no match for a functioning US intelligence apparatus intent on disinformation.

Any competent government intelligence agency. Every government puts their own spin on information.

The cynical side of me: accurate reporting isn’t profitable because there is very little demand for accurate reporting from the general public.

People want to read/watch news they want to believe or agree with, and instinctively they shun away from facts that causes cognitive dissonance with them. What they want is to believe their preferred news sources are objective, but they will do everything they can to avoid doing actual due diligence to verify that objectivity.

That’s why if we leave it to purely market economy, completely objective, facts based journalism will never be as profitable as sensationalism based stuff you see these days.

But does it make sense to frame this as a statement about human beings, rather than a statement about what is logically required by the evolutionary-style process of information multiplying? Regardless of where society's starting point was, when it's really easy to transmit information, the information that gets transmitted the most is the kind that doesn't demand analysis first.

Even more meta, but one of the really big questions is: How to give people what they need instead of what they want?

I think it was pg who said "make something people want". If the plan is to get rich that is the way to go. If you want to improve the world, you should build something people need. Unfortunately, they often don't want it or don't want it enough to pay for it which makes the endeavor unsustainable.

Journalists were lied to and have very little means of independent investigation. The "fake news" here was actually the lie pandered by the White House. The fact that the House and Senate went a long with it is concerning but I imagine you only have to get the House and Senate intelligence committees onboard to get the rest of them to tag along.

> Journalists were lied to and have very little means of independent investigation

Plenty of foreign journalists didn't buy what the US government was saying and refuted its claims with facts, the whole idea that Saddam Hussein had nukes or WMD was completely preposterous, it's just that US journalists were afraid of being labelled traitors if they didn't go along with the Irak war. Journalists are people after all. So they "chose" to listen to Rumsfeld instead of Hans Blix.

Journalists aren't trusted anymore and their work isn't valued by the public because things like the Covington school incident happens, where Journalists just keep using tweets as a meaningful source of information, with no serious investigative work.

Journalists don't like Facebook, Google and co because they are direct competitors to them in terms of attention and ad revenue. Ironically they somehow like Twitter and are all on Twitter even when Twitter is actually competition to them, way more than Facebook or Google. Why bother visiting the nyt when all the drama is already on Twitter?

I don't understand why you're being downvoted. This was a fact during the Iraq invasion. Anyone who got the story right got fired. Everyone who parroted the administration line, even when it was blindingly obvious it was a lie, got promotions.

>Soon after the show's cancellation, an internal MSNBC memo was leaked to the press stating that Donahue should be fired because he opposed the imminent U.S. invasion of Iraq and that he would be a "difficult public face for NBC in a time of war"

https://en.m.wikipedia.org/wiki/Phil_Donahue#MSNBC_program

They absolutely got fake news. "Aluminum tubes" anyone?

Do I need to tell you what the fuck you can do with an aluminum tube!?!

How was the nytimes not “fake” during the lead up to the iraq war?

The entire build-up toward the Iraq War was based on fake news as was true for most of our wars.

That's an excellent point!

Verifying sources has always been important. I recently learned about Stanislov Petrov - he helped prevent a nuclear war in 1983 by ignoring a false alarm reported by Russia's early warning system [0]. The system reported that the USA launched up to six missiles. Fortunately, Russia's stance was that nuclear retaliation requires multiple sources to confirm an attack. Stanislov went against his orders, and declared the attack a false alarm.

As was true before the internet, we need to make sure we have reliable sources to trust the news.

[0] https://en.m.wikipedia.org/wiki/Stanislav_Petrov

> In a world where anyone can make a video or photo of anyone doing anything, we'll need to rely on reputable journalists to vet that material for us.

signed media is going to become FAR more important in the future.

We're going to have cameras shipping with crypto so that news agencies can sign their video.

Sounds like the spam wars from the 2000s again.

Tyrannical governments would love the idea of all camera images being signed.

The signing could very well be completely optional/pseudonymous. Don't fall into the trap of false dilemmas BigTech loves to impose on its users: the wonderful thing about technology is how malleable, and thus user-empowering it can be.

I had the thought a day or so ago that a system could be created to tie images to their respective cameras with a private key stored inside of a chip that self-destructs when you attempt to read the key from outside of it, along with a trust hierarchy of certs from various camera manufacturers. A little like hardware auth tokens mixed with PKI.

Does that sound like something that would be feasible to produce/practical in the real world?

Feasible? Sure, to the limits of what we can do with secure enclaves today.

Desirable? Well, you've created a system that authenticates a camera as being at a place at a time. It's a good way to authenticate photos, but a bad way to stay anonymous.

How do you feel about photographers and journalists becoming even larger targets for anyone who wants to keep a secret?

In the past the actual image authenticated that the camera was at a place at a time, and anonymity was preserved.

The issue is : does it authenticate that a particular camera that belongs to a particular person is/was at a place at a time and produced an image - because if so then if the device is found in a search the owner/user/keeper is in hot water.

So the camera must be anonymous - but it must be impossible for a lie about the place and time to be encoded into the image.

If you assume that "Anyone could have been using that camera!" is a defense, you're absolutely correct.

It's perhaps worth considering that an authoritarian government might not trouble itself with such legal niceties. All they have to do is mandate that journalists with cameras register their keys, make possession of a camera with an unregistered key criminal, and track sales / border entries. China springs to mind as a country that might do such a thing.

Again, you're absolutely correct in asserting that anonymity of camera-user is important! It's just perhaps worth considering how linking hardware to images might undermine that in dangerous ways.

something similar was done a number of years ago but cracked later. I think it was Nikon who had it and the proof of the crack was someone signed the Beatles crossing the road with a Nikon camera key.

Meanwhile secure enclaves are now possibly a lot safer, but as Kalium mentioned it might not be very attractive for everyone.

For forensics experts however it could become very useful I believe.

In most countries journalists double as officers of propaganda ministry. You only have to look how certain topics are being reported to get an idea.

Well, it's not like the professional journalists don't lie at all for profit and self-preservation. Lol.

Facebook does a lot of disageeable things but making everyone a journalist isn't one of them. This just decentralised journalism and actually empowered freespeech. There are issues with it, but that's a better problem we can work on to mitigate than giving a few channels a monopoly over truth.

Facebook provides a medium where most people can disseminate their message, with little restriction or cost, to a potentially large audience. It doesn't make everybody a journalist.

> Saw a video on YouTube of the President saying something bad? No idea if it's real anymore.

I've lived through enough presidencies to assure you that it never mattered if it was real or not. Supporters ignored or excused it, opposition treated it like the end of the world. Sane people stopped paying attention long ago to keep their sanity.

They need to be paid appropriately and must be absolutely unbiased, also.

Reported by Snopes for months now, not “discovered” by Facebook https://www.snopes.com/news/2019/10/11/pro-trump-outlet-link...

Would anybody expect the truth from FB? No way they wouldn't spin this to their best possible advantage.

I would expect the NYT to do some proper reporting though instead of copy/pasting Facebook’s press releases.

What leads you to expect that?

This isn’t any the bl but rather accounts that helped disseminate the bl’s stuff, which snopes doesn’t discuss

And so it appears that the inevitable has 'finally happened'.

Mark Zuckerberg's website is now a place not to be trusted as a source of news, finding friends and meeting others if there's a possibility of AI generated profiles being proliferated on the website.

I've heard that meeting real humans outdoors makes it less likely to be deceived. Their authenticity is vouched via word of mouth and experience by other humans, rather than robots on the internet.

Require manual profile validation for new Facebook accounts?

Probably would get downvoted if I said SSN + a credit check.

Phone number required + needs to be validated + can not be reused + can not be Twilio.

Prove your address? Prove your ISP cable bill? I don't know. Everything I guess, people here are going to tell me it's an invasion of privacy and they don't trust Facebook to do it.

But they'll let CreditKarma have every bit of financial information related to them possible.

People will apply for a credit card online (employer, address, social, income)... make a universal login process for that.

Maybe like blue Twitter verified checkmarks next to all profile names. "Verified human"

The problem is that a lot of people can't even fathom the idea of automating something like account generation. It takes them two support calls and an 1 1/2 hour to log into their email. They don't even understand what's possible, and a blue checkmark won't/doesn't solve that.

Seems like an incredible amount of work for something that at the end of the day doesn't actually provide much value to users or the world.

or simply charge a few bucks for account creation.

$4 Payable in Zuckbucks, which can only be bought in $5 packs

You then run the risk of excluding developing economies. Unless you adjust the payment to be region specific, in which case you’d just get foreign powers (be that corporations or state sponsored) using their wealth to fund hordes of new registrations from said developing economies; thus negating the benefit of a sign up fee.

a low single-digit dollar amount of money isn't really prohibitive any more in almost all regions on the planet for a one-off payment (networked regions, at least). That's an argument facebook itself likes to make but if you look at the WhatsApp growth which used to have a model of 1$/year it seems completely fabricated.

And arguably a small payment amplifies anti-abuse measures because you have a hard financial trail. If a foreign entity buys thousands or more of accounts, not only are you putting a cost on it, it should be trivial to see the anomaly in the system.

and a single digit isn't really prohibitive for making fake accounts either. I'd gladly pay $250 for 50 fake fbook accounts spewing the commercial and propaganda crap that makes me profit.. or just feel good about skewering another side I don't care for.

As someone who grew up in a developing economy, I find this view extremely patronizing. Usually, what happens is that a local company fills that demand gap.

Also, I think it is good for all parties concerned, because most developing economies will have a big problem becoming developed economies if their people are under constant surveillance from entities who may benefit from their continuation as developing economies.

Case in point: the US dollar is a horrible currency if looked at in isolation - after all, why the heck should the rest of the world pay American kids lots of money to have a great time at college and then turn around and default on their debt? Student loan forgiveness is almost a certainty over the next few years. But people still flock to the US dollar because it is far better than the other places money could go. There may be a hint of conspiracy theory in this, for sure, but it does not require a wild stretch of imagination to expect that the country where Facebook/WhatsApp/Instagram is headquartered actually has a real incentive to foment violence in other countries and make their own economy and thus currency look better in comparison.

> As someone who grew up in a developing economy, I find this view extremely patronizing. Usually, what happens is that a local company fills that demand gap.

...which means if a western subscription services wants to compete with a cheaper local equivalent, they need to charge a comparable local rate. That’s not patronising, this literally how all businesses work when having to compete with regional price differences.

As for the rest of your post: it sounds like you have a proverbial axe to grind. I’d be the first to agree that there are some serious issues with some of America’s domestic and foreign policies but none of your points are even remotely relevant to the conversation (and some weren’t even remotely based on reality)

>>...which means if a western subscription services wants to compete with a cheaper local equivalent

Interesting comparison. Let us discuss that a little further.

https://www.nytimes.com/2018/10/15/technology/myanmar-facebo...

So Facebook provided free internet to Myanmar, and it was better than the "cheaper local equivalent" because it was actually free. Soon all the ISPs in Myanmar were out of business. Facebook was now synonymous with the internet in Myanmar, and suddenly it became a lot easier to influence people into doing pogroms. Now clearly a "western subscription service" successfully competed, and unfortunately, the pogrom did not affect the "western subscription service" in any way while millions of real humans became refugees.

Generally, people's answer to this is something like "yeah, it is too bad the people were such idiots to be provoked into such violence" or in some way blaming the naivete of the local population. But imagine if Facebook was actually asked to pay compensation for every affected person, or to stop business operations in Myanmar altogether. See, you can actually do that with a local company because they have some real skin in the game.

To complete my point, I don't actually give a damn if a "western subscription service" can compete for my business unless I have a way to drag Zuckerberg (figuratively speaking of course, a minion of Zuckerberg will do) to my country for trial and then send him to the local prison if found guilty of some local law. Think of it as "terms of providing service".

>>it sounds like you have a proverbial axe to grind

Indeed I do. It is called "stop patronizing the people of my country".

>>none of your points are even remotely relevant to the conversation

If you feel so, that's OK, although I would obviously like to see a more specific refutation.

Facebook is an edge case because that’s a problem in every country and frankly that whole “free internet” mission was clearly a bullshit move from the outset (even people in the west weren’t the slightest bit convinced by Zuckerburgs motives). However that still doesn’t prove your claim that every western company exists in developing counties specifically to further your economic strife (which is the accusation you’re levelling against the west).

If you want to argue that American corporate culture is a toxic breed of short-sighted greed or that developing economies are better served with local businesses then at least try to do so without all the conspiracy theories and personal attacks. It’s not exactly a hard topic to argue rationally after all.

> Indeed I do. It is called "stop patronizing the people of my country".

It’s not “patronising“ to say that companies need to offer regional rates to compete in regional economies.

You might argue that its unfair to put local companies out of business or that western corporations deserve no claim in developing nations; but that’s categorically not what “patronising” means (and also a very different point to the one I was making).

Don’t just take my word for it though, look the word up in any of the numerous online dictionaries :)

>> none of your points are even remotely relevant to the conversation.

> If you feel so, that's OK, although I would obviously like to see a more specific refutation.

I wasn’t trying to refute your claims (mainly because most of them were far fetched - even by your own admission). I was saying they’re a complete tangent to the topic of conversation.

Look, if you want to bleat on about the evils of western influence on developing economies then fine. But at least acknowledge it’s a largely unrelated rant to the conversation that precedes it.....and also don’t make accusations about other HN posters when you don’t understand the terms you’re accusing them of.

>>Don’t just take my word for it though, look the word up in any of the numerous online dictionaries :)

>>You then run the risk of excluding developing economies

"Patronizing: apparently kind or helpful but betraying a feeling of superiority; condescending"

I see. So, let us back up a little to your first statement. Why is the risk of excluding developing economies a problem (for the developing economy), again? I will be very happy to take back my assumption about what you implied there, provided you can give me an explanation for why it is a "risk" and who actually bears the risk.

>>Facebook is an edge case because that’s a problem in every country

It is not an edge case if it leads to making assumptions which are clearly bad for our future. Because Facebook isn't getting any real punishment, people are only going to say "Nothing happened to Facebook. Why should the other companies worry about lying to governments/ lying to customers/ privacy issues/ data collection / retargeting / lookalike audience" etc.

>>However that still doesn’t prove your claim that every western company exists in developing counties specifically to further your economic strife (which is the accusation you’re levelling against the west)

I am not sure why you feel that way. Specifically, what I am saying is that I don't like people starting off with statements such as "you then run the risk of excluding developing economies" as if that is automatically a bad thing for the developing economies.

> I see. So, let us back up a little to your first statement. Why is the risk of excluding developing economies a problem (for the developing economy), again?

This is the question you should have opened with :)

I never said it was a problem for the developing country. I was talking purely about the economics of Facebook. If they (Facebook) want to expend their demographic then they would need be affordable in each economic environment. Whether you (the respective county) want them there or not is another matter entirely.

Frankly I’d be happier if Facebook failed in my own country rather than expanded their global presence into yours as well; but that wasn’t the topic being discussed at that time.

> It is not an edge case if it leads to making assumptions which are clearly bad for our future.

Technically it still would be when using the correct definition of edge case ;)

> Because Facebook isn't getting any real punishment, people are only going to say "Nothing happened to Facebook. Why should the other companies worry about lying to governments/ lying to customers/ privacy issues/ data collection / retargeting / lookalike audience" etc.

I agree that’s a problem but, once again, you’re arguing a completely different topic.

> I am not sure why you feel that way.

Frankly put: because I’m not biased. You feel like you’ve had an injustice so you’re lashing out at everyone. I totally get that. I’ve been really privileged to grow up in the U.K. but I’ve also spent a lot of time in developing countries so I’ve seen both sides of the argument.

> Specifically, what I am saying is that I don't like people starting off with statements such as "you then run the risk of excluding developing economies" as if that is automatically a bad thing for the developing economies.

That literally wasn’t what was said though. The context was never about what was good or bad for those counties; economically, socially, not by any other metric. The context of my original post and the entire chain of conversation that preceded it was a theoretic one about how Facebook might counter abuse on its platform. The context was always about Facebook specifically. This is why I have repeatedly said you’ve gone waaaaaay off topic.

If you want to change the conversation to the evils of western corporations then please bare in mind that HN is ostensibly a tech and business forum so people will often talk about things from the perspective of tech giants even if they don’t actually support the operations of those corporations. Thus it would benefit everyone if you didn’t jump to conclusions, making personal accusations about the opinions of HN posters on entirely different subject matters just because they happen to make an impartial point on a company you have a personal vendetta against.

When it comes to account creation, whatever method you come up with will be solved with hordes of call centre workers in a lesser affluent county.

The only solution is to police the content and that then brings up questions about whether Facebook are an “ISP”, how start ups then follow precedence, etc.

I would love an online forum with aggressive verification. It would really weed out the noise.

By now, people have the gaming of Real Name policies and Verified Accounts down to a science. Indeed it's harder to use false accounts anymore, that much is true.

However, it becomes much easier to Dox people. This allows you to put pressure on work+family+neighbors. Or alternately, if you're one of the more violent groups: you can outright go after your opponents in physical space.

This may be important and may be not. Numerous governments already do that: require everyone to disclose their [e-]life and punish for what they had to say. But if you do that in US or EU, will they turn into such oppressing states? Is total control a cause or an effect, or both?

To make the problem trickier, even if you do have a "Verified human" today, they are likely to download virus.exe tomorrow and become part of the next bot net

All of this is why I think Nextdoor is extremely well positioned to eat Facebook's lunch and drink their milkshake.

Which would be a pretty glorious downfall.

So more big brother stuff is what you are suggesting. How about people not rely on FB for news.

Easy to say, but people will still do it. And those people vote and affect policies that, in turn, affect you and me.

Eh, Wikipedia learned to deal with it too (and sockpuppets and meatpuppets, and more).

I'm sure Facebook can work it out too. Though they have some very game-able elements.

The problem is it's not directly in their interest.

Fake accounts are just as billable for boosting reach as real accounts. It will only be when companies stop advertising that they will be forced fix it. In the meantime they take the money and run. (or invest into business streams with actual long term value)

> a place not to be trusted as a source of news, finding friends and meeting others

Sounds like Usenet in the 90’s. The more things change...

Eternal September? :-/

>AI-fueled eternal September

We will see in our lifetimes how droves of spureous personas flood the internet forever

I dont "friend" anyone I dont know IRL and even then I verify that I got the real them via other channels, usually by phone or in person.

I don't think social media should ever be trusted as a source of news (by itself) – and frankly, I don't know many people who have seen Facebook as such.

obviously someone must use FB for their news because that's all I read about in the news now.

Wait, that sounds circular. Bottom line is profit overrides truth most times now.

I do know people.

Well real people can be fake as well.

The cynic in me thinks Facebook let this slide to hide the anemic "growth" of their userbase and activity. Bot accounts will still click on ads, like pages, and fake interactions that generate revenue for Facebook.

Aka the usual problem these social media services have; bad behaviour from bots and users likely looks great on the stats spreadsheet, even if it's at the expense of the quality of the site/community.

Why take them down? That's someone's free speech right? This is what we're told about false information and political ads with outright lies in them, aren't we? Why would this be any different?

Rather predictable that the "line" for taking something down suddenly appears when you're lying to Facebook. Lie to other users and it's fine... as long as you are a real commodity, ahem, I mean "person."

Cease to lie in a way that allows Facebook to make money (not much point serving ads to bots, is there?), and all of the sudden you're taken down. Funny that.

Facebook is a private enterprise. Free speech doesn't apply. Compliance with government/social pressure is what applies (for better or worse).

That's the thing that so many people don't get.. Facebook is not a public space. You have no inherent rights there, and they can do whatever they want with your acccount/content/etc (or at least whatever their TOS says)..

No. The First Amendment doesn’t apply. The concept of free speech can be applied anywhere.

That's extremely pedantic and doesn't really add to the discussion.

My point was that these private companies police their content based on the impact it has on their bottom line and any laws they have to respect, and that's it.

They don't do it to protect any kind of speech, groups or individuals, and no one has any rights (related to speech - yes AODA and other such legal rights apply), in the civic sense, when using Facebook or other social media platforms.

Because it breaks ad targeting, or to be more precise makes ads obsolete because you're promoting products to someone who isn't who they say they are. Advertising is the essence of Facebook's existence. And they'll go a long way to ensure it works as a clock.

“Discovers”

People have been talking about this group for months and Facebook hemmed and hawed and delayed and delayed

lol, I came here to put that word in quotes.

it's actually hard to overstate how significant the effort of the epoch times is. they gathered billions of views on facebook, more than any other news organisation.

"In April, at the height of its ad spending, videos from the Epoch Media Group, which includes The Epoch Times and digital video outlet New Tang Dynasty, or NTD, combined for around 3 billion views on Facebook, YouTube and Twitter, ranking 11th among all video creators across platforms and outranking every other traditional news publisher, according to data from the social media analytics company Tubular."

https://www.nbcnews.com/tech/tech-news/trump-qanon-impending...

but were those views REAL?

People have also been talking about every other group. The sensitivity of "people talking" may be 100% but the specificity is near zero.

http://archive.md/abR53

FYI, I actually just published a guide on how you can help ensure that ("deepfake") tools that you make are less likely to end up misused like this: https://www.technologyreview.com/s/614912/ethical-deepfake-t ...

(Also at /newest; 3rd page now, and the other submission. Ideally those topics at least would be merged?)

Summary of mitigations:

# Limiting users or use cases

- User Vetting: Carefully vet those who can use a tool (Example: Synthesia only working with vetted enterprise clients.)

- Usage Constraints: Limit the synthesis possibilities. (Example: Humen is providing only a limited set of movements for generated videos.)

# Discouraging malicious use of flexible and public tools

- Clear disclosure: Ensuring and requiring that manipulated outputs are clearly labeled (including via metadata).

- Consent protection: Requiring the consent of those being impersonated.

- Detection friendliness: Supporting those working on detection and not seeking to fool detection systems.

- Hidden watermarks: Embedding data through robust watermarks.

- Usage logs: Storing information about usage and media outputs.

- Use restrictions: Enforcing contracts that prohibit forms of violative or malicious use.

Not all apply to every system of course. The article has more detail!

Good article, but for the sake of argument: Don't you think any technical "solutions" to this issue will also eliminate the possibility to use these tools for ethical goals (let's say the roles were reversed or whatever) – in addition to all the other gray-area, and inevitable arms-race of detection/avoidance?

Definitely.

I mean I think e.g. you could have incredible art projects and experiences created out e.g. of giant undetectable fake crowds. There are real tradeoffs in terms of what we value or need in a society.

I think technical limitations like this are the worst possible solution. The genie can't be put back in the bottle with any particular technology, and the lack of widespread access and awareness means the damage from any particular deepfake that _does_ get released can be that much higher. There is zero chance this tech won't be in the hands of every government-sponsored group on the planet, not to mention all the other independent bad guys out there online.

I just leave this here https://www.thispersondoesnotexist.com/ Probably the source of these images.

This other article linked here, discusses it:

https://news.ycombinator.com/item?id=21847583

Interesting that the eyes on every TPDNE image have a specific position that could (for now) be used to detect them.

I have said for a while now that the future is their AI vs our AI and nobody can turn the AI off because then we'd lose.

For many use cases it would be great to have a real/verified oauth service. Dutch government does something like that with digid. Imagine how much fraud could be prevented if there was something like that in the US.

Estonia's long been ahead of the game here.

https://en.wikipedia.org/wiki/Estonian_identity_card#Cryptog...

Why doesn't Facebook have an identity verification process? They literally allow anyone to sign up as any person. There is a show on MTV called Catfish that is dedicated to people who create fake profiles to trick people into relationships.

Because they're incentivized to show continuously growing subscriber numbers?

Recently I played around with the newest iPhone filters. The pictures had weird shadows and distorted backgrounds.

Are distorted backgrounds really an indicator for fake profiles? Smartphones do all kind of weird stuff to "real" images.

GANs tend to leave a very distinctive kind of artifact which is hard to describe but I think anyone who goes through a lot of GAN imagery knows what I am talking about. It's a particular kind of distortion which is a liquidy psychedelic weirdness, that you don't really see in other media fakery.

If you are interested, media forensics is an entire field about this sort of thing.

This gives me flights of fancy. Imagine (something like) FB (re-)built as a fake AI-generated company (with fake AI-generated users). It would feel like a circle being closed, something reaching its ultimate consequence.

People are faking their profile pictures for sometime using filters or AI cameras, bamby eyes. Already a digital ego fairytale now deepfakers would open new positions as ’ghost busters’

You better believe black hat SEOs are using AI for spam like this.

> Facebook Discovers Fake AI-Generated Profiles

I think the profiles are human generated. I see no evidence otherwise.

If they have a GAN why did they repeat the photo?

I'd think they got the photo from some open GAN project.

[edit] Researchers actual say they might be from thispersondoesnotexist.com [3.14] But I guess that spoils a good HN story.

[3.14] https://graphika.com/uploads/Graphika%20Report%20-%20Operati...

It makes me reconsider the early RealID efforts in social media. Perhaps perhaps.

Seriously is it what nytimes is now?

https://gfycat.com/maturefarflunghagfish

Cool. Now it needs to work on algorithms which detect whether a journalist is working for the CIA.

That'd be progress.

Ironic, since Facebook itself uses a sort of AI to create fake shadow profiles.

Source?

"Discovers"

Keep calm. It is false positive. It was only Facebook customer service bots

"Discovers", yeah sure.

I look forward to this block of fake accounts registering to vote and voting by mail. Will any accounts go rogue and defy their creators? How does a candidate appeal to this influential group?

There's been fake generated profiles on Facebook for over a decade. I know this because I had some, I was targeted by some, and I had "security friends" who generated some. Over the years I've seen several campaigns of groups of fake profiles trying to collect access (by "friending into" friend circles).

At university my security club had a member who gave a presentation on algorithms for detecting and generating fake profile posts for Facebook, through which he created sock puppets and also found a ton of others belonging to "who knows which group". That was around 10 years ago.

A fascinating thought on this for me is the basic American belief that "bigger is better". What I mean here is that the companies generating these fake ids will generate lots of the fake ids, under the belief that the more they have the more influence they have. At some point, the fake ids will reach a tipping point and outnumber the real ids thus making Facebook itself irrelevant. Sort of a "DOS" using fake ids, in this case a denial of relevance. There are some interesting paths that this could take, but I don't see any incentive in Facebook really doing much about this because their power actually lies in also "Bigger is better". So, generation of fake ids could be a blessing by overloading Facebook and making it irrelevant.

The most fascinating thing to watch is the democratization of propaganda. Now you no longer need the printing press. You can speak with a million voices without being a Getty or Rockefeller.

The NYT no longer has a monopoly on convincing lies.

These are interesting times. I think we're close to the emergence of trustworthy celebrity journalists.

I'm not sure why people are downvoting this

Scuttlebutt is Randolph Hearst bragged about getting the US into the Spanish American War.

This is just using GANs to generate fake profiles. It does scale better than making up fake people by hand, but to what end? I'm not sure if there's a direct relationship between the ease of creating fake accounts and their cultural influence. And it seems like facebook doesn't have trouble detecting them.

Convince me that this isn't just "add AI to the headline" clickbait

Reply on news.ycombinator.com