Hacker News Reader

Comment by everdrive | Hacker News Reader

People already believe garbage at a pretty alarming rate. It's easy to guess at a number of possible outcomes here:

- More junk text moves the public to doubt legitimate information even further than they currently do.

- There is so much human-generated junk text that adding more of it via AI actually doesn't have much of an effect.

- People return to lean on experts, perhaps even more than before. (just as a number of tech-literate folks have now returned to relying on brand name.)

Speculation is easy of course, so who knows what will actually happen.

View on HN

True. But a simple API to generate junk text? It can scale, cheaply, beyond measure.

No need for a troll farm, hiring, managing and training tens or hundreds of people.

A reasonable amount of cash, a bit of motivation, some moderate technical skills, and voilà! Anyone can compete with the Russian troll farms now and build their own networks of hundreds or hundreds of thousands sufficiently credible (as humans) fake accounts spewing garbage and patting each other on the back via likes, retweets and whatnots.

All with the appropriate fake news blogs and sites happily churning out grammatically correct nonsense that makes (enough) sense.

Basically, this kid’s dream: https://www.nbcnews.com/news/world/fake-news-how-partying-ma...

But even Russian/Chinese bots can step up as it is much easier now to flood forums (like reddit) etc, where e.g. a China critic article appears to kill any discussion.

Oh, definitely. Existing operations also absolutely can leverage that in order to amplify their reach and capabilities.

I find it easier to identify humans that flood forums though. Especially non-native speakers usually are somewhat easy to spot, I assume that's true in any language. That's different for ML-generated texts. On the other hand, human texts are more "on message", but if all you want to do is create noise, I guess you don't need to have targeted communications.

> if all you want to do is create noise, I guess you don't need to have targeted communications.

This is key in anti-extremist operations on anonymous boards. 4chan and other similar sites are absolutely nothing like they were a decade ago, I presume because of such bots flooding them with noise.

And unfortunatley, the web will be forced to move toward verified human identities to fight with such junk and anonymous browsing will become a thing of the history.

That wouldn't work. You can already rent humans and sim cards to do captchas and phone verification. There's nothing stopping someone from verifying their bots.

It might be neither easy nor impossible. You could imagine s reputation system that depended on what others thought of you. Start the network with just people the developers know personally. Admit new people when enough existing members couch for them. Sometimes fake cancers will appear, but they'll sometimes get sniffed out, and the people who let them in punished somehow (temporarily blocked? Delete photos at random? Idk) strictly enough that people generally don't want to let in bad actors.

I think we may come to see the era of roughly 1990-2010 as the golden age of information: relative abundance creating new opportunity, before the noise drowned it all out.

I suspect that in the future people will, ironically, return more strictly to tribal knowledge, as the media and the internet will be (already is) a vast ocean from which you can pull anything you want to believe. Thus nothing you see or hear from mass media or the internet can be trusted, there are no experts, and you go back to information scarcity as you have to rely on your immediate human network for trust. Actually I think we’re already seeing the return to tribal authority, the early waves are already here on Facebook and YouTube... they just haven’t devolved to strictly local circles of trust yet.

neal stephenson has a cool prediction on how to handle this in one of his latest books: curators. people of means will have a curated view of the internet and information that costs $$$

It’s already here. How many people are paying for newsletter subscriptions etc, and how quickly is that growing? :)

In many ways, predicting the future is just about looking at how to apply past trends to our modern world. People and organizations with the resources to do so are actively working to cram their "curation" into the public sphere, to try and sway the masses. That's advertising. It's the same thing it was in the 20th century, just more personal and using more sinister means of hogging your attention span. Peoples' intuition about "advertising" is as a commercial enterprise, but I'm certain all the textbooks that talk about the 2010s and 2020s will mention the role of mass bot/shill campaigns to sway the public by governments, companies, organizations, individual people.

There is a market (well, a need at least) for nonsense detectors that work similarly to the way ad blockers work. Detect internal inconsistencies, non-sequiturs, low information density and other similar reasons to avoid reading the text -- and visibly flag or block that.

That should eliminate 80%+ of existing human generated text content and lead to text generators composing useful articles.

> just as a number of tech-literate folks have now returned to relying on brand name

out of curiosity, what are you referring to?

In the early-ish days of the consumer internet, consumers had a new and huge information advantage over companies. People moved from relying on brand name, to reading online reviews. Often finding niche brands which they had otherwise not heard of.

Now, in 2021, that experience is flipped on its head. Amazon reviews are gamed and cannot be trusted. Companies build niche brands like fly-by-night companies, and the lesser known brands have a very high chance of being both seriously inferior, and also short lived.

At least, this has been my experience, and the experience of some others.

[edit]

And as further anecdotal proof that things have come full circle, my elderly mother in law keeps getting tricked by Amazon purchases. "The reviews were good," she'll say before returning something.

I just rely on reviewers like NYT Wirecutter and then buy whatever the reviewers suggest (and is cheap) on Amazon.

Why do you imagine the NYT isn't taking a bigger paycheck for saying the same thing the fake Amazon reviews are saying? We live in a world where brands are trash. They're built by the lowest bidder with the absolute cheapest parts ("the Toyota Way", heavy airquotes), and packaged with lies (marketing). Where did this idea come from that it's only the foreign resellers of Chinese products that are posting fake Amazon reviews, when it is absolutely the same case with household brand name companies? Go to Fakespot and look up random brand name companies you would normally "trust" and see what how Fakespot rates the company. We are WAY past the point where the vast majority of companies (governments, organizations, etc) aren't using these types of mass propaganda campaigns. A brand name in the general case is no more trustworthy than some imitator.

I am not inspired to trust Fakespot given their privacy policy.

"When you use our Services, we automatically collect the following information about you (collectively referred to as “Personal Information”):

    Your device information which includes, but is not limited to, information about your web browser, IP address, time zone, and some of the cookies that are installed on your device.
    Individual web pages or products that you view, what websites or search terms referred you to the Service, and information about how you interact with the Service.
    Your first and last name
    Your email address
    Your username associated with your Apple ID or Google Account  
    Your account information such as your account name, account password, other credentials, security questions, and confirmation codes"

That's fine, but it doesn't answer the question of why you would trust a NYT product "review": https://www.nytimes.com/privacy/privacy-policy

NYT's entire value proposition is that you can trust them. Amazon's is (mostly) that they'll get you what you asked for, fast and cheap. A bad Amazon review seems to redound, for whatever reason, mostly to the discredit of the particular seller, not Amazon. NYT therefore has more incentive to give you accurate reviews.

> People return to lean on experts

The problem with this is that people look at anybody confirming their bias as an expert. I can't tell you how many FB posts I've seen where some armchair poster claims that a researcher is wrong because of xyz and it's being reposted thousands of times.

People believe what's believable (even if backed up by garbage). GPTs dont make believable stuff, but they can be used to flower up some b.s. idea. Nothing that can't be done with a few hired trolls, and the proliferation of garbage will endanger the troll industry, as people will start becoming suspicious. So i doubt its impact can go beyond generating spam and noise.

Concrete prediction: There will be a global cult similar in nature to Qanon driven by an AI spitting out generated bullshit within the next ten years.

That's assuming some percentage of Qanon word salad isn't the output of Markov chain generators. A lot of it resembles low-order statistical text generator output after having been trained on a corpus of 1990s Usenet alt.conspiracy and the Protocols of the Elders of Zion.

Reply on news.ycombinator.com