Hacker News Reader

Comment by swayvil | Hacker News Reader

This reminds me of the legend of the Tower of Babel. God cursed humanity, rendering them unable to understand each other.

Except in this case it's only "anonymous humanity" that's screwed. And it isn't that we can't understand but that we have lost all trust.

Is it a human with something to say or is it a robot playing a game?

View on HN

People still think about it in quaint ways.

Spammers are a minir annoyance. I have one word for you: swarms.

Swarms of sleeper accounts can be deployed with existing technology. They can astroturf revolutions, rile people up against others, make reputational attacks at scale — and perhaps most destructively, they can overwhelm anyone with a different viewpoint, with 10-1 content ratio (eg I could deploy 20 bots on HN to downvote and argue with the people who disagree with me ad nauseum).

As you can imagine, forums could use “grandfathered” accounts to sort of mitigate this. But when regular people start preferring bots, it is game over.

You think it is hard for bots to endear themselves over humans? Karma / likes / upvotes is actually a measurable and gameable metric, and the bots can excel at it better than humans. Not to mention if you own a dating site you have access to all the messages that have ever worked to get a date or get a laugh. HN anyone can analyze the type of messages that got the most upvotes.

We keep analyzing one bot among 8 humans at a poker table. No one seriously considers the exponential jump to 1000 bots surrounding every human.

At best - we are building a zoo for ourselves. But we can’t stop!

What are your thoughts on requiring payment to participate as a way to reduce spam? I realize that in this particular context payment could prevent participation for some, however, are there other reasons why payment would be ineffective?

That was always required for sybil attacks. But if the long-term benefit exceeds the cost of membership, it will be paid. Think of a conqueror renting an army for a small bit of time to go pillage and gain very long term control of a village.

I am in a good position to speak about this since I designed decentralized social platforms like https://qbix.com and smart contract platforms like https://intercoin.app that build coins for communities.

If anyone can make an account, your community has unlimited cost dealing with it. It could be a sleeper account acting like a human, until one day the cost starts getting imposed in the form of, say, coordinated swarm vandalism:

https://en.m.wikipedia.org/wiki/Wikipedia:Wikiality_and_Othe...

But it could be a LOT more subtle and plant the seeds to move opinion of a forum gradually towards anything you want, including simply accepting and upvoting articles from your domains where your bots create content.

If you just mint a coin and give it out for basic tasks, people will take advantage of that by creating many accounts. You have to start asking people to appear in person to get a certificate at least — which is what Sam Altman’s Worldcoin is about. But you don’t need to go that far, every community can just have in-person get togethers once in a while to say hi to each other (“shocker!”)

I foresee a lot of people starting to retreat to gated communities. The thing is that anyone can invite a bot in. If you give everyone N invites, and the people they invite N invites, they’ll eventually use them tk invite N to the power X people. And those people might be real, but eventually may run a bot once in a while. You may think that’s not likely but actually it has already has been happening, people loove to run automation on their own behalf, and have no idea what it does, exposing themselves to compromises and attacks at scale:

  packages in package managers
  NYC landlords pricing apps
  wall street trading bots

I wrote about it here: https://magarshak.com/blog/?p=385

But as I say, the real trouble begins when people start to prefer bots. They choose convenience over security and truth. Bots won’t have resistance to an agenda and a lie like humans will.

I guess our only hope is to dilute the bot swarms with opposing bot swarms, like Bitcoin miners competing kind of. But at that point the entire internet will be a DARK FOREST.

The first reason that jumps to mind is that cost is not a factor for everyone. For almost any given price level, there are actors for whom the cost is still worth it.

Some examples and their given cutoff point, at which the payments are no longer worth it:

Lower cutoff:

- Marketers selling a product -> when customer acquisition is more than the product margin, or investors decide the growth isn't worth it

Medium cutoff:

- Private organizations pushing an ideological viewpoint -> when the private money runs out, higher tolerance here because profit isn't the point, and actors are more likely to consider pushing the ideological viewpoint as worth spending money without direct profit return

High cutoff:

- State actors working against either other states or to squash internal resistance/political opponents -> virtually unlimited depending on how rich the country is

Requiring payment to prevent spam makes sense in narrow use cases, like Steam's $100 fee per game preventing the worst of the flood of cheap garbage. However, I don't think there is any price level that can "clean" general human communication online.

Now, our vision with Intercoin is that communities require payment in their own currency and require badges earned by attending their events.

https://intercoin.org/communities.pdf

Much harder to game at scale

This seems like it could work in some way, but it's important to recognize that spam, and LLM usage already require payment in some form or another. There's a cost to sending out millions of emails just like there will be a cost for generating millions of LLM generated posts. So with that framing, it's obvious that spammers, using AI or otherwise, are willing to expend capital to produce spam. There may be a threshold of cost they won't be willing to exceed, but there's no principle here which can be applied universally.

Sam Kriss has a pretty wild post about this, with a running metaphor of LLMs as golems: https://samkriss.substack.com/p/the-cacophony

Thanks. I'll read that.

I was thinking demons mself. Being, above all, liars.

A quadrillion synthetic liars. Breeding and speaking at the speed of light.

Reply on news.ycombinator.com