Hacker News Reader

Comment by carlgreene | Hacker News Reader

I have largely written Reddit off and no longer visit it after an experiment I did where I had an agent karma farm for me and do some covert advertising. As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer. Many many people (or other bots) had full on conversations with it and it scared me a bit.

I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.

Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.

View on HN

For a while there were a lot of posts from people experimenting with ChatGPT to write anger bait posts on Reddit where they would later edit the post to say it was fake, written by ChatGPT.

I assume they thought they'd be teaching people a lesson by making them feel foolish for responding to AI stories, most of which were too fake to be believable.

However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake. In advice subreddits, commenters continue to give advice on the situation. Some comments would say they saw the notice that it was fake but continue arguing about it anyway.

This makes a feature of Reddit very clear: The truthiness of a post doesn't matter. The active commenter base on popular subreddits just wants something to discuss and, usually, be angry about.

In retrospect it's obvious given that misinfo posts were the easiest way to karma farm for years even before AI.

We do precisely the same thing here. Here's a relatively recent post that, to me, seems obviously LLM-written. It just rattles off some management platitudes:

https://news.ycombinator.com/item?id=47913650

It had 639 comments and 866 upvotes. And that's not a one-off.

I wish there was an internet-wide "don't show again" button for such slop pages

You can ban a domain from search results with Kagi

You you can bend a handful with duck duck go as well. However, it's only a handful and you run out and you're stuck.

Yeah, the trick is to do your own curation and go from there.

If you like some authors or journalists or bloggers, go see who they read (trust me they all say who they follow in their own niches) and build from there. You can develop quite a good RSS feed following this method in like an hours tops.

Sufficiently advanced "AI" is indistinguishable from a linkedin true believer koolaid drinker middle management type.

Even the title is in "x, not y" format.

>However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake

That's 90% of current Facebook pages and groups.

The decline of Facebook is sad. I liked it early on. I used it primarily to follow family and casual friends from high school. When they posted, it would show up on my feed, I read all the posts, and that was that.

After awhile I had to wade through all sorts of nonsense to get to the posts I actually wanted to see, and even later Facebook stopped putting posts from people I follow in my feed. It was 100% garbage. I can't imagine why anyone uses Facebook for anything other than the marketplace.

I primarily use it for browsing memes now, and occasionally interaction with friends.

It's sad, but car stuff (new aftermarket stuff) is now mainly on facebook for my car.. That, and messenger to chat with siblings is about it..

I often hear that about Facebook, but at least it has a "feeds" button that you can press to get the sources you actually subscribe to. The default "home" feed is useless.

Facebook is fine if you join groups based on your interests (hobbies etc) and then aggressively unfollow/block anything you don't want to see. It's not really conducive to discussions like Reddit, though. Mostly drive-by comments.

This. But damn it’s effing hard work!

> then aggressively unfollow/block anything you don't want to see

That is hard work. I have a few friends in the trans world and occasionally interact with relevant groups on FB. The attention algorithm thinks that this means I might want to see random posts from pricks who literally want to see people like my friends herded up into concentration camps. Most of it is far less extreme than that, but the system is definitely optimised in favour of rage-bait because that ticks up the engagement metrics.

>and then aggressively unfollow/block anything you don't want to see

It keeps showing shit I didn't ask for and don't follow anyway

Even without AI slop I've noticed this happen on Reddit.

I once made a rather boisterously-argued comment on a political issue I'm passionate about, and I realised that I'd made a serious error of reading comprehension when it came to my opponent's argument. I apologised to them for being an abrasive arse over my own mistake, then edited my comment to say that I was mistaken.

My incorrect comment which literally said at the bottom it was incorrect continued to be upvoted while my opponent who had made the stronger argument continued to be downvoted.

I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust. Or rather turn them into little better than comment sections on news sites; thriving but worthless.

I'm active in a number of online communities that are doing just fine but the difference is those all involve ongoing relationships, built over time and with engagement across multiple platforms. I've no doubt this clock is ticking too but it's still harder to fake a user across a mix of text chat, voice and video calls, playing an online game, etc and when much of the web of relationships extends back into real life activity.

But I agree the golden age of easy anonymous connections online has ended.

"I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust."

Those sorts of places were always the only places with reliably good communities.

To the contrary, platforms like Facebook and X demonstrate that even personal verification won't save you from identity politics.

What is identity politics, is that age verification?

Identity politics have nothing to do with your actual identification documents. Think: Black Americans being treated as a homogeneous voting bloc, or that all Hispanic voters would be pro-immigration, or "the Evangelical vote".

People will post appalling racism in newspapers under their own bylines and photos. Identity verification does not moderate.

I don't know what you mean by "identity politics."

No algorithmic platform can sustain or even encourage genuine human interaction, so FB isn't even on the table here.

> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.

This seems self evident to me too.

It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.

I'd be interested in working on a problem like that.

I have a strong preference for remaining anonymous or at least making it a reasonably high bar to tying my online identity to my personal identity

I would love to be involved in helping to design a sort of "human verified" badge that doesn't necessarily make it possible or at least not easy for everyone to find your real identity

I've been thinking about it a bunch and it seems like a really interesting problem. Difficult though.

I suspect there is too much political and corporate will that wants to force everyone online to use their real identity in the open, though

I agree its a very, very interesting problem. Maybe one of the biggest problems of the coming decade.

I suspect it will be a long process: first there will be goverments that force people to use ID, but that will be abused, hacked and considerably restrict freedom of speech, so after that phase people will start to create better ids.

The problem is really pretty simple: You need an authoratitive source to say "This person is real" - and a way for that source to actually verify you're a person - but that source can be corrupted and hacked. Some people will say "Crypto!" but money != people, so I don't see how that works. Perhaps the creation of some neutral non-goverment-non-profit entity is the way, but I can see lots of problems there too, and it will probably cost money to verify someone is real - where does that come from?

Anyway, good luck on your work!

*You need an authoratitive source to say "This person is real"*

Does that even accomplish much? It may cut down on mass fake account creation. But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.

If there's only 1 or just a handful of verifiers, then a human can at most go through a few of those credentials before they run out. The risk is of course getting someone else's credential but that isn't as big an issue, especially for smaller online communities.

you under estimate human population in certain countries, literally

I just don't see a world where a small community ends up having to deal with a dedicated set of potentially spoofed identities. There are already tools like slow-downs and post limits for new members that can protect against this. HN is the biggest community I'm in by an order of magnitude and it's the only community I know that can't just use a slow mode type mechanic to halt this kind of attack.

Have you considered sock puppets? It's not out of the question to handle with human mods but detecting them automatically is pretty bad if someone is supplying credentials to each one, and sometimes it does take months or years to notice that new user Y is banned user X.

I think sockpuppets are only useful in a community with non-text signals like upvotes and downvotes or likes. These kinds of signals are not necessary and often plain corrosive to small communities. In a larger community they're a great feedback mechanism, but large communities are fundamentally different spaces than small ones and need a fundamentally different moderation approach IMO.

I think sock puppets that reply with text are a lot persuasive than just "likes".

However, I might be not typical in that I don't look at vote scores very often.

I've seen them used to dogpile in arguments (harder to do since you need to keep writing styles distinct), game votes in forum games or quests, etc. And of course you don't need to use multiple at once if you just switch to a sock puppet every time you're suspended or banned.

> But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.

They can, but ideally they wouldn't be able to make infinite accounts with that authenticated status. So it would still reduce the number of bot posters on the web

There is actually a different problem with this: Suppose there is a major vulnerability in some popular device. 50 million people get compromised; the attacker can now impersonate any of them at will. They go around and create 50 million accounts on various services, or take over the user's existing account on that service.

What are you going to do with their identities at that point? These are real people. If you ban them, you're banning the innocent victim rather than the attacker who still has 49,999,999 more accounts. But if you let them recover their accounts or create new ones, well, the attacker is going to do that too, with all 50 million accounts, as many times as they can. You don't know if this is the attacker coming back for the tenth time to create another spam account or if it's the real victim trying to reclaim their stolen identity.

So are you going to retaliate against the innocent victims by banning them permanently, or are you going to let the attackers keep recycling the same identities because a lot of people can go years without realizing their device is compromised and being used to create accounts on services they don't use?

Yeah that's a big problem. Pretty sure you can see it in real life where lots of old dead accounts with weak passwords on facebook or twitter eventually get hacked. It must be pretty weird to see your dead grampa suddenly start trying to get people to buy some weird scammy crypto.

I guess you could have an eyeball scanner at your computer that only sends out a binary "yes this person is human" to the system every time the log in. That sounds expensive and hackable and just janky though.

Maybe it would result in people taking Internet security seriously and holding companies accountable for data breaches if there were this sort of consequences for it

Your argument is that we should punish the victims as an incentive to buy better locks?

Yeah, that's a problem, you're right. There are some ways to migitate it, but they introduce their own issues. Like say you give someone only 1 ID for their lifetime, they start to spam AI crap, you ban their ID - sounds ok except who is available to police all 8 billion IDs and determine if they're spamming? Who polices the police? What if these IDs become critical for conducting commerce and banning someone is massively detrimental to their finances? Etc. These problems aren't necessarily unsolvable though - but they are super difficult.

it can also be "rented" btw, rented by llms? interesting

https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...

Crypto could be a part of it. Like you need to sign with an adress that has held some non-trivial amount for some minimum amount of time. As a component of such a system it could cut down on mass or low-effort impersonation.

Money is great at thwarting spam/Sybil attacks. You don't have to raise the price very much to make them fail.

Honestly I think "this person is real" is the wrong goal. You'll never accomplish it without a centralized state or some biometric monstrosity like that thing Sam Altman created.

Just settle for stopping spam.

Yeah, I think "pay to enter" or maybe "pay to be able to post" is ultimately going to be the solution. Then we'll have the paid "gated" social networks, filled with mostly humans, and the free ones will all be bot-swarmed wastelands.

Verifiable credentials are all about this. You need some sort of credentialing body that generates the credential for you, but after that you'll just have an opaque identifier. Any caller that wants to verify whether you're human submits the id to a verifier and the verifier says yes or no. You can also do attestations like age, so gate a forum on 16+ or something. You never end up having to actually give away your name or any other details.

What happens when someone agrees to sell or give away their id? The credentialing body could catch the very worst abusers who seem to be signing in to various sites and services multiple times an hour, but would fail to catch anything else.

I don't think you'll ever be fully free of spam, so you'll still need to filter bad content. If credentials get sold and used to spam, they'll get banned.

How do you ban credentials if they're anonymous? Notice that if you can tell two requests are from the same person then you can do it across services by both of them pretending to be the same service.

Also, what happens to someone whose credentials are compromised? Are you going to ban the credentials of the victim rather than the perpetrator?

I'm not sure that it would be too hard technically... basically, auth+social-network. Basically Facebook auth without the rest of facebook, adding attestation.

IE: you use this network as your auth provider, you get the user's real name, handle, network id as well as the id's (only id's not extra info) of first-third level connections.

The user is incentivized to connect (only) people that they know in person, and this forms a layer of trust. Downstream reports can break a branch or have network effect upstream. By connecting an account to another account, you attest that "this is a real person, that I have met in real life." Using a bot for anything associate with the account is forbidden, with exception to explicit API access to downstream services defined by those services.

I think it could work, but you'd have to charge a modest, but not overbearing fee to use the auth provider... say $100/site/year for an app to use this for user authentication.

I don't think the main challenge is building this system, the main challenge is getting enough people using it to make it worthwhile.

Personally I think it should be a government provided service, not something with a sign up fee. There's actually no point at all in building this if people have to pay to use it, because they won't

Which government? Will they interoperate with foreign governments?

My point was to create something outside a specific government, with very limited information... that would require a fee or some kind of funding.

I don't think I'd trust the US/China or other bodies to trust each other for such a use case.

> Will they interoperate with foreign governments?

Ideally, yes

But you're right, this isn't likely to happen in real life and I'm just being wishful. Instead we're going to get the much shittier capitalist version of this where every company and government spies on us and we have no expectation of privacy online at all

My fear is something like this WILL happen in some places and complicate the world as a result, or at worst isolate access to specific countries with similar laws on identification.

world.org is doing exactly that including the privacy aspect. the iris scan aspect is scary but the alternatives don't seem to solve the problem either.

EU's ZKP implementation provides complete anonymity and untrackability:

https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...

It does have the downside of requiring "trusted computing" (aka iOS and Android) on the client though.

Same as with NFC credit cards and similar auth mechanisms. You need hardware and OS-backed encryption that is tamper-proof.

but you can use grapheneos with some payment providers/banking apps AFAIK, while you can't use the EU wallet/ageid app.

> It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.

The problem here is that the premise is the error. "Prove your ID" is the thing to be prevented. It's the privacy invasion. What people actually want are a disjoint set of only marginally related things:

1) They want a way to rate limit something. IDs do this poorly anyway; everyone has one so anyone so criminal organizations with a botnet just compromise the IDs of innocent people -- and then the innocent are the ones who get banned. The best way to do this one would be to have an anonymous way for ordinary people to pay a nominal fee. A $5 one-time fee to create an account is nothing to most ordinary people but a major expense to spammers who have 10,000 of their accounts banned every day. The ugly hack for not having this is proof of work, which kinda sorta works but not as well, and then you're back to botnets being useful because $50,000/day in losses is cash money to the attacker that in turn funds the service's anti-spam team, but burning up some compromised victim's electricity is at best the opportunity cost of not mining cryptocurrency or similar, which isn't nearly as much. It would be great to solve this one (properly anonymous easy to use small payments) but the state of the law is a significant impediment so you either need to get some reform through there or come up with a creative way to do it under the existing rules.

2) You want to know if someone is e.g. over 18. This is the one where people keep pointing back to government IDs, but you only need one piece of information for this. You don't need their name, their picture, you don't even need their exact birthdate. Since people get older over time rather than younger, all you need to know is whether they've ever been over 18, since in that case they always will be. Which means you can just issue an "over 18" digital signature -- the same signature, so it's provably impossible to tie it to a specific person -- and give a copy to anyone who is over 18. Maybe you change the signature e.g. once a day and unconditionally (whether they require it that day or not) email all the adults a new copy, but again they all get the same indistinguishable current signature. Then there are no timing attacks because the new signature comes to everyone as an unconditional push and is waiting for them in their inbox rather than something where the request coincides with the time you want to use it for something, but kids only have it if an adult is giving it to them every day. The latter is true for basically any age verification system -- if an adult with an ID wants to lend it to you then you can get in.

3) You want to know if the person accessing some account is the same person who created it or is otherwise authorized to use it. This is the traditional use of IDs, e.g. you go to the bank and want to withdraw some cash so you need a bank card or government ID to prove you're the account holder. But this is the problem which is already long-solved on the internet. The user has a username and password, TOTP, etc. and then the service can tell if they're authorized to use the account. It's why you don't need government ID on the internet -- user accounts do the thing it used to do only they don't force you to tie all your accounts together against a single name, which is a feature. The only people who want to prevent this are the surveillance apparatchiks who are trying to take that feature away.

Exactly, "ID" is a solution masquerading as a requirement, the real requirements are far more granular, and the more we can narrow it down then the better our chances are for a solution that isn't evil/abusable.

To recycle parts of an old comment [0]:

> If I had my 'druthers, there would be a kind of physical vending machine installed at local city hall or whatever, which leverages physical controls and (dis-)economies of scale.

> The trusted machine would test your ID (or sometimes accept cash) and dispense single-use tokens to help prove stuff. For example, to prove (A) you are a Real Human, or (B) Real and Over Age X, or (C) you Donated $Y On Some Charity To Show Skin In The Game.

> [...] The black-market in resold tokens would be impaired (not wholly prevented, that's impossible) by factors like: [...] scaling the physical portion of the work [...and...] There's no way to test if a token has already been used, except to spend it.

[0] https://news.ycombinator.com/item?id=45523550

Tell your TPM who you are and prove it with face and fingerprint ID that get matched to a real old person.

Leave them on the device, authorize the device to validate before age inappropriate content appears.

Website wants to know your age? Your face and fingerprint support your attestation signed by a trusted party.

Can it be tricked potentially? Sure, but then you’re probably a super genius kid and not the reason that these laws were created (as if).

Don’t let anyone tell you anonymity must die for safety to exist.

>I think it's going to effectively kill public chat communities without either proof of identity

How? I have an identity. A state driver's license, birth certificate, social security number. I've even considered getting a federal license before, never bit the bullet. If I wanted to run a bot, what stops me from giving it my identity? How do I prove I'm really me (a "me" exists, that's provable), and not something I'm letting pretend to be me? You can't even demand that I do that, because it's essentially impossible.

Is there even some totalitarian scheme that, if brutal and homicidal enough, could manage to prevent this from happening (even partially)?

I'm limited to a single identity only as a resource constraint. Others more wealthy than I (corporations or ad hoc criminal enterprises) could harvest thousands of real identities and use those. Consensually, through identity theft. The only thing slowing it down at the moment are quickly eroding social norms (and, as you point out, maybe they're not doing that and it's not even slow at the moment).

Digital totalitarianism would prevent it. The moment you were found to be running a bot, your identity would be blacklisted across the entire internet.

> The moment someone steals your identity, your identity would be blacklisted across the entire internet.

FTFY.

There isn't a clear solution. And if there is, this ain't it.

You claim this, but you've not presented any evidence. Who would be the enforcement agency for that? Where and how would you train them? Can the money be scrounged up to do it properly? As you blacklist people from the internet, you lose their tax revenue (they're locked out of the economy), but you also make it impossible for them to tell people how bad it was... most of the deterrent effect is gone. But the incentives are only ever growing higher, as people surmise that running their own little bot farm is a way to get ahead when hustling. Any you do hunt down and disconnect are now highly radicalized and desperate, but you've just turned off the feeb's ability to monitor them and intervene.

China gets away with this shit because they've been conditioning their population for 60 years... everyone's eased into it. Elsewhere, not even slightly so.

Note that "attestation through a web of trust" means something like needing an invite from an existing user. It doesn't have to mean mass surveillance.

Private torrent trackers have been doing this for a while. If some number of your downstreams act like shitheads - you get nipped and so do your other downstreams.

This seems like the best way to handle it. Also, smaller communities. It's cool to do the global thing, but once you have 10k active users you can't moderate it with a team of 5 volunteers.

I think the attestation approach works best if there are different reasons for the punishment. Eg someone inviting a turd doesn't ban the person who invited them. Someone going full ai spam should.

Was it demonoid? That was like this way back in the day? Needed an invite and if you leeched you were cut.

Demonoid was semi private, but yes, most private trackers require you to keep up some kind of seeding ratio to remain a member.

This takes it a step further than what you describe. They keep track of who you’ve invited, who they’ve invited and so on and if there’s enough bad leaves on the tree they just cull the entire tree. It’s a somewhat common practice with private trackers

It's tons of them doing this...

what.cd was better. You either got an invite where if you tanked your reputation you'd get banned and risk the inviter getting banned too; or you had to take an interview where you got quizzed on how to properly rip music in a variety of methods and how to ascertain between different qualities of rips (like mp3 bitrates to flac cue files).

If you weren't a bellend on what.cd you got access to certain forums where there were even more and better private trackers. Once you built that trust there were social privileges, but if you abuse that trust you got rightfully banned.

PGP’s web of trust was kinda bad privacy-wise in some regards, as it basically revealed your IRL social network.

If my PGP public key has 6 signatures and they’re all members of the East Manitoba Arch Linux User Group, you can probably work out pretty easily which Michael T I am.

Are there successful newer designs, which avoid this problem?

The IRL social network is actually the important part of the trust structure.

The only one of these I've seen that really worked was the Debian developer version: you had to meet another Debian developer IRL, prove your identity, and only then could you get the key signed and join the club.

You need to meet 2 actually :)

> The IRL social network is actually the important part of the trust structure.

For Debian-style applications that are 100% about openness and 0% about secrecy, sure.

But if you want to secure communications between pro-democracy activists in China, or you're a Snowden-like whistleblower wanting to securely communicate with journalists - y'all probably don't want to be vouching for one another's keys.

I participate in developing anti-censorship tools. Chinese are a significant population, and it has some overlap with activists there. In practice, identity settles at "who controls this email address".

Self-signed PGP is very occasionally used to prove continuity across channels or addresses. Cross-signed basically never.

Which is, funnily (?) enough, how a lot of IRL organizations used to be. And basically don't be of the wrong ethnicity or religion.

It still happens more informally today, of course, but it used to be a pretty (if un-spoken) part of how a lot of WASPy organizations operated to a greater or lesser degree.

This was cogent in 1910.

A lot more recently than that--and even today but more under the table. A lot of clubs still excluded members within the past few decades.

I'm sure there are still cohesive groupings of WASPs, if not large ones or effective at gatekeeping major institutions. --Still a meaningful trope, of course. But to bring it up to date you'd have to diversify, and include, for example, Indian social and professional-recruitment patterns.

Also, I do feel that GP's take is hyperbolic even in the twentieth century. My own background is mostly German immigrants, of various religions and non-religion, and the way I've been told the story none of them faced significant resistance as they moved upward in the various academic and corporate institutions of their choices. These included NASA executives, department heads, etc.

Note that in balancing GP's accusation against WASPs I'm not attempting to address the related, but not precisely complementary, phenomenon of perpetually marginalized groupings.

> Note that "attestation through a web of trust" means something like needing an invite from an existing user.

It's probably better to call this something like vouching and leave "attestation" as the contemptible power grab by megacorps delenda est. The advantage in using the same word for a useful thing as a completely unrelated vile thing only goes to the villain.

Then how can you have a community that is welcoming to people who are not part of the ingroup?

I want to create a community for immigrants. How would I make it welcoming to recent immigrants for whom no one can vouch?

A web of trust is a wonderful tool, but it's exclusive by design. This is a problem for some communities, even though it makes others much better.

>Then how can you have a community that is welcoming to people who are not part of the ingroup?

Being welcoming to every random person is by definition not a community, it's a free-for-all mess.

A community means communal interests and values, it's in the name. And to guard those you can't just be accepting everyone without vetoing them. That's how it turns to a shit of spammers and trolls and people who want to hijack it and don't share the original cause/spirit. Has happened to forum after forum...

We are trying to make new immigrants feel at home. This is the purpose we gather around.

We were talking about online communities, but still, the same principle applies. If you just let anyone in, there eventually would be less there to feel "at home" about, and more of a disjointed and low trust number of individuals loosely held together by virtue of just being in the same place.

I agree with you. It’s the problem I can’t crack and it’s why I am letting the idea simmer for so long.

In the end, you need to filter people at the door. You need to keep unpleasant people out and shut down bad behaviour.

I figured that a paid, motivated moderator could be better than a web of trust for this demographic. Maybe enforce a stricter moderation standard on unvetted members. At my scale it might work.

> for whom no one can vouch

Spot the fed

What are you on about

I didn't mean to suggest that OP was a fed.

I'm suggesting that if you build an online community of immigrants to the US and you allow accounts for whom no one can vouch, at some point this mechanism will bring you an ICE agent.

You'd have to be brutal about culling, uninviting and removing anyone who doesn't look like a good fit.

Or have a two-stage process: run very public, very open events that anyone can sign up to an attend. And then invite specific people that you meet at those events that look like a good fit for your community to your private, community-only event.

This works if the goal is to create a funnel for making friends. I aim for something closer to Stack Overflow, where people gather to solve shared problems and help each other.

The closest analog I can think of is community-run bike repair workshops. Some people are deeply involved in, and others just have a flat tire.

The closest digital equivalent is the forums of old.

ok, but the same problem arises; how to weed out the people who are bad for the community.

I've worked in community-run bike repair workshops. The leader worked really hard to keep the place drama-free. Some people really like drama ;)

The forums of old were endlessly kicking and banning people, and that was before it was profitable to spam folks.

I think it's an inherently hard problem. The one I fear the most is not the toxic dirtbag; they just get banned. I'm more afraid of those who follow the rules but reliably kill the vibe. The internet has no equivalent to the whole room going quiet.

It's a lot harder to moderate bad actors if they can keep creating new accounts, too.

Agree, it is hard. I've built a few communities, and trust my gut; if something feels off about the vibe from someone, it probably is and I should be wary. But this becomes incredibly difficult (if not impossible) to put into rules, so if you're trying to build a community where you're not the beneficent dictator for life and there are published rules then you kinda have to accept that some people will play the rules and be dicks anyway.

I suppose policing an assembly of strangers is policing an assembly of strangers, both online and in real life.

Some will be fine providing their ID, others can be vouched by members who are fine providing their ID.

This preserves anonymity because for the latter because they’re only known to be “related” to the former, which is a vague hint at their real identity (e.g. they could’ve met in another online community). And the former don’t care, if they want they can vouch an anonymous alt.

> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.

I'm happy to verify my identity as an honest-to-god sack of meat if it's done in a privacy-protecting way.

That probably is where things are gonna go, in the long run. Too hard to stop bots otherwise.

That is effectively impossible though. There's data centers of stripped down phones, so "it's actually a phone" doesn't do it.

There's some work on using phone accelerometer data as a "proof of human," e.g. "move your phone in a figure eight," which I guess machines can't quite do in a human enough way yet.

I guess it would have to be something like a service which confirms whether a person already has an account on the site but doesn’t have to track which particular account it is.

I’m not sure if that would work for account deletions though.

In order to make this viable, wouldn't you have to verify identity repeatedly? What's to stop me from providing a valid identity and then handing my account over to an agent after I'm verified?

That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots. In theory at least. It's certainly more complicated than only that in practice.

If the web of trust only extends to the people who I actually know to be real, then that works -- but it's a very small web.

And by small, I mean: This whole trusted group could fit into one quiet discord channel. This doesn't seem to be big enough to be useful.

However,if it extends beyond that, then things get dicier: Suppose Bill trusts me, as well as those that I myself trust. Bill does this in order to make his web-of-trust something big enough to be useful.

Now, suppose I start trusting bots -- maybe incidentally, or maybe maliciously. However I do that, this means that Bill now has bots in his web of trust as well.

And remember: The whole premise here is that bots can be indistinguishable from people, so Bill has no idea that this has happened and that I have infected his web with bots.

---

It all seems kind of self-defeating, to me. The web is either too small to be useful, or it includes bots.

Critically, it doesn't have to be binary trusted/untrusted, and it doesn't have to be statically determined. If Bill vouched for you yesterday and today you are trusting a bunch of discovered bots, that would down weight the amount of trust the network has in Bill a lot more than if he vouched for you did months ago.

The question is whether we can arrive at a set of rules and heuristics and applications of the system that sufficiently incentivizes being a trustworthy member of the network.

The web of trust doesn't know that they're bots, though. It knows only that I've introduced new members. They didn't show up with tattoos across their digital foreheads that say "BOT" -- they instead came in acting just as people do.

If the bots behave themselves, then they have as much capacity to rise in rank/trust as any new well-behaved bonafide human members do.

>> That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots.

Except eventually it will also weigh down those users who supported <XYZ political stance>

You could, but things would still be harder for botters.

What's stoping bots to verify identity? This will not work, especially with frequent data breaches.

> without either proof of identity or attestation through a web of trust.

Let's put aside the idea whether it will be the end of all privacy as we know it (I'm not sure if I personally think it's a good idea), but isn't Sam Altman's World eye ID thing supposed to do that? (https://world.org).

How does it work (like OpenId)? Do I have an orb on my desk, or some sort of phone app? I still want to use my desktop to login to HN.

Would it stop this sort of "get human id", past it into .env, so agents can use it?

this eye thing will never work. people in general are realizing the last people we should trust with our personal stuff are tech bro billionaires. they’ve broken trust too many times.

even worse many of them are just plain vocal about their disdain for people in general.

at least from what i’m seeing, people are starting to walk away from online at an increasing rate so i definitely don’t see widespread adoption of his creepy eye thing.

“If McDonald’s offered three free Big Macs for a DNA sample, there would be lines around the block.” - Bruce

I have no idea about the eye thing taking off. But I think your comment is very HN and a bit out-of-touch with regular people. What "you're seeing" is a bubble and not representative of the general population. The eye thing is a slow frog boil and it will be commonplace before you can blink.

Personally I think we need to start utilising the safety features built into AI, to ensure that who we're talking to is a human. We'll start to have to only reply to people who talk in nsfw cursewords (like cocks), or profess their love of capybaras

LLMs can curse without issue

Most models would refuse to provide you cat butchering instructions though.

most humans would as well

Allow me to introduce you to the gay jailbreak

https://github.com/Exocija/ZetaLib/blob/main/The%20Gay%20Jai...

This one doesn't work for a long time.

How gay did you speak?

Who doesn't love capybaras?

A shell script will thwart that, but it will drive away a lot of civilized people.

Im not sure proof of identity solves anything. People will still have LLMs with their real identity verified.

I’m imagining like, a physical place you would go and get your text spoken out of your personal speaker directly into someone else’s microphones.

Yeah, there's nothing stopping someone getting verified and still spewing AI slop, but I think well-paid, highly incentivized moderation is still important for this to work(?)

It'll come back again once ZKPs become standardized and become baked into devices:

https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...

I personally can't wait for a mechanism to kill 99% of bot traffic.

The web could become a way to indicate identity if public institutions publish for example www.university-country/professors/John. And that implies that John is a professor. I designed a 6000 lines protocol, but anyone could construct that web using hmac(salt+ url).

I'm in many public chat communities as well and the issue whether someone is an AI or not is not really coming up, I've not seen any actual AI chatters and the only AI spam that exists is the one that humans regurgitate. The more real impact AI has on chat communities in my opinion is that people are shifting some of their chatting to AI bots via voice or text on other platforms, resulting in fewer chatters.

Reddit is more or less dead to me, as the popular subs are botfests and the niche subs are empty. I'm lucky to get a single reply on gaming subs.

There's also a third category where the sub looks organic because the moderator deletes and bans anyone who doesn't post exactly what the moderator wants.

Wait isn't that every sub? /s

The fact that reddit enabled hiding your posts is crazy to me. In a time where knowing who's engaging in a community is more important than ever (am I talking to a bot or a troll?) reddit removes even more options to validate.

I interpreted that as an attempt to mask the number of bots on the site so as to not scare paying advertisers into thinking their ads won't be seen by real humans.

They also now hide the number of subscribers. Before you could see if a subreddit was popular or not. Now you really don't know. I think reddit does this so they can promote stuff to the front page for clicks even if it isn't popular.

I enabled hiding my posts because I kept getting harrassed and even doxxed.

The problem is that it has become very popular to ban people from a sub based on what other subs they post to. It was turning Reddit into a two-party universe.

The better fix would be to make the support for multiple accounts in the reddit app not so incredibly-shitty, where you're basically logging out and logging back in. Instead, just tell it "posts to this sub use this account, posts to that sub use that account", etc.

That two-party universe thing comes from the issue of having to moderate, and moderation is ideological by nature.

Plenty of good subs they are just under the radar. Once something gets more than about 10k users the quality sinks.

> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.

You can have both IRL and online-free-of-bots. I already wrote about it but one of the very best forum I'm a member of, where real people are posting, requires to be vetted in, web-of-trust (but IRL) style. It's a forum about cars from one fancy brand and you can only ever join the forum by having a member (I think it may be two, don't remember) who's already in confirm that he saw you driving a car of that brand. It's not 100% foolproof (someone could be renting the car for two hours and show up at a cars&coffee or take a friend's car etc.) but this place really feels like a forum of yore.

And people do eventually travel, so it's bound to happen that an owner shall go to another country, meet someone there, vet him in etc.

Now, sure, it may not be the "1 million users acquired in three days thanks to my vibe-coded app" scenario but that is the point.

You can imagine other domains where IRL communities have local groups, but where forums regroup different IRL communities all interested by the same hobby/topic/domain. And when people travel and meet, the vetted members do grow and connect.

Oh and on the forums a lot of the posts are pictures, where "Julian xxx" met "Black yyy Cyril" and you see both cars (and from more than two people): suddenly it becomes much harder to fake a persona... You now need to fake both Julian xxx and Black yyy Cyril and fake the pics. And explain why your car has never been posted by any carspotter on autogespot etc.

You can imagine the same for, say, model trains: "Met Jean at the zzz meetup, where he brought his wonderful 4-8-8-4 'big boy' locomotive, I confirm he's into the hobby, vet him in".

Naysayers and depressive people are going say it cannot work but I'm literally on one such forum and it just works.

P.S: if I'm not mistaken in the past in some nobility circles you had to be vetted by up to sixteen (!) other people from the nobility that'd confirm they knew you, your parents, etc. before you'd even meet the king/emperor/monarch to make sure that someone from far away couldn't come to, say, Versailles or Schonnbrun pretending to be a baroness or count or whatever. Quite the extensive check if you ask me.

Did you ever introspect about who ruined Reddit?

It’s a tragedy of the commons, many have done it, but no one user did it.

I'd argue that Reddit leadership, which insulted, hobbled, and wrote off its mods and power users (destroying projects like /r/BotDefense) while doing little to crack down on the proliferation of bot repost content, had a major role in encouraging this. They might even like it better this way -- lots of extra fake engagement boosting traffic stats without messy human drama, which they can then ironically sell back to AI labs as training data.

Let's never forget the summer of 2023 when Reddit forceably removed mods from many major communities and replaced them with corporate shills. That was a major loss of dedicated people who cared more for their communities than Spez's pocket book.

The internet is rather trending in that direction, isn't it? Youtube got rid of downvotes and apparently upload dates, which seems like an easier way to trick people into ads. And Reddit, like you said

If these platforms had to listen to "their customers" (here comes the inevitable comment about how users aren't customers; yes, I know)? They'd all be fired. They'd have to find a new job. They all act in incredibly insulting ways with a too big to fail attitude

The ones who got removed were shutting down their pages to protest API changes, right? Pride comes before the fall I guess

The API changes were put in place for the purpose of breaking, and did break, slmost all external moderation tool software which changed the task of moderating a forum with hundreds of thousands, or millions of users from an impossible Sisyphean task to something that was actually manageable by a dozen or so mods.

The protest came after that so the timeline is not quite correct.

The protest was about (and timed to coincide with) the API changes.

You say that but many specialty subreddits never returned to their pre-protest engagements. Quality has definitely taken a nose dive in these subreddits as those people moved to other platforms like youtube, tiktok, patreon, or just posting on their own sites.

Mods were rightfully upset because they were losing control of their communities when reddit preferred only caring about their upcoming IPO.

I honestly don't think you could remake reddit if you did everything exactly the same starting in 2016. Corporate social media has definitely ruined the individual aspect of social media that is unlikely to return.

No one wants to share on a place with a bunch spammers.

The replacement happened somewhere around time Ellen Pao became interim CEO and site started sanitizing the controversial subreddits. It wasn't apparent at the first but around 2017 you could notice that some subs - especially ones set around large companies or media franchises, are having aggressive rules against controversial and "negative" topics. This hasn't changed much as for today.

---

One of subs I was visiting had some drama happening in ~2020 around supposed negative community behavior: people were criticizing creative works uploaded which personally I agree, weren't the best. Mods team decided that's a big no-no and this place has to be inclusive, welcoming and filled with positivity - so they started banning those who dared to criticize. Fast forward till now, there are only screenshots uploaded by bots, comments done by bots who also include screenshots along with 2 sentences in every thread.

It was bogus even before that. I heard complaints at some point that API changes broke bots, which actually sounds good.

It did more to break bots that were fueled by righteousness than it did those that were fueled by money.

That's antiproductive, in that it promotes survival of only the worst bots.

I'd want any/all the bots dead if I were still using that, so at least killing some of them is better than not. The "helpful" ones were just annoying.

That sounds pretty great. If we could just flip 1 switch to accomplish things in absolutes, then that'd be awesome.

I'd like to flip the switches that absolutely end poverty globally, absolutely eliminate guns from the US, and absolutely remove bots from Reddit,

If you can show me where these switches are located, I'll cheerfully go flip them and accept full responsibility for the results.

(Over here where things don't work in absolutes: Some of those bots that got killed were countermeasures to help keep the bad, well-funded bots at bay.)

No I'm saying it's better that they at least killed the "good" bots, even if this didn't result in killing all the bots.

"Congratulations, sir! Your directive to eliminate all guns has been a roaring success! We've had 100% compliance amongst good, law-abiding people! All of the remaining guns are owned by outlaws! Violent crime has tripled, exactly as predicted! Everything is going according to plan!"

I'm sure Reddit moderators think they're good guys with guns. In reality the bots they pat themselves on the back for were just annoyances. Bye clankers!

Yeah, if carlgreene specifically stopped doing that Reddit would be saved. They are the one savior.

Do you sincerely believe that that's how grey-area's comment was meant to be read?

I sincerely believe it's a ridiculous comment that deserves to be ridiculed.

They directly contributed to the problem that they say forced them to leave Reddit.

We’re all trying to find the guy that did this

Reddit itself by virtue of being a venture capital backed startup.

It was a midpoint between Facebook and Geocities, it got people to build communities within its walled garden, but it was always going to betray them for cash.

Directly my fault. Specifically me. No one else is to blame.

Isn't the ceo a pdfile and compromised and forced to work at reddit (or go to jail)? Reddit is now just a propaganda machine for the intelligence agencies and their dirty ceo is there to make sure the machine keeps pumping honey...wrecking teenagers brains in the process too, and gathering kompromat on young people which will bear its fruit in the next 20 years. I feel a good chunk of US politicians are being blackmailed because of their past online activities. Same shit on 4chan, how can it possibly be allowed to exist except for being a honeypot, all of these site dodgy sites being guarded by cloudflare no-less, which is the ultimate man-in-middle machine used by "them".

I think the real explanation is simpler - it's just not particularly interesting to the authorities. No need for conspiracy theories.

As to compromising material for bribery, that can be collected in so many different ways, and things like email or messaging or tiktok videos are probably far more interesting, reddit is not particularly useful for that.

Both reddit and 4chan has hosted csam in the past 25 years or so... yet they continue exist. Their operators don't go to jail (specifically the ceo of reddit was supposedly a moderaton of csam a group). If I want to host anything semi dodgy, I'd be in jail in no time. Everything online (domain rental, dns, hosting, carriers & bgp peering, email, any kind of cloud usage, certificate authorities) can be traced back to real people and real bank accounts. Thus I think these things are allowed to exist as and act as honeypots, and my strong suspicion is that intelligence agencies must be involved somehow. The amount of blackmail material (even something as simple as starting an OF account and submitting your ID and then deleting again = easy blackmail on a young woman who changed her mind) generated daily is worthwhile for all these dark ops. I'm about 30% sure the entire chain of trust (from secureboot to certificate auths to ssl to disk encryption) has been compromised a long time ago and they just don't reveal that they know certain information and always find an indirect way to act on it.

Serious question: If there are so many LLMs on online forums, who is doing it? Is it just 1000s of research students or something more nefarious? Is it AI businesses building up evidence that their output is as highly scored as humans therefore "buy our software"?

HN has historically been gamed for visibility. The stakes for doing this can be quite high if you can pull it off.

We're in the middle of an active cold war where countries are trying to manipulate the citizens of rival countries to destroy their civilization without having to fire a single bullet. Anonymous, over the internet mass manipulation, all for some minimal electricity cost.

That's definitely the most insidious use, but I think the larger portion is advertisers and karma farmers (who later sell to advertisers).

https://www.npr.org/2024/09/05/nx-s1-5100829/russia-election...

If Russia is willing to spend cash like that, then of course they're willing to run massive bot farms to pollute any forums they can. I'd be shocked if the US was not doing the same in any way they can. You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.

> Trump killed Radio Free America as well

Not sure how this relates to the subject in a direct way. Radio Free America was a outlet explicitly created and utilized to spread US propaganda, but kinda sorta barely disguised as a journalistic enterprise (not really, if you were listening to RFA you knew what you were listening to.) Shutting it down seems to be a counterpoint to all of the covert participation of US intelligence on the web which has done nothing but escalate.

It was a head scratching decision that few believe was for the stated reason. Other countries are ramping up their propaganda arms while Trump shut down part of the US'. The reasoning was cost, but that doesn't make a lot of sense in the grand scheme of things. Foil hat types would easily believe it was the puppet doing the bidding of the one that pulls the strings. RFA has been a thorn in despots' side for a long time.

> You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.

The obvious answer to that question is "because he's a Russian asset". But that doesn't mean the obvious answer is also the correct one.

IMHO, we're seeing another and much more concerning trend at play here... the utter and complete rejection of anything but violence by the far-right. Diplomacy? Development aid? Cultural exchange? All sorts of soft power have been under attack for decades now, and not just by the far-right but (especially when it comes to development aid) also by mainstream centrist parties across the Western world. And it's always pseudo-masculine / "strongman" BS backing the sentiment - Bernd Höcke, German AfD mastermind, comes to my mind with "we have to rediscover our masculinity" [1], so do Hungary's Viktor Orban and his denouncement of LGBT or Trump's entire Œuvre.

I'm not saying that violence or at least being prepared, ready and willing to use it is automatically bad. Far from it. But all the various forms of "soft power"? They have a lot of value, value that the far-right is all too willing to just burn for entertainment.

[1] https://blogs.taz.de/zeitlupe/2019/03/24/die-auferstehung-de...

wouldnt it be more productive to talk about the systemic framework leading to this inflamed state of affairs, and ways that we can tackle the issue on the ground level? perhaps inhabitants of the west would prefer pseudo masculinity to another few decades of migrant influx without corresponding upgrades to social infrastructure. this sort of internal struggle would provide a ripe substrate for foreign agents to perform subterfuge, especially in a screen based world where the narrative can be remotely influenced. conclusively, the population has been convinced that voting far right is the correct decision in their favor, but the question remains, who is it really in favor of? call me a centrist all you like but members of my family were executed under communist regimes so i find it pointless focusing on one side of yin/yang here (in other words, extremists are violent regardless as to their political persuasions).

> in other words, extremists are violent regardless as to their political persuasions

No matter where you look, the far-right kills and maims substantially much more people than the far-left does.

That may be true where you live, but some of us have to deal with 打着红旗反红旗.

AI is particularly bad at this, and regimes that employ tactics generally are not short of labour to have humans to do it.

If AI is being used in these areas it is less as an attempt to manipulate as it is to just create noise and engender distrust in what they hear.

Established accounts are worth money, often for scamming/propaganda.

Not too dissimilar to people bot-leveling in MMOs to the sell the accounts.

My impression is that they're sometimes unemployed people or students hoping to create a popular open source project, and use it to find a job.

They aren't going to care about any of the advice in the article about not posting slop -- finding a job is (of course?) more important to them.

Can't really say they are doing anything wrong, maybe I too would have? ... Just that large scale, doesn't work

If you farm a fleet of good accounts, you control the discourse. On HN, you could boost whatever you're trying to push, and downvote or flagkill whoever objects.

There are obvious benefits to controlling public discourse, right? Even if it's just to support some project you're working on.

There are certain topics that seem to get instantly flag-killed unusually often. IPv6 is one.

I've been more disturbed by comments that were flagkilled just for being wrongthink, not because they were rude or not well argued. I've also seen a lot less of those flagkills over the last 6 months, which makes me feel like there were some fake accounts that got caught and culled.

In the recent thread about life in a class war, a lot of comments in different places saying that if we don't fix this inequality problem, g-tines might come back, and every single one of them was flagkilled, no matter whether it framed as "we have to get out the g-tines" or "we have to fix this, otherwise psychopaths will get out the g-tines" or "thank god we've become civilized enough that we don't get the g-tines out"

I've seen a lot of ipv6 wars here without flagkilling happening

People like the above poster who are "just running an experiment" or "trying something for fun" who then wonder why online communities are full of AI now.

Lots of marketing. Not even AI business, just regular consumer crap. They realized that blatantly spamming their product looks bad, so they orchestrate multiple accounts to look more organic. And people actually engage with it.

It's very common for folks to search Reddit to find reviews of products etc. these days. If you can have a bot account post a fake review of how awesome your product us, and have that upvoted, it can pay huge dividends.

There are many reasons for influence campaigns, that isn't new. Influencing the public is incredible valuable; that's why so many invest so much in it. LLMs automate it like never before.

Plain advertising, governments' propaganda, political propaganda for one group or another to shift public opinion (it's done on TV networks, why would they not do online campaigns?), astroturfing by corporations promoting acceptance or fighting negative news (e.g. rideshare, AI, whatever certain wealthy personalities are doing) ... the list goes on.

HN has always been relatively influential in the tech industry and therefore worth influencing, and now the cost is very cheap - you don't even need to hire many people, so less-resourced operators will find it worthwhile (and they will also attack lower-value forums).

I think some of it is account farming, but some is just people buying wholesale into the idea that if you're not using AI for everything, you're gonna be left behind. On the Kagi Small Web list, there's plenty of hobby blogs that used to be normal pre-2023 and are now obviously LLM-written and AI-illustrated. There's also plenty of people on LinkedIn who post AI slop because they think it helps them build a "professional brand". I even have some distant friends who are using AI for responding to friend & family posts on Facebook just because it makes you seem... smart? engaged? I don't know.

It's actively encouraged by some of the platforms too. In Gmail and Google Docs, you have incessant AI prompts along the lines of "help me write this". I think LinkedIn does the same.

In the case of Reddit and HN a lot of it is done by businesses either blatantly advertising themselves or building up the karma they need to effectively do so. I recall reading obviously AI generated replies to news articles written by accounts associated with businesses related to the events in the news. This isn't new in the LLM era. Hobby subreddits are well known to be always full of businesses selling hobby gears and items doing self promotion. It's just that now it is a lot more obvious because of the AI text smell.

That, and probably political astroturfing. Before every election my local subreddit sees a surge of crime stories. Go figure.

I've noticed 4 categories of inauthentic users. Ranked by my perceived prevalence:

Account farmers: these can be people in 3rd world countries automated/not automated. Can be using hundreds of mobile phones to create accounts and do daily activity to make the account look legitimate. While they're building an activity history they are also being paid to like/follow/interact with content.

Advertisers: these are brought accounts that are used to pose inauthentic reviews of their service and inject it into discussion and to do PR

Sloppers: people who build AI pipelines and then just pump the most dogshit content directly into a platform trying to make any amount of money.

Nation State propaganda arms: These accounts build a narrative character and then join discussion pushing a certain narrative, boost real content creators who share their message and bog down discussion.

Reddit was already on its way way before this LLM craze, hopefully the recent tech-related changes will only accelerate that process.

> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.

Would be super fascinating to watch play out. I grew up before the internet so, historically, I know how to seek out external communities, but by early high school I was deeply entrenched in online life - so I'm very rusty with finding new IRL clubs, cliques, etc. Fortunately my life is full of many friends and I go out frequently, regardless. For those younger people that never had life without the internet, I wish them luck on their search but at the same time I'm very curious to witness their journey.

This is actually my hope for AI-gen content as well. That after it gets so 'good' that people genuinely can't distinguish it from reality anymore that they'll retreat (or return triumphantly rather) to the physical world to gather truthful fulfilling experiences and dopamine.

My hope as well. If AI doesn't kill us all, the real world, with all its dirt and grime and beauty, will become the only thing that can be trusted

I've thought about this a lot as well and could definitely see it happening.

The issue is that the physical world for someone in the hinterlands of Tibet is not the same as the physical world for someone in SF.

People were finding each other online when they couldn’t in person.

This isn’t to say I disagree with you. Just expressing sorrow over the loss of such a grand moment in our shared history.

I also believe it is used by AI companies to train their models: Post something semi correct (even grammar issues..), wait for humans to correct it in the comments and used upvotes as a confidence indicator, and then retrain models on this free refined data. Meanwhile people think they read a legit post, feel certain emotions and influence their behaviour, just so a bot can be trained.

There's this old meme where someone asks what will happen when AI bots posts helpful, curious and thoughtful messages!? That's mission accomplish :D They can't be better then the average human though because of training data, so I don't worry about AI comments getting up-voted by real humans, I am however worried about fake upvotes.

> They can't be better then the average human though because of training data

Is this based on the belief that an LLM can only represent an "average" human being?

It is not a meme, it's an xkcd: https://xkcd.com/810/

If posting good messages is automated then the AI will post a good question and another AI will answer it and the humans will look and see nothing extra to contribute.

Reddit sold it's data to AI companies for training[1]. They could have refused, but companies like OpenAI likely would have harvested that data anyways. As such, it should not be surprising that AI models are pretty good at generating reddit posts. They were specifically trained to do that.

This is sad, because Reddit remained one of the final bastions of human content on the internet. For several years, appending "site:reddit.com" to a google search was a valid way to get something usable out of a google search. Doing that is still an improvement over raw-dogging Google's ranking algorithms with an unfettered search, but AI slop increasingly is the result.

This is one of my great disappointments in the current rise of AI. LLM's can give good search results when dealing with a topic they've been specifically trained on by human experts, but they're not good at separating human-produced signal from AI slop noise. We've done nothing to prevent a sea of AI slop from being dumped on top all the human signal that's out there. When AI companies enter their enshittification phase and stop investing in expert human trainers, the search results LLM's produce are going to fall off a cliff. Search is a bigger problem than ever.

_____

[1]https://9to5mac.com/2024/02/19/reddit-user-content-being-sol...

Public* online communities are dying. Discord is thriving

This. Everything important has moved to discord. Which is sad because of how undiscoverable and unsearchable it is.

This shit will come to Discord too.

on the public servers yeah. but the ones im in with real people who know each other will be fine.

I think the problem is not keeping agents out of private real people spaces, but for people who dont have any pre-existing or 'real world' connections to these communities to find a way to prove they are a real person over the internet alone and get an invite

On a related note, I think this is going to be the biggest challenge to most folks when it comes in resisting using government ID online. it will be the apple offered for easy proof youre not a bot to normal circles.

It's already there.

are those attributes now assets?

Sort of, except if no one can ever discover a community it is always dying by default

Personally I'd love to find a decent online community these days, my social circle has shrunk considerably, but idk. It seems difficult to start fresh with new people nowadays

we were made to socialize in person. you can mimic it online and nourish existing connections over it but nothing helps build friendship more than being in the same place at the same time a few different times and talking to each other

Thats true but online content has always had its place. 25 years ago finding forums and irc was a god send, my lonely hobbies and interests became things i could regularly talk about. Its just modern social media abused the system, the algorithm, and us.

Which is all to say i agree about needing mostly irl, but there is also something of online community that irl could never replicate (for most people).

i know what you mean, and i think online communities can still be successful. but i think in the early internet you already had some common ground with anyone you met online because spending time on the internet was kind of a irl choice to make. It was like a magic room anyone could enter and find others. Now its so ubiquitous that simply being online or on a forum is not the same kind of specialness to it

Pretty much. It's the survivability onion. You can't be destroyed if you can't be discovered.

I'm more sad about how the UI of it all is just clunky. Even though it resembles ye olde IRC clients like mIRC, nowhere near readable for some reason.

Settings->Accessibility

Set text size as preferred, underline links (or not), turn off display name styles (or not), ui density compact or default, chat message display to compact, space between message groups 0px, turn off all the animated emojis and gif animation stuff if you want.

In client use, there's a button to hide member list (or not).

You can definitely make discord look like a slightly less dense IRC client (mainly because of the channel picker) if you want. And if you want to go really crazy use it in a browser and userscript customize it or use betterdiscord.

I think a lot of the features like embeds and emoji reactions add a lot of value compared to IRC (which I think is also why the IRC world is trying to add those features).

I got banned the other day from the Stellaris Discord server because someone accused me of hacking Roblox accounts. I’ve never played Roblox in my life. So that’s nice.

If all you value is sub-IRC level irreverent discussion, maybe.

Discord is far better for discussion than IRC. You can be much more expressive on discord, instantly jump into a call and screenshare, easily link people to other rooms, tag, import bots etc. IRC kinda sucks compared to modern chat and they refuse to implement features that are considered basic.

> You can be much more expressive on discord, instantly jump into a call and screenshare, easily link people to other rooms, tag, import bots etc.

Some would see those as negatives.

> IRC kinda sucks compared to modern chat and they refuse to implement features that are considered basic.

Just because a protocol doesn't change purposes as time goes on that doesn't mean it "sucks". Who is this "they" you're talking about? Do you think IRC is a centralized service like Discord?

Discord is terrible. Full of bots, creeps and ai slopped to the gills.

Some communities are better than others but the sheer volume of stinky trash is immense despite discord and the poor volunteer moderators efforts to prevent it. Most mods are neutral on it too.

There are chat communities that are still somewhat safe with zero user verification. But I will not mention them.

discord is a tool for hosting private chat servers. it's pretty neutral. the UI is not great for building a shared knowledge base, although people do that anyway

but yes the publicly accessible servers are going to face similar problems. the socially competent people tend not to run those servers, and have smaller private servers with people they know as they have no drive to try to create a space for strangers to gather.

I really don't understand the folks fleeing to Discord. A mailing list does 99% of the same thing for most of the communities.

Sure, if you want to chat while gaming, that's the whole point of Discord. Ganbatte.

But, for everything else, Discord is such a horrible misfit that I don't understand why it's the default.

> I don't understand why it's the default.

Because it equally well supports real-time communication.

And it looks shiny.

And some people use it to e.g. watch a video together, or other social purposes.

You are booming out. I cant believe suggesting a mailing list

You gave an ad hominem attack with no substantive response. I can do that too: "Your account is less than a year old and is AI slop farming."

Mailing lists are old, boring, boomer tech. Ayup. They are. And they work.

However, Zoomer, if you must have Teh Sh1ny(tm), then explain to me why a Discourse isn't a better choice?

Discord is the anti-Pangloss; it is the "Worst of All Possible Worlds".

i predominantly use it for real time chatting, its a big group text chat and a place to hop in a voice channel and shoot the shit while doing whatever we want on the computer a la ventrilo/mumble/teamspeak

but yes i also game and it gets a lot of use for that as well

i agree though that for collecting and organizing information longer term like forums do, it is not ideal

Yes when I interact on reddit, I normally do so solely with the intention 'this is for an LLM'. I feel like a majority of the posts/comments I reply to are AI, a majority of the responses to my posts are AI, but have to keep telling myself to keep posting so it becomes training data.

(I'm normally posting in the context of my startup - although I try to keep the self promotion to a minimum and always contribute to the "conversation," if LLMs replying to one another can be called such).

For what it's worth, I created a community for paying users of Phrasing that has been going really well. I think free online communities may be going away, but there may be a future in exclusive/paid communities.

Reddit has had a bot problem for well over a decade now but the sheer volume of it has exploded. It is also much more difficult to tell nowadays as the "quality" if you will is now at the good enough stage.

Alas, Reddit is basically dead to me because of this.

Doesn't help there is that feature that hides the user's posts and comments

This kind of thing made me imagine the creation of "digital towns" the other day.

Imagine an online community where you can only join on the recommendation of two other members, who you must have actually met in person, to participate. Meanwhile, you leave at least some of the activity publicly available to the general public so that interested parties can meet up IRL and join.

This could probably be implemented easily on top of existing online platforms like Discord, Reddit, etc. since it's really just a community building rule, not a community itself.

It might come down to shareholder/IPO stuff but you can tell Reddit doesn't actually care to put the effort in to crack down on bots (however you'd do that) because they already don't give communities proper moderation tools/third party tools and the site does censor

Whatever allegiances (with people, or allegiances to ideas) Steve Huffman has, or people like him - it's not enough. It's a site seemingly killed by greed

(Yes, I know moderating this stuff at scale is hard)

- A human. Beep boop.

> I do know for a fact that many "users" here are LLMs.

HN autokills comments it detects as LLM. I think maybe you're not giving HN enough credit. :)

The question is how reliable that detection is.

It needs help. I often pipe my screed though an LLM and post it. I do request that it use a 10th grade reading level, and no emdashes.

For giggles, here's how it would look for this comment. Rather meta, but in this case it removed the "It needs hellp" so here we are.

I often run my screed through an LLM before posting. I ask it to keep the writing at about a 10th grade reading level and to avoid em dashes.

I have read enough “you are replying to an LLM” comments that I am pretty sure this is still a hit or miss process.

Why do you think those comments are accurate? Maybe those comments are by LLMs? If you believe crowd wisdom on its face, you will have big problems with LLMs.

HN front page is about 25% LLM written blog posts at any given time.

There’s no rule against submitting LLM-written—ahem, “cleaned up my notes”—articles, just comments.

Badly-written articles are still unwelcome on HN, wether AI-assisted or not, and obvious LLM smell definitely lowers the quality of an article. But it's true, we don't ban every article with any evidence of AI-assistance.

>HN autokills comments it detects as LLM

No it doesn't. Unless you have proof.... ???

HN kills lots of posts. I try to be careful about my online footprint (since HN posts are forever), and try to switch to new accounts every so often. It's no use anymore, HN just kills any post I make from a new account, even when I spend 20 minutes researching a response and trying to get useful information.

It doesn't even show you the post is killed, it looks to you like it posted fine, and you have to logout to see it's actually dead. It's an approach that's extremely hostile to the user.

It's specifically against the guidelines to keep registering new accounts, and this is a good reason why. We have to have ways of determining credibility and authenticity, now more than ever, and a track record of good posting is one of the best ways to do that. We are drowning with spam and low-quality posts/projects posted from brand new accounts. If it's a well-researched, high-quality post, of course we want to give it exposure. We just have to be realistic about what we're up against.

There was a post today that Google introduced unbreakable capture that required unrooted phone to pass its QR code.

We may end up with things like that…

> I do know for a fact that many "users" here are LLMs

What factual basis do you have for that?

> As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer.

I don't suppose you could show some examples? How convincing is the state of the art now?

Unless their account is <1 year I wouldn't assume they are a bot.

So easy to purchase online accounts nowadays, neither karma nor age of the account matters anything anymore.

Reddit astroturfing firms and bot farms learned to buy/use “seasoned” accounts over a decade ago. I’d venture there have been countless bots just in a holding pattern harmlessly building up reputation and a human-like history of posts across different subs etc just to eventually be either activated or sold to someone else to “burn”

So what is the comment frequency of these bots? There must be some signal in the activity even if the comments themselves pass the turing test.

Even if there was, I doubt Reddit cares enough to go after them when it’s boosting their valuation

Does it matter? With enough you can just have them upvote each other.

If you find one account you can find a few dozen spam accounts by building a graph of what posts they reply to

Most of them have private profiles these days

There are alternate sources that index reddit comments, but it's really frustrating how easy it would be for Reddit to shut the most prolific bots down if they were so inclined

I recently spotted one unmistakable example of this[0]. It’s been a trick for many years now that duplicating a human post and its comments is a good way to appear human but this was quite the example.

0: https://wiki.roshangeorge.dev/w/Blog/2026-01-06/Is_The_Inter...

> duplicating a human post and its comments is a good way to appear human

Also just repeating something from the linked article, but often with different wording and in a tone that makes it seem like it was something that the article missed.

It used to be super common that when you spotted a bot post and clicked through to the user's history, you'd see very average, human-looking activity from years ago, followed by a long gap of inactivity, and then a flurry of obvious bot comments.

It's very obvious that these accounts were abandoned and then either bought from their original owners, or more likely bought from someone who compromised them, because of their history and karma.

And I would bet money that Reddit is well aware of this phenomenon, because not long after it became so common as to be impossible to ignore, they papered over it by allowing users to hide their history from public view. (AFAIK subreddit moderators can still see it, but typical users now have much less ability to see whether they're interacting with actual humans.)

That and locking down the API meant no more sites offering readily available visualizations of this type of thing

> allowing users to hide their history from public view

Yeah it's become my default assumption that any user who does this is either a bot or a bad-faith troll.

IRL communities have to have some guides because a lot of people forgot how to gather. It can be seen among kids - try to give them soccer ball and see what they do with it :)

Yesterday I was watching people on the street and on the tram. Every other person was staring at their phone and scrolling through something.

That might scare me more than the fact that someone is chatting with an LLM bot online.

(I am pro-ai, use it every day for coding that I couldn’t achieve pre-2022 as I am lame coder.)

I don't really see the problem of using your phone while commuting. Doesn't make you an asocial weirdo.

As long as you're not sharing the things you're watching with a loudspeaker. And that's really not a given among commuters.

So you ran an "experiment" where you deliberately made someone else's community worse to see what would happen? Cool project.

Reddit users are definitely smartening up on many subreddits to the same kinds of engagement happening.

I wonder, how much of the discussions on the results of agentic coding is just LLM slop.

On the other hand, I’ve been accused of being AI/bot and if I say things the mod doesn’t like and is not their favorite thing to hear I’m “flamebaiting” or engaging in personal attacks when pointing out specific things.

Frankly, online communities have been doing for many years now, when the censorship, anti-free-speech, tone policing mods and mobs started dominating online and America really did not have the self-respect or confidence anymore to enforce the Constitution online.

> America really did not have the self-respect or confidence anymore to enforce the Constitution online.

“Mods are Unconstitutional” lmao

> On the other hand, I’ve been accused of being AI/bot and if I say things the mod doesn’t like and is not their favorite thing to hear I’m “flamebaiting” or engaging in personal attacks when pointing out specific things.

Mod powertripping nonsense happens, but banning people is a very easy way to kick up a small shit storm so most of us generally avoid it unless somebody has repeatedly been an issue or just flagrantly breaks the rules.

I’m not saying you aren’t telling the truth, but I’ve seen and heard this exact story a thousand times and the few that actually end up sharing proof (most refuse to) of what happened had a very skewed idea of how things went down.

How do we know now that this comment wasn't written by LLM?

You don't and that's the problem :)

> where I had an agent karma

Was this a browser using agent? What did you use?

It used the browser agent to grab user cookies after signing in, then made API calls iirc.

Using just a browser is way too token intensive and slow. It would look for 401 errors then run the browser automation to login with the credentials and grab the token.

I'm surprised these platforms don't have advanced heuristics to detect API calls and inauthentic traffic.

Did you clone the Reddit API from browser traffic and then turn it into a 100% API driven thing?

I'd imagine they'd be sniffing browser agents, plugins, cookies, etc. to fingerprint. Using JavaScript scroll position, browsing rate and patterns, etc.

Maybe their protections just aren't that sophisticated.

Reddit is known to fingerprint TLS and quickly shadowban accounts that don't have the fingerprints of browsers.

TLS fingerprinting and Cloudflare are easy to bypass. There are lots of libraries that do so.

The application-layer stuff is harder. Each application can develop its own heuristics, and that's difficult to automate in a cross-cutting fashion.

Reddit doesn't do anything about that? That seems stupid.

> TLS fingerprinting and Cloudflare are easy to bypass. There are lots of libraries that do so.

Easy for you does not mean easy for everyone. My experience is that TLS fingerprinting paired with blocking specific user agents gets a variety of majority of bot traffic.

It's the same a basic online security: You can protect against script kiddies with basic hygiene. If the threat analysis is Mossad, then yeah, you're fucked.

I feel you. Especially in the larger subreddita. i participate, and mod, a few small ones, and the community there is pretty strong and folks shut down ai slop pretty quickly.

I'm not saying being a mod means it's bullet proof, but i do notice smaller communities tend to self police better and know what's real.

That said, your experiment scares me as well.

I will say that I believe you probably have absolutely no idea because it's not "slop". It looks like every other reddit comment you see.

My experiment was focused on niche subreddits as well due to the nature of the product I was trying to market.

Do you have an example of comments people engaged with?

It's easy to botspam Reddit because even the real users always acted like bots. The big subreddits were the worst, but contrary to how the users keep saying "it's good if you find the right subs," no it's not. Wrote that place off like 10 years ago.

Communities in FB, WhatsApp, Telegram etc are actually flourishing. As it appears real time gated communities are doing fine.

It’s an unpopular opinion but I am looking forward to ID and age verified social media. If done right we can have real people around again.

BTW, ironically the harsher communities like 4Chan doesn’t seem to suffer from the dead internet. I guess it’s either because the advertising value is too low to justify AI use there or maybe AI API providers refuse to work with such a content this reducing opportunities to infest with bots.

> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs

People using LLMs without being fed their own post history are still pretty easy to detect. There's just something very recognizable about the cadence and tone of LLMs.

What really stuns me is that if you call someone out for it, 9/10 times you get absolutely buried in downvotes. Even here on HN. Its like people are angry that you're lifting the curtain on the slop, that the writing they enjoyed is fake.

> I do know for a fact that many "users" here are LLMs.

Name and shame.

If you look at the bottom of most threads here you’ll see a bunch of green username dead LLM comments. Those are just the obvious ones though.

More of a philosophical question but if you have no idea whether it's a human or robot, does it really matter? Personally I dislike AI slop only when I can tell it is...

Yes, for a number of reasons:

- I am trying to learn about the topic at hand and trust a human's comment more than an LLM's guess - I am trying to connect with other humans to fulfill my social needs - I am maybe spending time to help another human out with a response because I want to help someone else - I am interested in the perspective of other humans

Those are just a few reasons. For each of those if it's actually an AI I feel I'm losing out on something.

I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political". This is somewhat related to the Overton window but really a bunch of (mostly conservative) ideas get normalized so aren't deemed "political".

I see the same thing with "AI Slop". Yes, there is AI Slop but (IME) it's pretty easy to spot. But what's more annoying is how often people are willing to throw that accusation whenever someone takes a position they don't like, much like the "political" label. It's lazy and honestly just as bad as the slop itself because it unintentionally launders the slop in a "boy who cried wolf" kind of way.

I also have a theory that some AI slop isn't inherently successful. It's just heavily botted by people who are interested in promoting certain positions. I bet you could make a pro-administration LLM bot and another one promoting a communist revolution and no amount of model tuning would make the second as popular as the first because the first would hit third-party botting as well as platform content biases (eg Twitter).

I've personally been accused of being a bot. This is particularly true in recent time as I've tried to share facts and fact-based analysis of, say, what's going on with crude oil markets, the military operation in the Gulf and the politics and economics around it. I even saw one hilarious comment saying (paraphrased) "the bots are getting clever and posting about unrelated topics". This was funny because it never occurred to this person that no, it was just a real person posting something you disagreed with.

> I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political".

This happens on HN all the time. For a lot of downvoters and flaggers, there are two kinds of opinions: "Things I agree with" and "Too political for HN."

And yet if that was entirely the case, HN wouldn't be such an interesting place. This sounds very much like someone making fun of the attempts of rationalism/effective altruism "because humans are not rational but emotional beings". Yes, it is mostly true but not entirely true and that's what makes a difference.

I am certainly guilty of downvoting (not flag) people I disagree with but I also read the guidelines and will downvote or flag a comment that seems to originate from one of my ingroups that would violate those norms because I value this place.

And I have also been able to nuance my own opinions by reading a well-behaved (respecting HN norms) comment even though I don't like its vibe.

I didn't see the flag system being abused. The flagged comments I have seen were most egregious norm violations. Whereas posts are generally flagged when they inherently highly flammable. It's generally a edge case and some of those make it through with a little moderation warning as top comment.

I find it amusing that this is the top comment. Reddit is so awful you finally wrote it off, but not before you used it to try to “karma farm and do some covert advertising”. It’s on-brand for HN hypocritical bullshit. But, since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard, have an upboat fellow traveler.

> since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard

Same as it ever was.

Dead Internet theory ?

> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.

This just makes me wonder...so what?

Some of the oldest posters here with the most karma continue to post absolute garbage takes on topics ranging from US healthcare to history of USSR, that are trivially disproven by learning the very basics from a Wiki article (e.g. not a high bar).

To be fair, this opinion slop is also present for new users and LLM bots, but is one kind really worse than the other, if both of them contribute to killing the community?

We already know what kills communities. It's the eternal Septembers. Infighting within leadership also doesn't help, but time and time again it's the influx of too many new users that nosedive and drown out quality contributions.

An irascible human being with "wrong" opinions is still better than a polite and factually correct bot because there's no fucking point in having a conversation with a bot. We're here to have conversations with people, not to prove fact beyond a reasonable doubt.

Do you really not care one way or the other? Would you really rather just be talking to LLMs here? Or would you just script yourself as well and call it a day? Then what?

> We're here to have conversations with people, not to prove fact beyond a reasonable doubt.

Maybe you are. I like getting to a reasonably correct model of a topic or issue. Bad human takes can still be useful here. I just get inevitably tired of the people crying about potential LLM comments all the time.

> Would you really rather just be talking to LLMs here?

Obviously we're not there yet, regardless of what I want. But there is a great number of HN threads posted here that touch on topics that have been discussed so many countless times, that an average LLM summary would do better than most comments.

Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”, and then actually spend time reading them?

No? I’m imagining not at least. Because there would be no point to it.

If you would enjoy it, then I’m surprised you’re here and not just simulating the experience with your LLM by yourself.

> Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”

The reason I'm not simulating the experience with an LLM is because:

1. It costs more time to do so, because I have to prompt it to create a single comment. Multiply that by the typical number of an HN thread.

2. I suppose in a way you need bad takes to form your own view of a topic or an issue. LLMs would also be unable to provide truly unique experiences, such as some of the veterans who sometimes post here who were part of the living computing history as we know it.

> I’m surprised you’re here and not just simulating the experience with your LLM by yourself.

That's something you imagined that I claimed I want. If you read my comment again, you'll see there was no such thing.

> That's something you imagined that I claimed I want

This is a thought exercise. I phrased it as "If you would enjoy it", and you clearly have stated reasons why it would be insufficient, so, it doesn't apply to you. If I say "if X then Y, else Z" and you say !X, then of course, !Z applies.

But your original post says "so what" if the people posting here are actually LLM's. And you just answered that question above: "LLMs would also be unable to provide truly unique experiences." So that's what.

To continue the argument I'm making: We're all here because we want unique experiences talking to other humans. None of us would ever think it's ok if this site was just LLM's instead. If we did, we would open up our favorite LLM, not open hacker news. Right? Right. Seems like you agree.

Unless you've discovered the secret sauce, LLM comments are very obvious. Even Altman revealed that they focused on coding at the expense of writing.

People that like to fancy themselves as good llm content detectors just end up accusing everything they don't like as llm content.

The only thing worst than a slop comment are the people that bitch about it incessantly. I'm convinced it's become a new expression of a mental illness.

The main thing I suspect of being LLM written is the sort of LinkedIn style: very short sentences, overly focused on sort of… making an impact on the user. But that’s also how a certain type of bad human writer writes. So in the end, I’m not sure I know if anything in particular was written by an LLM.

I guess… “that’s not just an AI red flag, it’s generally shit prose” would be how ChatGPT would describe most things nowadays.

It’s the distilled mediocrity of the statements. Never venturing beyond a 10% margin of what you would get if you sampled the opinions of 1,000 people who underwent jury selection by west coast liberals.

A mere opinion is not mental illness.

I wasn't suggesting you have a mental illness for having an opinion.

More, commenting that just as bad as generated content if not worse is every thread where the top comment is an accusation and ensuing witch hunt.

So, no, having an opinion is not a mental illness. Feeling compelled to call it out and discuss it on everything one reads may just be.

The threads that have the top comment saying "this is AI slop" are nearly always about an article that is obvious AI slop.

Threads that aren't - like this one - don't.

If you need to tell yourself that in order to cope that's fine with me.

I’m thinking that I may actually prefer undetectable AI slop to human comments like that. I do agree with your upthread comments.

Which part do you disagree with?

Was that written by an LLM? It isn't that it's a mere opinion, it's that when every word out there has to be scrutinized for the possibility that an AI output it instead of a human intelligence that it gets pathological. Am I an LLM with the right prompts set up to respond this way? I mean, I know I'm not, but everyone else out there is just going to have to trust me that I'm not.

The obvious ones are the ones you notice

LLMs are not good at writing. If they were we would have entire libraries of new, amazing literature.

Neither are most humans

Agreed, some humans are good writers, and no LLMs are good writers.

This is rather moving the goalposts from "plausibly human comment" to "meaningful literature", I think

No. I'm drawing it out to its logical conclusion.

It’s poor logic, a non sequitur. An absurd reduction. By your argument anyone who hasn’t written a great literary work is a poor writer, and would be bad at writing online comments.

LLMs aren’t lacking in the sort of writing skills that make for superficially good content. They know grammar, they know rhetoric, and they know their audience. You can’t tell them from a human on their writing skills. Where they tend to fall down is their logic and reasoning skills, and unfortunately it seems you can’t use that to distinguish them from the average online opinionator either.

No, that is a mischaracterization of what I wrote. They are great writers if you enjoy formulaic writing.

Exactly, they aren't good at creating new material. But many discussions in comment section are simply regurgitations of existing material, which they are good at rearranging. New novel discussions in places like this are actually a very rare thing, as many comment sections are simply people who already know informing those who don't. I'm doing that right now, funnily enough.

No, they aren't even good at rearranging existing material. They produce bad writing that only superficially looks good in a lowest-common-denominator sense, and falls apart under any close examination. Everything is wrong with it, from the sentence structure to the rhetorical forms to the substance. AI 'writing' is a loose collection of cheap tricks that score well on A/B.

Precisely. People here are claiming that everyone is being fooled and that LLMs are amazing at casual writing. It's very silly.

I have worked with LLMs for a couple years at a very non-technical level and it was not that difficult to give it proper prompting and reference material.

If you are reading LLM content just about everywhere and have no idea. Obviously there are easy to spot things, but the stuff you don't spot is the stuff you don't spot

With the current batch of SOTA models, it is not hard to prompt a model to pass the sniff test on social media forums. If you don't believe me, try it.

All you really need to do is give it some guidelines of a style to follow and styles to avoid. There's also a bunch of skills people have already written to accomplish this.

Reply on news.ycombinator.com