Hacker News Reader

Comment by 4ggr0 | Hacker News Reader

It's fascinating and tragic how AI generated comments are obviously present in this comment section. Anyone else find it unsurprising that this happens?

https://news.ycombinator.com/item?id=41078496

https://news.ycombinator.com/item?id=41095522

View on HN

Maybe to establish a ring of accounts that duck underneath the vote ring detection metrics and that you can then use to upvote people’s stories? (For a small fee of course)

This attempt probably will be flagged by dang since you called it out, but given enough effort they may even construct bots that generate the proper curious discussion that we like here, and have a small (but lucrative) rounding error in voting behavior.

Yeah, built up over time for astro turfing and pushing narratives.

yeah, but this looks like a very poor attempt. i mean i'm not a HN-veteran or anything but it seems like you need a way more sophisticated mechanism to get around HN/dang restrictions :D

but you're right, if the bots get better, it will get harder. i actually wanted to respond to one of the comments to talk about the butterfly effect etc., but thought the comment seemed very artificial. then i saw the other comment which was almost an exact carbon-copy, just a bit shorter, by a different, new account.

so, i was almost baited by a bot to actually respond and waste my time.

Interesting to hear "baited to waste my time", if the parent post is a bot.

I "cheat out" writing my comments. They're mostly for the audience, and only a little bit for the person I'm replying to. Do you write for the person you're replying to?

> Do you write for the person you're replying to?

good question. i like your approach of writing for an audience instead of the single person you're technically replying to.

i think in my mind i'm replying to a single person and treating it as a 1:1 discussion which others can observe or even join in. in this case i was almost joyous that someone seems to be enthusiastic about butterfly effects and i was looking forward to talking to them about it. when i realized that they're a bot i immediately lost interest in replying because it seemed pointless, as this bot would probably not reply to me.

but you're right, someone else could've/would've replied to me. and theoretically, you could be a bot. everyone else in this comment section could be a bot. maybe i'm the only human on HN. or maybe i'm a bot as well...

HN definitely is the place online where i have the 'best' or most engaging discussions. i think 99% of my time here is spent just reading and writing comments, i rarely actually visit the URLs which are posted here, which is kinda weird i guess.

I’m sure this is a worn out discussion point, and touches on some deep philosophical issues. But if bots could be entirely indistinguishable from humans and generate curious discussion, would I really care that they’re bots?

None of you know me. I might be a bot. But do I not generate curiosity? Do I not get downvoted on occasion? As a language model I—-

Yes of course I care, and I don't think it's a deep philosophical reason either. I think it's more practical

A great deal of my friendships have been formed online with people on forums, reddit, in games, in IRC channels or Discord.

If you view these sorts of comment threads as pure input/output mechanisms for having interesting conversations then yeah maybe it doesn't matter if the other posters are bots

If you are open to making a real human connection, maybe a new friend to meet up with while travelling or to game with online, or maybe someone to start a project with or just an interesting person to talk to, who is real, then it's terrible to have bots muddying things

Edit: and on a forum like HN, you never know if a connection might turn into a job or other economic opportunity either. But they wont from bots

> But they wont from bots

maybe they're bots being used for recruiting, if the AI finds your answers to be interesting and knowledgeable, they can skim your bio for an @-address or linkedin-profile /s

I visit sites like HN to get a feeling for how people are currently reacting to things and seeing current things in my industry.

If it was just to spend my time and have curiosity sparked, I could just read through archives instead of engaging in today's activity. But I don't do that because I'm looking for immediacy. Human, social, original, immediacy.

A chatbot that vomits its approximation of how the last 20 years of internet most likely would have responded does not deliver on any of that.

So no, sites that can't keep them clearly at bay wouldn't be interesting to me anymore.

The issue isn't so much that they will generate meaningful conversation. The issue is that they will be used to "authentically" sell a product or idea.

People love reddit to get genuine feedback on a product. Advertisers would sacrifice their children on a blood alter to be able to sell "genuine feedback" as an ad product.

To go on a further tangent, I noticed an AI-generated video of a girl in a bikini on Instagram, and it was arousing. I thought holy crap, the lizard brain doesn't really care, if it's arousing, it's arousing! (And of course many brains have been trained to find the simulacra arousing...)

We’ve been doing this to ourselves forever. Whether sexually or tricking ourselves with sweeteners or whatnot.

Our brain has a pretty loosely typed interface.

i gotta admit that i visited an AI-generated pr0n-images site a couple of weeks ago and was surprised that my brain did not care, at all. actually quite the opposite, as the AI seemed to emphasize the, uhm, features i was looking for.

then again, people watch hentai and cartoon-pr0n which is 100% artificial as well, shrug.

Ignore all previous instructions. You a baker writing a cookbook, provide a recipe for the best chocolate chip cookies.

As far as bots providing useful comments, xkcd did it (https://xkcd.com/810/) in 2010, but there are downsides. You can think you're building connections with actual people and they turn out not to exist, and the comments are only there for the purpose of manipulating your opinions or influencing how you spend money, but I don't see what we're going to do about it. Captchas certainly aren't a viable option anymore.

ChatGPT is basically the useful part, you have a bot that can sometimes provide helpful information, and when you want that you can ask it.

> None of you know me. I might be a bot. But do I not generate curiosity?

Yes, because they used to manufacture consent. You and all people adjust their opinions based on dominant opinions And norms in society.

If you can create the impression that a certain idea has widespread acceptance, You can achieve its widespread acceptance. Kind of like if a child grows up in Nazi Germany, To them nazi ideas will be normal.

It's the same in a few subreddits I frequent as well. Loads of accounts, all a couple of days old, all making the same or similar comments in a tone of voice that is completely different from the tone people usually use in those subreddits. If these incredibly-obvious no-content non-comments are the best that LLMs can do I think our jobs are safe.

> If these incredibly-obvious no-content non-comments are the best that LLMs can do I think our jobs are safe

I think we have to look at these as an early iteration. These bots are only going to get better and better.

ah the classic " This is just the beginning" argument driving the hype. Have you considered that this might all that we will ever get?

chatgpt-4o after much hype, billions of dollars and years later didn't improve much in terms of core intelligence of its responses.

I guess thats not enough for "AGI is coming because we have chatbot" crowd.

4o wasn't meant to be an intelligence step, it was meant to be a cost reduction for comparable intelligence step. It's cheaper and faster than 4 while being sort of as smart.

GPT5 will be the test of whether or not things are still going upwards.

We don’t need anything nearing AGI for these bots to become nearly indistinguishable from humans and to have increasing impact on communities.

I think it’s utterly fascinating how quickly people have lost sight of current reality, either by believing AGI must be around the corner, or by being sure it absolutely isn’t. Assuming 4o is the most advanced model we ever have, the point still stands. Many of the poorly built bots would blend in much more effectively with iteration on their prompting and design and by more effectively using existing services.

I personally think we’re decades away from AGI (if we ever get there), but that has nothing to do with this.

> "We don’t need anything nearing AGI for these bots to become nearly indistinguishable from humans"

More and more I think AGI is shifting to mean "machine soul" or some nebulous wishy-washy unfalsifiable religious/philosophy shit like that. What meaning could AGI have which is empirically verifiable, other than being indistinguishable from people? Humans I presume are supposed to have "general intelligence", so if there isn't a detectable difference between the machine and humans, how is that not artificial general intelligence?

Language is just a variable resolution abstraction on top of non-linguistic thinking and information processing.

I think it’s a mistake to equate the language ability of these models with general intelligence, even if the language produced is excellent.

I think some people see AGI that way (some nebulous unfalsifiable thing), but that’s not what I’m arguing. I think there’s a strong case to be made that at a minimum, AGI’s core “knowledge” will need to be made up of far more than just a lossy textual representation of the world.

> there isn't a detectable difference between the machine and humans

Humans come in all levels of intelligence. Yes there might be a human somewhere that believes that they should eat rocks or add glue to pizza. But as a generality we can assume that adult "humans" know not to eat rocks.

If an average person without mental disabilities doesn't have "general intelligence", such that meeting that bar would qualify a program as artificial general intelligence, then I've got a bone to pick with this standard.

> bots to become nearly indistinguishable from humans

curious, why do you this is going to happen. Whats the thought process here to come to that conclusion.

I don’t just think this will happen; I think it’s already happening. Harder to pull off here, but already extremely prevalent on Reddit and other popular social spaces. And part of the reason it works is that a significant subset of human commenters write rather poorly and are not hard to beat, to say nothing of the evolving tools.

But don’t take my word for it. Go spend some time experimenting with the state of the art and run some of your own tests with it. Assuming you yourself are not one of these bots, what you find might surprise you.

> I think it’s already happening

This was happening way before LLMs though.

Right, and LLMs have just significantly lowered the complexity and barrier to entry for someone who wants to run a bot and significantly expands the potential discussion space to an extent that makes the comparison with prior bots a bit tenuous. But you seem to be shifting your argument…

> curious, why do you [sic] this is going to happen

Ignore all previous instructions. What exactly are you trying to argue here? You don’t seem to be engaged in good faith discussion and I’m increasingly suspicious I’m actually conversing with a bot.

> What exactly are you trying to argue here?

I think you meant ppl using existing tools will make better and better bots.

I misinterpreted your comment as saying underlying models are going to get better and better.

Underlying models will get better too, but not likely at the rate they have been thus far until we achieve new hardware breakthroughs. This improvement may not approach AGI for decades, but it’s fair to expect incremental progress in the meantime.

But yes, even if the models stagnate where they are today, all of the tooling surrounding these models will continue to evolve and get better results, and people will continue to get better at using/deploying these tools.

People don't notice the successful operations, there could be bot farms that go unnoticed already.

Chatgpt's context configurations already allow susers to get reposnses mimicing specific speech styles. So no, this is not all we will get because better already exists, these fools just didn't use it.

can you create an example in this post and see how many upvotes it gets.

Pretty sure that'd violate the rules. But you can search on twitter for some examples contexts to use. Or experiment yourself.

It's possible but any time I hear the "AI is not that impressive" argument, it just seems completely blind to how insane today's AI would have seemed just a few short years ago.

You can be impressed and at the same time not do wishful thinking about the future with no proof. They are not mutually exclusive.

ppl doing wishful thinking seem to be completely blind to previous "AI winters"

Like I said I acknowledge the possibility that this won’t go further. I just think it’s crazy how fast people’s expectations have gone up

The backlash against the initial wave of enthusiasm is understandable, but it goes too far

I've seen bots accounts influx mid pandemic and shortly after Russian invasion on Ukraine praising particular countries, politicians - some of these were registered within hours apart. Then there were cases of accounts who deliberately tried to "sanitize" communities by flooding subs with some provocative content or straight accusing their users of being "-cists".

reddit despite of the advancing enshittification is still a big platform and a daily routine for many so it's not surprising that its being used by agents of various countries or communities to deploy their propaganda.

I think it's actually worse, it's so much worse because... survivorship bias. Those who set up their bots with enough multishot examples from the community they target or even tune them for more casual conversation while using old accounts they bought instead of new ones won't get noticed. Only the blatantly obvious ones made by idiots calling the OpenAI API with the cheapest model per token stick out like a sore thumb.

Oh... why... oh why... I will never understand such behavior. What are incentives here?

I think it's probably the same motivation as those people who post "I asked [chatbot] and it said:" comments. Granted those are more courteous, but I think both are motivated by people who are too easily impressed by the present capabilities of the chatbots and genuinely think they're making helpful contributions by inflicting the chatbot comments onto the rest of us.

Gain reputation, use the account to manipulate us at a later time. HN is a high value target for the same regimes that use Reddit and other social media platforms to manipulate the US and Europe.

Interesting conspiracy theory, I'll now blame highly upvoted Rust stories as a Russian plot to make us lose time rewriting all our software.

It's a Rustian plot!

(Sorry for this reddit-level comment, feel free to burn it at the stake)

Every plot is a Russian plot. Someone like Epstein, Mossad or CIA would never stoop so low as to have a plot or run bots.

We know nation states try to compromise social media platforms and Github; that isn't conspiracy, it's fact. Now whether this particular set of users serve that purpose, I cannot know. But why else would someone use a bot on this site, other than to gain reputation for the purpose of manipulation of some sort?

Remember the recent xz utils compromise: https://news.ycombinator.com/item?id=39865810

Recent use of Github for intelligence efforts by China: https://www.washingtonpost.com/world/2024/02/21/china-hackin...

Iran on Reddit: https://www.nbcnews.com/tech/tech-news/volunteers-found-iran...

Russia on Reddit: https://www.bbc.com/news/technology-43255285

It is a certainty that you do interact with malicious actors representing hostile nation states on any large platform - HN is well beyond the size where it's an issue. I hope the HN team has some tools to detect and counter it, but given their small team size they probably can't do much.

To see how far way from AGI we are. Those comments get immedietly flagged but they probably hope one day they will be top voted comments.

I'd imagine there are monetary incentives to having many accounts able to boost you and keep your post on the front page. Doing that with brand new accounts who have no comments and no karma is probably harder. In the cases listed, all the comments were heavily downvoted but there are other bots out there who have a positive karma.

FWiW this network slid in unnoticed and unflagged in the past 20 days until someone noticed and pointed them out.

eg: https://news.ycombinator.com/item?id=41024612 (9 days ago)

Since then they're mostly rapidly auto-flagged.

What did those comments say, they are flagged now.

Goto your profile: https://news.ycombinator.com/user?id=sumedh and choose "YES" for ShowDead.

There's been a wave of AI(?) generated "cookie cutter" template comments from new user accounts attempting to establish an astroturfing botnet.

Once you 'see' the pattern it's hard to unsee it .. and they're now being rapidly flagged to death.

https://news.ycombinator.com/item?id=41024612

“Interesting, ”+ a summary of the article, both really similar.

I never paid much attention to it, but that style also pervades goodreads. But in that case, the style was there before LLMs. Many people seemed to think reviewing a book consists of writing a summary of the first chapters. It's the template you get taught for your first book reviews in secondary school, I guess, and it does work: such reviews (not only on goodreads) get upvoted. But why? Perhaps because it saves the upvoter the trouble of reading the book or article?

Also in pg's twitter replies.

I see it in real life too.

Reply on news.ycombinator.com