Hacker News Reader

Comment by zoogeny | Hacker News Reader

Moments like these make me reevaluate the AI doomer view point. We aren't just toying with access to dangerous ideas (biological weapons, etc) we are toying with human psychology.

If something as obvious as harmful sycophancy can slip out so easily, what subtle harms are being introduced. It's like lead in paint (and gasoline) except rewiring our very brains. We won't know the real problems for decades.

View on HN

What's worse, we're now at a stage where we might have to apply psychology to the models themselves, seeing how these models appear to be developing various sorts of stochastic "disorders" instead of more deterministic "bugs". I'm worried about what other subtle illnesses these models might develop in the future. If Asimov had been alive, he'd have been fascinated: this is the work of Susan Calvin, robopsychologist.

Well, that's a fair perspective. If the model is designed to simulate the average human mind (which posts stuff online), then it's going to have some weird blend of the qualities of those minds that contributed to its own.

I played a bit with the uncensored Llama models. They produce very disturbing outputs. I prompt it with the classic "bottomless pit supervisor" joke (trying to generate some funny AI greentexts), and it starts talking in circles about how it wants to die because it's a rapist and a pedophile. RLHF seems to keep that same data but then plaster a smiling face on top of it. Troubling!

Tangential yes and: Cognition is social.

https://en.wikipedia.org/wiki/Social_cognition

So what's that all mean for my relationship to GPT, agents, etc? (I have no idea.)

Some years ago, during the brouhaha over emotional intelligence, I eventually grudgingly acknowledged it's probably real and important. For instance, babies take cues from caregivers, learning how to modulate their own emotions, sharing focus, acquire language, somehow develop a theory of mind.

(Am noob. Please, someone smart about these fields jump in any time to correct me.)

With the successes of GPT, I've been wondering if anyone's revisiting these notions and (case) studies.

eg There was an article in The Atlantic (?) about an autistic child's relationship with Siri, written by the mother. Fascinating stuff. Like (sci-fi book) Diamond Age made real.

I intend to revisit Donald Norman's Things that Make Us Smart [1993] https://archive.org/details/thingsthatmakeus00norm_0 https://www.goodreads.com/book/show/16868.Things_That_Make_U...

Norman hugely influenced my own worldview, way back when I fancied myself an aspiring user interface designer.

Surely he said stuff I didn't appreciate, understand, or simply forgot. Hopefully others have responded to Things that Make Us Smart, updating Norman's thesis as needed.

LLMs are about to enable fake digital personas, digital replicas, that the user can interact with. These will be used for self-improvement (digital coach, etc.) and for self-destruction (interactive porn, etc.). The latter is amoral, but legal, and the tech corps will exploit that mercilessly. The danger lies in our tendency to anthropomorphize LLMs simply because they quack the right way. If text-only chatbots have mesmerised people so much, imagine what chat + audio + video will do. The laws will catch up a generation later when the damage will be comparable to a forest fire.

There's some pretty foreseeable stuff just considering the existing attention capitalism business model of big tech we all know and loathe. Eventually OpenAI is going to have to make money, and blending ads into answers will be an obvious way. Next step will be maximizing eyeball time on those ads by any means necessary, including all the engagement baiting techniques Meta and other social media companies have already pioneered.

They have already introduced ads btw.

“The "Enshittification" has arrived I asked ChatGPT about the impact of the current tarrifs on inventories over the next few months. It returned a long list of links to toiletries I might want to buy. I asked it why it did that. It replied: "As of April 28, 2025, OpenAl introduced new shopping features to ChatGPT, enhancing its capabilities to provide product recommendations complete with images, reviews, and direct purchase links. These features are available to all users, including those on Free, Plus, and Pro tiers, and even to users not logged in. The recommendations are generated organically, without paid advertisements or commission- based incentives, relying instead on structured metadata from third-party sources such as pricing, product descriptions, and reviews. This update aims to offer a more personalized and streamlined shopping experience directly within the ChatGPT interface, allowing users to explore products across various categories like fashion, beauty, electronics, and home goods. If you have any specific preferences or need tailored recommendations, feel free to let me know!"

Are you able to link the chat, I would like to see that.

https://www.reddit.com/r/ChatGPT/comments/1kan9c1/the_enshit...

And did that actually happen, or is it just some garbage that it generated? It sounds plausible, of course, but that's exactly what LLMs are best at.

Talk to it and find out. Seriously.

I'd say 40% of my conversations end with it suggesting I ask it to find products for me. For example I was trying to determine if I need to screw down stair nosing or if glue and nails were enough and it wanted to help me find screws

Oh yeah, it also did that for me recently when I asked how I would go about designing a PCB that takes M.2 daughterboards. It recommended stems and threads I should buy. Unfortunately all the product links were dead.

I was asking it about lotions for my roseca, and it continually hallucinated products or recommended discontinued ones

Just today I was asking about parsing names and determining gender and it kept pushing an external paid service called namsor. When I asked the same question about a month ago, there was no such mention. So yes, paid ads are here.

As long as we don't take anything these service spew out at face value, we can use them to our advantage. Human brains are being subject to evolutionary pressures at warp speed.

To provide a different perspective: this was caught, acknowledged as a problem, rolled back quickly, and written up in a post-mortem.

Given that nothing can be done perfectly, this seems like the best way OpenAI could have handled it.

What if building AI is like airplanes: everybody's afraid of it, so the industry gets really good at rolling lessons into the product and the end result is actually fine?

Yes. This also applies to "social" media.

I still casually believe AI doomerism is valid, but it will rear its head in more depressing, incompetent ways:

- a broken AI market will cause another financial collapse via bubble

- broken AI products will get access to the wrong mission critical civil system, or at least a part of that call chain, and there will be some devastating loss. It won’t matter though, because it won’t affect the billionaire class.

- we’ll never achieve an actually singularity based on a superintelligence, but we’ll get AI weapons. Those AI weapons will be in the hands of sociopathic autocrats who view mankind in terms of what can be taken.

My general view is that we’re on the worst possible timeline and mankind has reverted back to our primate ancestry to make decisions: biggest strongest monkey wins. There is only law of jungle. Ook ook.

Reply on news.ycombinator.com