Hacker News Reader

Comment by shermantanktop | Hacker News Reader

ChatGPT has taken to saying things like “What I would do now is…” or “if I were you I’d…”.

I know these are figures of speech, but it reminds me that this thing doesn’t do anything, it doesn’t learn anything, it can’t try anything and find out. And yet it uses speech patterns drawn from real humans who can and do all those things.

View on HN

I had claude throw something like "the last time I did <x>..." at me.

They seem to be trying to pump up the "humanity" to keep people engaged with it, which really backfires with me.

You're right to push back...

Fair points all around — let me actually check rather than guess.

That's still slightly better than GPT's insanely dry and objective manner of speech.

Never forget…there is no “they”.

The most likely explanation is that those patterns were common in the training data the LLM is mimicking.

Listen! And understand. That terminator is out there. It can't be bargained with. It can't be reasoned with. It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead!

The terminator you talk about is a movie character. It might be built one day, it might not. It is also totally irrelevant considering there is a real terminator which can be described with the same adjectives and built in to every personal life that ever existed or will exist. Knowing how to deal with that seems more important than worrying about some specific scenario.

Reading these two comments has me thinking that it's really the same Terminator.

I feel your comment is deeply ironic... Just shy of self-awareness. Have you considered that the conceit of the fictional character is representing that reality?

I have no doubt it plays on the same fears - whether it was deliberately done so, I don't claim to know. My assumption was that in a discussion on AI, the comment about the terminator was not alluding to a literary metaphor but to a fear from the actual thing. There was no intent in downplaying the fear of AI - just adding a new perspective. I'm am curious to know what you think self-awareness has to do with any of this.

Claude does that too. “That always confuses me” or “I usually realize”.

Not only do these imply that the thing has a personality and preference, but also continuity and a life outside the chat window.

I had to add an explicit instruction not to impersonate a human, it was just too weird for me.

Disabling this feature makes you judge its performance less accurately —for people pretending to learn from their mistakes do the same.

I think it was the Aliens universe where the Soviet megacorp makes the androids blue despite their being basically equivalent to humans

I've noticed it commonly uses phrasing like "that's usually the next step" when I'm using it to design something that I can't find an existing implementation of.

Not a day passes with my LLM of choice making completely baseless claims about "many people", who supposedly share all my problems and solve them like the LLM proposes

I only use it sporadically, but I am always irked by it saying things like "I personally like to..." or "I prefer...". It does it so often, that I am convinced it's part of the system prompt.

When I ask it to tweak recipes and stuff, it frequently says stuff like "my favorite way to..." or "I really like [x]".

I have a viscerally negative reaction to a machine claiming it has a favorite anything.

Recipes! Dude, the only thing you eat is other people’s data.

I wonder why, out of the many things models definitely can't do, you choose "try" and "find out". Surely every time it proposes a solution and then gets possibly corrected by the human minder its "trying something out" and surely it can use tools like web search and code execution to "find out" stuff?

> it can’t try anything and find out

Talk to an agent. It definitely learns things. Maybe not the taste of strawberry but about what is really going on in the software you are building with it.

>It definitely learns things.

By the very way this technology works they can't learn anything after training. What you think is "learning" it's just a session log written back to the context when you resume the session.

And what is written when you wake up from sleeping?

It's absolutely not the same. If you think llms and brains works the same way you clearly don't know how either works.

For a LLM learning what you wrote your last session would be update the weights with the new relationships and factual knowledge created in the session. That doesn't happen. The weights are static and fixed after training. There's no online training in the transformer architecture or any variant. If the weights don't update, the network doesn't learn. Period.

It's literally called in-context learning. The fact that it doesn't go into weights and is retained only for a session doesn't mean no learning occurs.

Also agentic systems may choose or be instructed to retain some information between sessions in files and/or databases which is also a form of learning.

There are experiments with retaining session information in weights in some form of lora but there's no consensus if it's even desirable. There's a value in being able to start from a clean slate.

These things are a centrifuge seperating humanity into people who say things like "it definitely learns" and those who do not.

It can't do a thing. You tell it how to do a thing. It now can do a thing. That's the definition of learning. Your centrifuge separates people who are not afraid to believe in what is reality, from the ones that do.

My car can't do a thing. I operate it's controls. It then does a thing.

In fact, I put a brick on the pedal and it went off and did a thing all by itself. Spooky.

Great. You astutely observed that your car, unlike an agent, in fact, does not learn, because you operate it, instead of teaching it, how to perform a skill, by demonstrating how it should be performed.

How do you know that "real humans" do that and aren't simulacra? We know that it is physically possible to hook up a brain to simulated inputs, so perhaps you are simply living in a simulation.

Reply on news.ycombinator.com