LLM Problems Observed in Humans
embd.cc
embd.cc
As critical as I might be of LLMs, I fear that they already outpaced a good portion of the population "intellectually". There's a lower level, which modern LLMs won't cross, in terms of lack of general knowledge or outright stupidity.
We may have reached a point where we can tell that we're talking to a human, because there's no way a computer would lack such basic knowledge or display similar levels of helplessness.
From an economics perspective, maybe a relevant comparison is to people who do that task professionally.
And then there are some brilliant friends of mine, people with whom a conversation can unfold for days, rewarding me with the same rapid, incisive exchange we now associate with language models. There is, clearly, an intellectual and environmental element to it.
I feel similar about self driving cars - they don't have to be perfect when half the people on the road are either high, watching reels while driving, or both.
This article is from 2021 - https://www.iihs.org/news/detail/crash-rates-jump-in-wake-of...
The conclusion seems to be that if you _only_ smoke marijuana you're actually less likely to be involved in a crash than a sober driver, but if you combine marijuana with alcohol you're _more_ likely to crash (which, duh).
Obviously not totally conclusive, but interesting none the less. Anecdotally, coming from a high school where folk smoke and drove all the time because they couldn't smoke in their houses or on the street where they'd face police harassment, it was always the alcohol that got them nabbed for DUIs. It's anecdotal, but my anecdotes are many and I'm not sure I've heard of any one I've ever known crashing while just smoking weed.
So... maybe everyone should toke a little before they drive, sounds like they'd leave more distance between the cars in front of them, and go at a more relaxed pace, and not try to do any crazy passes of the people in front of them. Road rage is a very real thing in America, and the stereotype isn't of your typical stoner.
I assume that's not a randomised controlled study? Ie there's probably all kinds of confounders etc.
Perhaps idealistic, perhaps unrealistic. I'd still rather believe.
What we should focus is that are they more or less prone to accidents than actual humans based on amount of km driven.
Again, there are those Expert Drivers who love their manual transmission BMW because automatics shift in the wrong RPM range and abhor any kind of lane assist because it doesn't drive EXACTLY like they do.
But the vast majority of average people on the road will definitely get gains from lane assist and lane keeping functions in cars.
Maybe. But the organisations who would need the defense most are the some of the least likely to apply them.
Eg it was better run organisations that had version control early, and the worse ones persisted with using shared folders for longer.
And strong type systems like what Haskell or to a lesser extent Rust have to offer are useful as safeguards for anyone, but even more useful when your organisation and its members aren't all that great. Yet again, we see more capable organisations adopting these earlier.
It's kind of depressing. I just want the LLM to be a bot that responds to what I say with a useful response. However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things making it even more grating to interact with them which chews away at my normal interpersonal patience which, as someone on the spectrum, was already limited.
But long conversations are never worth it.
I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that was wrong. Not always, maybe not even most of the time, but enough that it does give me pause and make me double check.
Also, when you have an LLM that is too agreeable, that is how it gets into a folie a deux situation and starts participating in user's delusions, with disastrous outcomes.
It's not a question of whether an LLM should be agreeable or argumentative. It should aim to be correct - it should be agreeable about subjective details and matters of taste, it should be argumentative when the user is wrong about a matter of fact or made an error, and it should be inquisitive and capable of actually re-evaluating a stance in a coherent and logically sound manner when challenged by the user instead of either "digging in" or just blindly agreeing.
What if your subjective opinion is that you think life isn't worth living, how should an LLM respond to that.
But to answer the question, it depends on the framing - if someone starts the chat by saying that they feel like life isn't worth living then the LLM should probably suggest reaching out to local mental health services and either stop the conversation or play a role in "listening" to them. It shouldn't judge, encourage, or agree necessarily. But it would probably be best to cut the conversation unless there's a really high level of confidence that the system won't cause harm.
So much easier to just make it agree all the time or disagree all the time. And trying to bottle the lightning often just causes degeneracy when you fail.
This is something I have not experienced. Can you provide examples ?
do you have examples of this?
asking because this is not what happens to me. one of the main things i worry about when interacting with the llm is that they agree with me too easily.
Back then, what we actually wanted was for them to push back and argue with us.
And if they get the answer wrong, don't try to correct them or guide them, there is a high chance they don't have the answer and what follow will be hallucinations. You can ask for details, but don't try to go against it, it will just assume you are right (even if you are not) and hallucinate around that. Keep what you already know to yourself.
As for the "you are an expert" prompts, it will mostly just make the LLM speak more authoritatively, but it doesn't mean it will be more correct. My strategy is now to give the LLM as much freedom as it can get, it may not be the best way to extract all the knowledge it has, but it helps spot hallucinations.
You can argue with actual people, if both of you are open enough, something greater make come out of it, but if not, it is useless, and with LLMs it is always useless, they are pretrained, they won't get better in the future because that little conversation sparked their interest. And on your side, you will just have your own points rephrased and sent back to you, and that will just put you deeper in your own bubble.
The trick here is: "Be succinct. No commentary."
And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified"
Also generally the very first time it expresses any of that weird shit, your context is toast. So even correcting it is reinforcing. Just regenerate the response.
Last time I bawled out an LLM and forced it to change its mind, I later realized that the LLM was right the first time.
One of those "Who am I and how did I end up in this hole in the ground, and where did all these carrots and brightly-colored eggs come from?" moments, of the sort that seem to be coming more and more frequently lately.
It seems like they really figured out grounding and the like in the last couple of months.
Lacking a theory of mind for other people is not a sign of superiority.
Also, what big words? 'Proliferation'? 'Incoherent'? The whole article is written at a high school reading level. There's some embedded clauses in longer sentences, but we're not exactly slogging our way through Proust, here.
I usually have the opposite experience. One a model goes off the rails it becomes harder and harder to steer and after a few corrective prompts they stop working and it’s time for a new context.
It's a natural inclination for all LLMs, rooted in pre-training. But you can train them out of it some. Or not.
Google doesn't know how to do it to save their lives. Other frontier labs are better at it, but none are perfect as of yet.
And the world wouldn't function if everyone operated at the exact same abstraction level of ideas.
Paying for someone to put some effort into giving a damn about what you have to say has a long history. Hire a therapist. Pay a teacher. Hire a hooker. Buy a round of drinks. Grow the really good weed and bring it to the party.
And maybe remember that other humans have their own needs and desires, and if you want them to put time and energy into giving a damn about your needs, then you need to reciprocate and spend time doing the same for them instead of treating them like a machine that exists only to serve you. This whole post is coming from a place of reducing every relationship to that and it's kind of disgusting.
Is it?
That's not my experience.
This may not apply to you if you regard LLMs, including their established rhetorical patterns, with greater suspicion or scrutiny (and you should!) It also does not apply when talking about subjects in which you are knowledgeable. But if you're chatting about things you are not knowledgeable about, and you treat the LLM just like any human, I think it applies. There's a reason LLM psychosis is a thing, rhetorically these things can simulate the ability of a cult leader.
On the other hand, LLM's are just text on a screen. There are zero of the human signals that tell us someone is confident or trustworthy or being helpful. It "feels" like any random blog post from someone I don't know. So it makes you want to verify it.
The key difference is mechanism. For humans, wandering off topic or repeating a point usually stems from attention, memory, or social dynamics. For LLMs, similar-looking behavior often arises from architectural and statistical limitations — e.g., limited context windows causing context drift that makes the model lose track of earlier facts over long conversations. That’s been documented as a core technical barrier in conversational AI systems.
Likewise, what the article calls “persistent hallucination” isn’t just “being confidently wrong like a human might be” — it’s a systemic property of how LLMs predict the next token based on learned patterns rather than grounding in external truth. Hallucinations are literally outputs that are factually incorrect or fabricated even if they read fluently. That’s why engineering teams talk about hallucination detection, grounding, or external verification layers for production use.
Framing them as the same as human flaws risks obscuring the real challenge: deploying LLMs in systems where correctness, consistency, and adherence to explicit intent matter. Those aren’t just philosophical problems; they’re system-design constraints that require mechanism-aware validation and monitoring, not just metaphors to human behavior.
Are people still debating that? I thought it was settled by the time GPT-4 came out.
Not sure why we think normal evolution wouldn't just route around such problems.
> Why is it we can feel so robbed when someone tells us a story we just heard isn't true, and yet so satisfied at the end of a fictional novel? I don't know. I don't know.
-- Randy Writes a Novel
If the thing made by a machine is indistinguishable from the thing made by a human, the thing made by a human will be more valuable, simply because being made by a human is an opportunity for a story, and we humans like and value stories.
The book is following the annual Turing Test competition, in which, humans are chatting with AIs or real humans without knowing which is which and give them a score out of 10 for being most human and the AI that is "the most human" wins the competition. The twist is, not all humans get 10/10 for being human either - so the human that's the most human also wins a prize.
It's like getting a gorilla to fly an airplane, noticing that it crashed the airplane, and saying "humans sometimes crash airplanes too". Both gorillas and humans do things that fit into the broad category "crash an airplane" but the details and circumstances are different.
At this point LLMs usually beat humans at the Turing Test! People are more likely to pick the LLM as the human, rather than the human. https://arxiv.org/abs/2503.23674
Society itself may benefit from cohesion or from truth depending on circumstances.
No. The first paragraph explains it quite clearly, IMO: "While some are still discussing why computers will never be able to pass the Turing test, I find myself repeatedly facing the idea that as the models improve and humans don’t, the bar for the test gets raised and eventually humans won’t pass the test themselves."
The point is not that the problems exist more in humans now vs before. It's that they can be observed more significantly in humans than in LLMs (and moreso over time) if one cares to look because LLMs improve and humans do not on sub-evolutionary timescales. And perhaps our patience with them in humans is now diminished because of our experiences with them in LLMs and so people may notice them in humans more than before.
There, lack of interest from the person you talking to or you when listening. It’s because you have different interests. This is a human feature not a flaw. But it’s interesting to think that LLMs might have similar behavior :-)
“I’ll never again ask a human to write a computer program shorter than about a thousand lines, since an LLM will do it better.”
From my personal experience with ChatGPT it can’t even correctly write few lines of code. But i don’t use AI often. I just don’t find it that useful. From what i see it’s mostly a hype bubble that will burst.
But this is my personal opinion and my own observation. I could be wrong :-)
this whole post reads like it's coming from someone who sees people as tools to get what they need. the reason I talk to people when I'm struggling with a problem isn't for reference, but for connection, and to get my own wheels turning.
I'll grant that it's interesting to think about. now that LLMs exist, we're forced to assess what value human brains provide. it's so dystopian. but there's no other choice.
But this type of... conflict aversion... is definitely more common in LLMs than in humans. Even the most positive humans I know sometimes crack.
Perhaps the author is just gaming out a thought experiment, but I’ll just take it at face value. I am genuinely baffled by the obsequiousness some people display regarding LLMs. Let’s assume it really is a more powerful form of intelligence (ugh) and it “replaces” people, how do you think that ends for you?
You are trying to convince yourself that you’ve happened upon a benevolent god that truly, deeply understands you while staring into a reflection pool.
For example I can’t just feed it weather data from the past decade and expect it to understand weather. It needs input and output pairs with the output being human language. So you can feed it weather data but it has to be paired with human description of said data. So if we give it data of a rain storm there has to be an english description paired with it saying it’s a rainstorm.
Even for human behavior: we don't have that much data. The current datasets don't capture all of human behavior - only the facets of it that can be glimpsed from text, or from video. And video is notoriously hard to use well in LLM training pipelines.
That LLMs can learn so much from so little is quite impressive in itself. Text being this powerful was, at its time, an extremely counterintuitive finding.
Although some of the power of modern LLMs already comes from nonhuman sources. RLVR and RLAIF are major parts of training recipes for frontier labs.
[0] https://www.schneier.com/blog/archives/2006/08/security_is_a...
Some failures like that are simply human failures reproduced faithfully. Some are rooted deeper than that.
And yes, it's true that children don't get bored in the same way adults do, which often leads to repetitive behavior. Boredom is an important heuristic for behavior, it seems.
I mean literally saying the same thing again and again. Like "And then I played and then I played and then I played and then I played..."
A lot of LLM behaviors are self-reinforcing across context, and this includes small stupid loops and the more elaborate variants. Like an LLM making a reasoning mistake, catching it while checking itself, and then making it again, 5 times in a row.
The issue with evolution is huge portions of it just happen to exist and not kill the host before they breed. It could be a massive bug that if corrected could cause the host to breed and spread their genes far further, but evolution itself can't reach there.
> I don't really know what it's saying
It's saying that complaints about deficiencies in LLMs, about a fundamental lack of LLM intelligence, about how LLMs are just statistical machines and not really thinking, about how LLMs are incapable of learning from past experiences, about how LLMs lack any coherent epistemology ignore how very deficient humans are in many same exact ways.
> Does this somehow make LLMs better in our perspective somehow?
Better is a relative measure not an absolute one, so possibly, because views of LLMs are inherently formed in relation to views of the human brains they're modeling.
Think of the most terminally online drama you've ever witnessed: the hysterics people work themselves into over what (to outside observers) seems utterly inane and forgettable, the multi-page Tumblr or 4chan posts that become the sacred texts of the "discourse", and the outsized importance people ascribe to it, as if some meme, album cover, or Qanon drop is the modern incantation of the shot heard around the world.
The people wrapped up in this stuff tend to self-select into their own communities because if you're not involved with or amenable to caring about it, why should they spend time talking to someone who will just nod, go "huh, that's wild", and proceed to steer the conversation elsewhere? In their eyes, you may even be a weirdo for not caring about this stuff.
So when I read:
> I’ve got a lot of interests and on any given day, I may be excited to discuss various topics, from kernels to music to cultures and religions. I know I can put together a prompt to give any of today’s leading models and am essentially guaranteed a fresh perspective on the topic of interest. But let me pose the same prompt to people and more often then not the reply will be a polite nod accompanied by clear signs of their thinking something else entirely, or maybe just a summary of the prompt itself, or vague general statements about how things should be. In fact, so rare it is to find someone who knows what I mean that it feels like a magic moment. With the proliferation of genuinely good models—well educated, as it were—finding a conversational partner with a good foundation of shared knowledge has become trivial with AI. This does not bode well for my interest in meeting new people.
I'm imagining the more academic equivalent of someone who got wrapped up in Tiktok drama or Q nuttery but couldn't find a community of kindred souls and, frustrated with the perceived intellectual mediocrity surrounding themself, has embraced LLMs for connection instead. And that's just hilarious. If Silicon Valley was still being produced, I'm sure this would have been made into an episode at some point.
The bits about not generalizing and engaging in fallacious reasoning are also quite amusing since, while yes, the average person likely would benefit from taking (and paying attention in) a couple introductory philosophy classes, expecting all humans to behave logically and introspectively is fantastical thinking.
Yes, that is exactly the point of OP's post, that humans are on average quite bad at behaving logically and introspectively and exhibit the very same behaviors that we righteously fault AI for doing. And then OP provides a list of faulty human behaviors that are the same faulty behaviors people give as demonstrating that AI lacks true intelligence.
Meanwhile, AI continues to improve and the human species does not.
And the conclusion is that the fact that the rise of AI has made human faulty behaviors more apparent may creepingly tear at the social fabric.
Read the first paragraph again. It sets the framing through which the rest of the post is understood (as first paragraphs tend to do).
I find this exchange to be a funny example of the truth of OP's list, where the part which sticks with you is some finer detail of one of the examples while the thesis statement itself, the very explanation of the overarching point of the post, seems to have fallen outside of the context window.
I don't think the article said anything about statistics?
This seems to be a sort of Rorchasch test but looking at it again:
>This does not bode well for my interest in meeting new people
It really does seem to me the article is making fun of people who think this sort of article is on point.
There's a genre of satire where the joke is that it makes you ask "Who the heck is the sort of person who would write this?"
It could fit in that genre but of course I could be wrong.
I don't think I said or implied that it did. It's merely one of the many positions that people commonly (and defensively) take for why LLMs aren't and/or can't be intelligent like humans, ignoring that humans exhibit exactly the same patterns.
We really haven't got a grip on what intelligence actually is, but it seems that humans and LLMs aren't really in the same ballpark, or even the same league.
Because intelligence isn't a thing, it's a bunch of different things that some intelligent things have more or less (or none of).
This is why measures of intelligence always fail because we try to binary it which doesn't work. Intelligence is spikey. Intelligence scales from very small and dumb to very smart. But even the things that are very smart on a lot of things still do very dumb things. We also measure human intelligence as a function of all humans and LLM intelligence on a particular model.
So yea, this is why nothing seems to make sense.
Well let's take a look at this:
>The best thing about a good deep conversation is when the other person gets you: you explain a complicated situation you find yourself in, and find some resonance in their replies.
>That, at least, is what happens when chatting with the recent large models.
The first sentence says a good conversation is between two people. The author then pulls the rug out and says "Psych. A good conversation is when I use LLMs."
The author points out humans have decades of memories but is surprised that when they tell someone they are wrong they don't immediately agree and sycophantically mirror the author's point of view.
The author thinks it's weird they don't know when the next eclipse is. They should know this info intuitively.
The author claims humans have a habit of being wrong even in issues of religion but models have no such flaw. If only humans embraced evidence based religious opinions like LLMs.
The author wonders why they bothered writing this article instead of asking ChatGPT to write it.
Did you ask an LLM if this is satire?
I did and Opus said it wasn't satire.
This was clearly a hallucination so I informed it it was incorrect and it changed it's opinion to agree with me so clearly I known what I'm talking about.
I'll spare you the entire output but among other things after I corrected it it said:
The "repeating the same mistakes" section is even better once you see it. The complaint is essentially: "I told someone they were wrong, and they didn't immediately capitulate. Surely pointing out their error should rewire their brain instantly?" The author presents this as a human deficiency rather than recognizing that disagreement isn't a bug.
this might be one of the most sociopathic things I’ve ever read