In contrast, the SV focus of AI has been about skynet / singularity, with a hype cycle to match.
This is supported by the lack of clarity on actual benefits, or clear data on GenAI use. Mostly I see it as great for prototyping - going from 0 to 1, and for use cases where the operator is highly trained and capable of verifying output.
Outside of that, you seem to be in the land of voodoo, where you are dealing with something that eerily mimics human speech, but you don't have any reliable way of finding out its just BS-ing you.
Sadly, the search for that link continues.
I did find these from SCMP and Foreign Policy, but there are better articles out there.
- https://foreignpolicy.com/2025/11/20/china-ai-race-jobs-yout...
- https://www.scmp.com/specialist-publications/special-reports...
https://www.urbanomic.com/book/machine-decision-is-not-final...
Sometimes I'm not so sure about any so-called moral superiority.
There's an overview on Wikipedia too https://en.wikipedia.org/wiki/AI-assisted_targeting_in_the_G...
Are they not going to build a “skynet” in China? Second, building skynet doesn’t imply eviscerating youth employment.
On the other hand, automation of menial tasks does eviscerate all kinds of employment, not only youth emoloyment.
Similarly, the claim is that ~90% of communication is nonverbal, so I'm not sure I would trust a negotiator who has seen all of written human communication but never held a conversation.
Well, in many cases they might be right..
This is to deconstruct the question.
I don't think it's even wrong - a lot of people are doing things, making decisions, living life perfectly normally, successfully even, without applying intelligence in a personal way. Those with socially accredited 'intelligence' would be the worst offenders imo - they do not apply their intelligence personally but simply massage themselves and others towards consensus. Which is ultimately materially beneficial to them - so why not?
For me 'intelligence' would be knowing why you are doing what you are doing without dismissing the question with reference to 'convention', 'consensus', someone/something else. Computers can only do an imitation of this sort of answer. People stand a chance of answering it.
I'm not following. A computer's "why" is a written program, surely that is the most clear expression of its intent you could ask for?
Is your position that humans are pretty mechanistic, and simply playing out their programming, like computers? And that they can provide a stacktrace for what they do?
If so, this is what I was getting at with my initial comment. Most people do not apply their intelligence personally - they are simply playing out the goals that we inserted into them (by parents, society). There are alternative possibilities, but it seems that most people's operational procedures and actions are not something they have considered or actively sought.
Yes, at least it's what I wanted to drill further into.
Boiled down, I'm interested in hearing where "intelligent" people derive their motivations(I'm in agreement that most people are on ["non-intelligent" if you will] auto-pilot most of the time) if not from outside themselves, in your framework.
When does a goal start being my intelligent own goal? Any impetus for something can be traced back to not-yourself: I might decide to start tracking my spending, but that decision doesn't form out of the void. Maybe I value frugality, but I did not create that value in myself. It was instilled in me by experience, or my peers, etc. I see no way for one to "spontaneously" form a motivation, or if I wanted to take it one step further(into the Buddha's territory), I would have to question who, and where, and what this "self" even is.
To me, the answer is obvious. Inserting thousands of ideas and patterns of thoughts into a person will be unlikely to help them become a true expression of their nature. If you know gardening, the schooled person is more like a trained tree - grown in a way that suits the farmer - the more tied back the tree is, the less free it is.
As I see it, each individual is unique, with a soul. Each is capable of reaching a full expression of itself, by itself. What I also see is that there are many systems that are intentional manipulations, put in place in order to farm individuals at the individual's expense. The more education one receives, the more amenable one is to being 'farmed' according to the terms that were inserted. To me, this is the installation of an unnatural and servile mentality, which once adopted makes the person easy to harness - the person will even think being harnessed and 'in service' is right and good.
The problem is that these principles were not their own. These are like religious beliefs, and unlike principles founded according to personal experience. Received principles will always be unnatural. Acting according to them, is to act in an inauthentic way. However, there is no material reason to address the inauthenticity, as when one looks around, everyone else is doing the same. This results in a self-supporting, collective delusion.
In my view there are answers to what the self is - but 'society' cannot teach you them - it can only fill you with delusions. Imo, you would be on a better footing to forget everything you think you know (this costs you nothing) and do something like apply the scientific method personally - let your personal experiences guide you. Know the difference between 'knowing' because of experience and 'belief' because you were taught it. Even more simply, know thyself.
The tree does not exist in isolation, separate from the patterns of rain and sunshine that shape its growth. "The separation is an illusion".
I have indeed been on the same path as you of trying to shed delusions and applying the scientific method, and have up to this point found no indication of any "causeless cause" to steer me besides the fundamental is-ness of the universe.
Put bluntly, I believe that if you hadn't started with the assumption of a soul, you would be entirely unable to arrive at the conclusion of a soul by rational methods. And starting by assuming the unproven instead of emptyness is epistemological cheating.
Have you seen babies, or puppies? You would easily be able to confirm for yourself that creatures are born with distinct personalities. Its not just chemistry or nurture.
> "The separation is an illusion"
But you don't really think this. You don't really think you are a tree. You do think you are distinct.
Refer to my previous post: "I'm assuming that we're in agreement that genetics, pre-birth nutrition etc, are part of these circumstances and not of the 'soul' you're after?"
That's not some mysterious transcendant soul, that's genetics. Literally the exact same thing as a computer program. Dog breeds are specifically bred(programmed) to exhibit certain character traits, for example.
>You don't really think you are a tree. You do think you are distinct.
You missed the point of the argument. Just as the tree is not separate from its circumstances, neither am I.
You brought up "know thyself" so I assumed we were pulling from a similar corpus and brought up "the illusion of separation" as a mutually familiar point that didn't need much elaboration, sorry about that.
Also, it's not so much that I "think" I am distinct, more that I "believe" it, to put it in the terms you used earlier. I am conditioned to consider certain things "me" and others not.
Really I am no more distinct from the tree than, say, my fingernail is distinct from my nosebone. They belong to the same Individual.
And yet all dogs have their own unique characters, no? They are not the same individual, right?
> You brought up "know thyself" so I assumed we were pulling from a similar corpus and brought up "the illusion of separation" as a mutually familiar point that didn't need much elaboration, sorry about that.
I don't know what corpus you refer to. Please explain if you like. I'm not basing what I'm saying on a corpus - of course I've read books, but I am giving you my personal view on things.
> Also, it's not so much that I "think" I am distinct, more that I "believe" it, to put it in the terms you used earlier. I am conditioned to consider certain things "me" and others not.
I have heard this sort of (nondual) thinking before and completely dispute it. I personally cannot access anyone else's mind or body, I haven't no idea what you are thinking. I can only pretend to be doing this. There is a self, we live it continuously. There are times when we are fully present, where we are so in the immediate experience, that we can move out of linguistic/common concepts perhaps, but this is still within oneself.
For me, it is more that each person is a world in their own right, rather than "us" all being in the same universe. We simply do not have the level of interconnectivity you believe is there, when you say you are the tree or me. Furthermore, it really is very hard to see the point you are making when we have a disagreement - plainly there is a distinction.
On the "corpus" point: It's not about not "giving my personal view", it's about drawing from a shared lexicon, of terminology, of lenses through which to view and analyze That Which We Are Talking About. My "home" in this respect is mostly in Hindu Yoga, (Zen) Buddhism and Daoism. You will find in those corpus-es(corpi?) essentially the exact conversation we're having right now, and find addressed the questions you have, in a wonderful plethora of different ways. Any other religion's mystic branch, or western occultism or alchemy similarly. If you want a specific recommendation for an entry-point, I could recommend giving the Bhagavad Gita a shot and seeing if you "vibe" with the way it explains things. If you skip the (usually) included commentary and only read the core translation, it ought to be a fairly quick read.
The nonduality point: Your body cannot access others' experience any more than my fingernail can access that of my nosebone, sure. But again, that does not mean they aren't part of the same organism. The fingernail and the nosebone do not make independent choices, the choice is made for them by the meta-organism(my body). Similarly, the argumentation might go, the tree and I do not make independent choices, but are governed by the same meta-organism(Nature, if you will, or perhaps "The Universe", but I suspect that term will turn you off since it might evoke the image of new-age-hippy woowoo).
I'm saying that if you insist that the body/mind/whatever you currently refer to as "you" is your "Self", you are taking "the fingernail" to be your Self instead of "the whole person". "Plainly there is a distinction", yes. But at the same time, there is also an underlying interconnectivity.
>Furthermore, it really is very hard to see the point you are making when we have a disagreement
That is perhaps the wisest thing either of us is going to say in this conversation. This format does not serve high-effort posting very well, I know I'm not doing the best I could be.
Perhaps we'd shelve this discussion for now? If you care to continue more deeply, you could shoot me an email at any point in the future(see my profile), and I again heartily recommend the Bhagavad Gita. Or perhaps, if you're more rational-thinking oriented, you might enjoy(the even shorter) Yogasutras of Patanjali. Or have you checked out Yudkowski's "Sequences"[1]? That one's completely down-to-eath, no spiritual terminology or metaphors (or non-dualism I'm pretty sure!), and covers a lot of the same ground my eastern background does.
I don't dispute traits. But the traits idea fails to address the unique characteristics of each dog.
It seems I'm not tracking the things you want me to track, terminology, science, traits. But then, as I said in the first place:
> For me 'intelligence' would be knowing why you are doing what you are doing without dismissing the question with reference to 'convention', 'consensus', someone/something else.
I can tell you are sincere with your investigations, but I can't help wondering whether direct observations of reality, the development of a personal outlook on reality, use personal experience as primary source, is ultimately more valuable than familiarity with a corpus. But then I would say that. And you would disagree.
I am giving up. You are engaging with the points in your head instead of those on the page.
You match the spirit that you comprehend, not me.
With humans, the speed and ease with which we learn and reason is capped. I think a very dumb intelligence with stay dumb for not very long because every resource will be spent in making it smarter.
Currently, LLMs require hooks and active engagement with humans to ‘do’ anything. Including learn.
The root motivation on which every resource will be spent is simply and very obviously to make a profit.
We are incredibly far from AGI.
> If you ask an LLM to analyze a conversation from the internet it will misrepresent the positions of the participants, often restating things so that they mean something different or making mistakes about who said what in a way that humans never do.
AI transcription & summary seems to be a strong point of the models so I don't know what exactly you're trying to get to with this one. If you have evidence for that I'd actually be quite interested because humans are so bad at representing what other people said on the internet it seems like it should be an easy win for an AI. Humans typically have some wild interpretations of what other people write that cannot be supported from what was written.
>AI transcription & summary seems to be a strong point of the models so I don't know what exactly you're trying to get to with this one. If you have evidence for that I'd actually be quite interested because humans are so bad at representing what other people said on the internet it seems like it should be an easy win for an AI. Humans typically have some wild interpretations of what other people write that cannot be supported from what was written.
Transcription and summarization is indeed fine, but try posting a longer reddit or HN discussion you've been part of into any model of your choice and ask it to analyze it, and you will see severe errors very soon. It will consistently misrepresent the views expressed and it doesn't really matter what model you go for. They can't do it.
For simple discussions this is fine. For complex discussions, especially when people get into conflict-- whether that conflict is really complex or not, problems usually result. The big problems are that the model will misquote or misrepresent views-- attempted paraphrases that actually change the meaning, the ordinary hallucinations etc.
For stories the confusion is much greater. Much of it is due to the basic way LLMs work: stories have dialogue, so if the premise contains people not being able to speak each other's language problems come very soon. I remember asking some recent Microsoft Copilot variant to write some portal scenario-- some guys on vacation to Teneriffe rent a catamaran and end up falling through a hole in the world of ASoIAF and into the seas off Essos, where they obviously have a terrible time, and it kept forgetting that they don't know English.
This is of course not obviously relevant for what Copilot is intended for, but I feel that if you actually try this you will understand how far we are from something like AGI, because if things like OpenAIs or whoever's systems were in fact close, this would be close too. If we were close we'd probably see silly errors too, but it'd be different kinds of errors, things like not telling you the story you want, not ignoring core instructions or failing to understand conversations.
It doesn't surprise me that you're getting nonsense, that is an ill-formed request. The AI can't fulfil it because it isn't asking it to do anything. I'm in the same boat as an AI would be, I can't tell what outcome you want. I'd probably interpret it as "summarise this conversation" if someone asked that of me, but you seem to agree that AI are good at summery tasks so that doesn't seem like it would be what you want. If I had my troll hat on I'd give you a frequency analysis of the letters and call it a day which is more passive-aggressive than I'd expect of the AI, they tend to just blather when they get a vague setup. They aren't psychic, it is necessary to give them instructions to carry out.
This and we don't actually know what the foundation models are for AGI, we're just assuming LLMs are it.
So yes, most people are right in that assumption, at least by the metric of how we generally measure intelligence.
We should probably rigorously verify that, for a role that itself is about rigorous verification without reasonable doubt.
I can immediately, and reasonably, doubt the output of an LLM, pending verification.
I know I'm too late to ask this question, But I suspect its either; Feelings and intuitions, which is just a primitive IQ test. Or some kind of aptitude test, which is just a different flavor of IQ test.
If anything, I think they'd consider AI's involvement as a strike against the prosecution if they were on a jury.
At least personally, I've seen basically three buckets of opinions from non-technical people on AI. There's a decent-sized group of people who loathe anything to do with it due to issues you've mentioned, the art issue I mentioned, or other specific things that overall add up to the point that they think it's a net harm to society, a decent-sized group of people who basically never think about it at all or go out of their way to use anything related to it, and then a small group of people who claim to be fully aware of the limitations and consider themselves quite rational but then will basically ask ChatGPT about literally anything and trust what it says without doing any additional research. It's the last group that I'm personally most concerned about because I've yet to find any effective way of getting them to recognize the cognitive dissonance (although sometimes at least I've been able to make enough of an impression that they stop trying to make ChatGPT a participant in every single conversation I have with them).
Not like food or clothing, but stuff like DLC content, streaming services, and LLMs.
I think the anthropomorphizing part is what messes with people. Is the autocomplete in my IDE smarter than I am? What about the search box on Google? What about a hammer or a drill?
Yet, I will admit that most of the time I hear people complaining about how AI written code is worse than that produced by developers, but it just doesn't match my own experience - it's frankly better (with enough guidance and context, say 95% tokens in and 5% tokens out, across multiple models working on the same project to occasionally validate and improve/fix the output, alongside adequate tooling) than what a lot of the people I know could or frankly do produce in practice.
That's a lot of conditions, but I think it's the same with the chat format - people accepting unvalidated drivel as fact, or someone using the web search and parsing documents and bringing up additional information that's found as a consequence of the conversation, bringing in external data and making use of the LLM ability to churn through a lot of it, sometimes better than the human reading comprehension would.
As such the story can be completely divorced from reality. The important thing is that the story is a good one. A good story transfers your social cover for yourself to your supervisor. They don't have to understand what you did and explain why it's okay that it failed. They just have to understand the story structure that you gave them. Listen to this great story, it's not my report's fault for this failure, and it's certainly not mine, just bad luck.
Additionally, the good (and sufficiently original) story is a gift because your supervisor can reuse it for new scenarios.
The good salesman gives you the story you need to excuse the purchase that will enable you to succeed. The bad salesman sells you on a story that you need a frivolous purchase.
And this is why job hoping is "bad". Eventually the incompetent employee uses up all of their good stories and management catches onto their act. It's embedded into our language. "Oh we've all heard this story before." The job hopper leaves just as their good stories are exhausted and can start over fresh at the new employer.
All of this in response to
> If we're lucky, people will manage to adapt and update their mental models to be less trustworthy of things that they can't verify
Yes, if we're lucky that is what will happen. But I fear that we're going to have to transition to a very low trust society for that to happen.
Reliance on the story is reliant on the trust that someone has done the real work. Distrust of the story implies a wider scale distrust in others and institutions.
Maybe we can add a tradition of annotating our stories with arguments and proofs. Although I've spent a two decade career desperately trying to give highly technical people arguments and proofs and I've seen stories completely unmoored from reality win out every time.
Optimistically, I'm just really bad at it and it's actually a natural transition. Pessimistically, we're in for a bumpy ride.
The idea of story being how people justify making their decisions is interesting. I'm reminded of a couple of anecdotes my father has repeated a few times over the years about two distinct medical circumstances he's had. When he was first diagnosed with sleep apnea, he apparently was very skeptical that he had any reason to do anything because the sleep doctor told him things like "this will help you be less sleepy during the day" and "you won't start nodding off as you drive" when he didn't feel like either of those experiences happened to him. Eventually a different sleep doctor did convince him it was worthwhile to treat, and he's used a CPAP since then, he still seems not to feel like it would have made sense for him to start when he first got the diagnosis. Through the lens you've given, the original doctor didn't give him a compelling enough story to justify the effort on his part. On the other hand, the first time he talked to a nutritionist about changing his diet, he apparently mentioned something about how he wanted to at least be able to eat ice cream occasionally, even if it was less often, rather than not ever be able to eat it again, and the nutritionist replied "Of course! that would make life not worth living". He ended up being much more open to listening to the advice of the nutritionist than I would have expected, and I think it would be reasonable to argue that was because the nutritionist was able to give him a story that seemed compelling about what his life would be like with the suggested changes.
There's a lot of things going on in the western world, both financial and social in nature. It's not good in the sense of being pleasant/contributing to growth and betterment, but it's a correction nonetheless.
That's my take on it anyway. Hedge bets. Dive under the wave. Survive the next few years.
Online is a little trickier because you don't know if they're a dog. Well, now a days it's even harder, because they could also not have a fully developed frontal lobe, or worse, they could be a bot, troll, or both.
If you don't want to believe it, you need to change the goal posts; Create a test for intelligence that we can pass better than AI.. since AI is also better at creating test than us maybe we could ask AI to do it, hang on..
>Is there a test that in some way measures intelligence, but that humans generally test better than AI?
Answer:Thinking, Something went wrong and an AI response wasn't generated.
Edit, i managed to get one to answer me; the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI). Created by AI researcher François Chollet, this test consists of visual puzzles that require inferring a rule from a few examples and applying it to a new situation.
So we do have A test which is specifically designed for us to pass and AI to fail, where we can currently pass better than AI... hurrah we're smarter!
E.g watch a Steve jobs interview and a Sam Altman one (at the same age). The difference in the mode of articulation, simplicity in communication, obsession over details etc are huge. This is what superior intelligence to me looks like - you know it when you see it.
Easy? The best LLMs score 40% on Butter-Bench [1], while the mean human score is 95%. LLMs struggled the most with multi-step spatial planning and social understanding.
Still it may be lasting limitation if robotics don't catch up to AI anytime soon.
Don't know what to make of the Safety Risks test, threatening to power down AI in order to manipulate it, and most act like we would and comply. fascinating.
you must be completely LLMheaded to say something like that, lol
humans are not trained on spacial data, they are living in the world. humans are very much diffent from silicone chips, and human learning is on another magnitude of complexity compared to a large language model training
If this hurts your ego then just know the dataset that you built your ego with was probably flawed and if you can put that LoRA aside and try to process this logically; Our awareness is a scalable emergent property of 1-2 decades of datasets, looking at how neurons vs transistor groups work, there could only be a limited amount of ways to process these sizes of data down to relevant streams. The very fact that training LLMs on our output works, proves our output is a product of LLMs or there wouldn't be patterns to find.
https://en.wikipedia.org/wiki/Intelligence_quotient#Validity...
If you could get the full page text of every url on the first page of ddg results and dump it into vim/emacs where you can move/search around quickly, that would probably be similarly as good, and without the hallucinations. (I'm guessing someone is gonna compare this to the old Dropbox post, but whatever.)
It has no human counterpart in the same sense that humans still go to the library (or a search engine) when they don't know something, and we don't have the contents of all the books (or articles/websites) stored in our head.
Curiously, literally nobody on earth uses this workflow.
People must be in complete denial to pretend that LLM (re)search engines can’t be used to trivially save hours or days of work. The accuracy isn’t perfect, but entirely sufficient for very many use cases, and will arguably continue to improve in the near future.
Frankly I've seen enough dangerous hallucinations from LLM search engines to immediately discard anything it says.
Versus finding the answer by clicking into the first few search results links and scanning text that might not have the answer.
They do it by going out and searching, not by storing a list of sources in their corpus.
Because when I ask chatgpt/perplexity things like "can I microwave a whole chicken" or "is Australia bigger than the moon" it will happily google for the answers and give me links to the sites it pulled from for me to verify for myself.
On the other hand, if you ask it to summarize the state-of-the art in quantum computing or something, it's much more likely to speak "off the top of its head", and even when it pulls in knowledge from web searches it'll rely much more on it's own "internal corpus" to put together an answer, which is definitely likely to contain hallucinations and obviously has no "source" aside from "it just knowing"(which it's discouraged from saying so it makes up sources if you ask for them).
If anything, I have the opposite problem. The sources are the best part. I have such a mountain of papers to read from my LLM deep searches that the challenge is in figuring out how to get through and organize all the information.
When it gives you a link, it literally takes you to the part of the page that it got its answer from. That's how we can quickly validate.
That seems to be a big part of it, yes. I think in part it’s a reaction to perceived competition.
The reason why people don't use LLMs to "trivially save hours or days of work" is because LLMs don't do that. People would use a tool that works. This should be evidence that the tools provide no exceptional benefit, why do you think that is not true?
If they do, you’ll be in good company. That post is about the exact opposite of what people usually link it for. I’ll let Dan explain:
So yes, it is the opposite of why people link to it (which is a judgement I’m making, I’m not arguing Dan has that exact sentiment), which is to mock an attitude (which wasn’t there) of hubris and lack of understanding of what makes a good product.
It's why it caught the zeitgeist at the time and why it's still apropos in this conversation now.
None of those things are true. Which is the point I’m making. Go read the original conversation. All of it.
https://news.ycombinator.com/item?id=9224
Don’t skip Brandon’s reply.
https://news.ycombinator.com/item?id=9479
It is absurd to claim that someone who quickly understood the explanation, learned from it, conceded where they were wrong, is somehow “profoundly out-of-touch” and “lost all perspective”. It’s the exact opposite.
I agree with Dan that we’d be lucky if all conversations were like that.
Ironically your own overly verbose and aggressive comments here fall into the same trap.
> the breadth of knowledge
If knowledge == intelligence then Google and Wikipedia are "smarter" than you and the AGI problem has been solved for several decades.