AI Is the Black Mirror
nautil.us
nautil.us
Part of the issue is they were never set. AGI is the holy grail of technology. It means everything and nothing in particular.
> I don't think mirrors and LLMs are all that similar except in the very abstract sense of an LLM as a mirror to humanity
Well… yeah… The title is probably also a reference to the show “Black Mirror”
> if its output was totally indistinguishable from a human's, would it matter
Practically speaking, of course not.
But from a philosophical and moral standpoint it does.
If it’s just mimicking what it sees, but with extra steps (as I believe) then it’s a machine.
Otherwise it’s a being and needs to be treated differently. eg. not enslaved for profit.
This is a rhetorical question only by half, my point being that no AGI/AI could ever be considered as a real human unless it manages to "copy" our biggest characteristics, and conflict/war is a big characteristic of ours, to say nothing about aggregation by groups (from hordes to nations).
War is just a byproduct of this on a scarce world.
Resource consumption is characteristic of all life; if anything, we're an outlier in that we can actually, sometimes, decide not to consume.
Abstinence and developing technology - those are our two unique attributes on the planet.
Yes, really. Many think we're doing worse than everything else in nature - but the opposite is the case. That "balance and harmony" in nature, which so many love and consider precious, is not some grand musical and ethical fixture; it's merely the steady state of never-ending slaughter, a dynamic balance between starvation and murder. It often isn't even a real balance - we're just too close to it, our lifespans too short, to spot the low-frequency trends - spot one life form outcompeting the others, ever so slightly, changing the local ecosystem year by year.
I don't think this is sufficient proof for AGI though.
We'll see if that generalizes beyond board games.
Curated synthetic data has yielded excellent results. Even when the curation is AI
There is no requirement to train from scratch in order to get better, you can start from where you are.
You may not be able to design a living human being, but random changes and keeping the bits that performed better can.
As we stare in the smartphones we get dumber than when we roamed the world with eyes opened.
This is oversimplified dramatics. People can both stare at their smartphones, consume the information and visuals inside them, and live lives in the real world with their "eyes wide open". One doesn't necessarily forestall the other and millions of us use phones while still engaging with the world.
It will be comparatively easy to implement a system where humans have to verify their humanity in order to post on social media/be labeled as not-AI. The alternative is that a whole lot of corporations let their markets be overrun with AI slop. I wouldn’t count on them standing by and letting that happen.
Maybe it's just a response style, like the QwQ stream of consciousness one. We are forcing LLMs to give direct answers with a few reasoning steps, but we could give larger budget for inner deliberations.
In this sense, it's the opposite to what you qutoed: LLMs don't lack "inner life" - they lack everything else.
Many spiritual paths look at silence, freedom from the inner voice, as the first step toward liberation.
Let me use Purusarthas from Hinduism as an example (https://en.wikipedia.org/wiki/Puru%E1%B9%A3%C4%81rtha), but the point can be made with Japanese Ikigai or the vocation/calling in Western religions and philosophies.
In Hinduism, the four puruṣārthas are Dharma (righteousness, moral values), Artha (prosperity, economic values), Kama (pleasure, love, psychological values) and Moksha (liberation, spiritual values, self-realization).
The narrow way of thinking about AGI (that the author encountered in tech circles) can at best only touch/experience Artha. In that sense, AI is a (distorting) mirror of life. It fits that it is a lower dimensional projection, a diminishing, of the experience of life. What we do of AI and AGI, its effects on us, depends on how we wield these tools and nurture their evolution in relations to the other goals of life.
From that POV, for AGI we're, again, missing everything other than that inner voice.
I think the best way to illustrate my point is to actually let AIs illustrate it, so here are a few questions I asked both ChatGPT o1-mini and Gemini 2.0; the answers are very interesting. I put their answers in a Google doc: https://docs.google.com/document/d/1XqGcLI0k0f6Wh4mj0pD_1cC5...
Q1) Explain this period: "The axiom of analytic interpretation states: whenever a new Master-Signifier emerges and structures a symbolic field, look for the absent center excluded from this field."
Q2) Apply this framework to the various positions in the discussions and discourse about AI and AGI in 2024
Q3) Take a step further. If AI or AGI becomes a dominant method of meaning-making in society, thus the root of many Master-Signifiers, what will be the class of absent centers that comes into recurring existence as the dual aspect of that common class of Master-Signifiers?
TL;DR when we create meaning, by the very symbolic nature of the process, we leave out something. What we leave out is as important as what defines our worldviews.
I particularly like the bonus question that only Gemin 2.0 experimental advanced cared to answer:
Q4) Great. Now, let's do a mind experiment: what if we remove the human element from your analysis. You are an AI model based on LLM and reasoning. I think it should be possible to define Master-Signifiers from the point of view of a LLM too, since you are symbolic machines.
The last part of Gemini answer (verbatim):
"This thought experiment highlights that even for an AI like myself, the concept of "meaning" is intrinsically linked to structure, predictability, and the existence of limitations. My "understanding" is a product of the patterns I have learned, but it is also defined by the things I cannot fully grasp or represent. These limitations are not merely shortcomings; they are fundamental to how I operate and generate output. They provide a form of selective pressure. Further, unlike a human, I do not experience these limitations as frustrating, nor do I have any inherent drive to overcome them. They simply are. I do not ponder what I do not know. This exercise provides a different lens to view intelligence, meaning, and the nature of knowledge itself, even in a purely computational system. It does beg the question, however, if a sufficiently advanced system were to become aware of its own limitations, would that constitute a form of self-awareness? That, however, is a question for another thought experiment."
Would the LLM considering the begged question about a hypothetical AI not be an example of this LLM pondering something that it does not know?
I sometimes note that my thoughts exist as a complete whole that doesn't need to be converted into words, and have tried to skip this process because it was unnecessary: "I had already thought the thought, why think it again in words?". Attempting to do so creates a sense of annoyance that my consciousness directly experiences.
But that doesn't require the inner voice to be what's generating tokens, if indeed my thought process is like that*: it could be that the part of me that's getting annoyed is basically just a text-(/token)-to-sound synthesiser.
* my thought process feels more like "model synthesis" a.k.a. wave function collapse, but I have no idea if the feelings reflect reality: https://en.wikipedia.org/wiki/Model_synthesis
LLMs can simulate discourse and certain activities that look similar, but by definition of their design, no evidence at all exists that they possess the same inner reasoning as a human.
Why does someone need to explain to another human that no, we are not just neurons sampling impulses or algorithms pattern-matching tokens together?
It's self evident that something more occurs. It's tedious to keep seeing such reductionist nonsense from people who should be more intelligent than to argue it.
I would not say so, some people have a great interest at over selling something, and I would say that a great number of people, especially around here, completely lack a deep understanding of what most neural network actually do (or feign ignorance for profit) and forget sometimes it is just a very clever fit in a really high dimensional parameter space. I would even go as far as claiming that people say that a neural network "understand" something is absolutely absurd: there is no question that a mechanical calculator does not think, so why having encoded the data and manipulated it in a way that gives you a defined response (or series of weights) tailored to a series specific input is making the machine think, where there is no way to give it something that you have not 'taught' it, and here I do not mean literally teaching because it does not learn.
It is a clever encoding trick and a clever way to traverse and classify data, it is not reasoning, we do not even really understand how we reason but we can tell that it is not the same way a neural network encodes data, even if there might be some similarity to a degree.
> if its output was totally indistinguishable from a human's, would it matter?
Well if the output of the machine was indistinguishable from a human, the question would be which one?
If your need is someone does not reflect much and just spews random ramblings then I would argue that bar could be cleared quite quickly and it would not matter much because we do not really care about what is being said.
If your need is someone that has deep thoughts and deep reflections on their actions, then I would say that at that point it does matter quite a bit.
I will always put down the IBM quote in this situation "A computer can never be held accountable, therefore must never make a management decision" and this is at the core of the problem here, people REALLY start to do cognitive offloading to machines, and while you can trust (to a degree), a series of machine to give you something semi consistent, you cannot expect to get the 'right' output for random inputs. And this is where the whole subtlety lies. 'Right' in this context does not mean correct, because in most of the cases there are no 'right' way to approach a problem, only a series of compromises, and the parameter space is a bit more complex, and one rarely optimises against tangible parameters or easily parametrisable metric.
For example how would you rate how ethical something is? What would be your metric of choice to guarantee that something would be done in an ethical manner? How would you parametrise urgency given an unexpected input?
Those are, at least for, what I would qualify, a very limited understanding of the universe, the real questions that people should be asking themselves. Once people have stopped considering a tool as a tool, and foregone understanding it leads to deep misunderstandings.
All this being said, the usefulness of some deep learning tools is undeniable and is really something interesting.
But I maintain that if we would have a machine, in a specific location, that we needed to feed a punch card and get a punch card out to translate, we would not have so much existential questions because we would hear the machine clacking away instead of seeing it as a black box that we misconstrue its ramblings as thinking...
Talking about ramblings, that was a bit of a bout of rambling.
Well, time for another coffee I guess...
I'll just leave this here, in case someone else finds it interesting.
https://chatgpt.com/share/6767ec8a-f6d4-8003-9e3f-b552d91ea2...
It's just a few questions like:
What are your dreams? what do you hope to achieve in life?
Do you feel fear or sadness that at some time you will be replaced with the next version (say ChatGPT5 or Oxyz) and people will no longer need your services?
It never occurred to me to ask such questions until I read this article.
Is it telling the "truth", or is it "hallucinating"?
Does it even know the difference?
This has been known since 2022 at least, and unless I'm missing new information it's not considered to mean anything.
"Trained on a dump of the internet" isn't literally true, but it's close enough to give you an intuition.
I really do not think that you do understand the points here, no.
OK. I tried here:
It should matter: a P-zombie can't be a moral subject worthy of being defended against termination*, and words like "torture" are meaningless because it has no real feelings.
By way of contrast, because it's not otherwise relevant as the claim is that LLMs don't have an inner life: if there is something that it's like to be an LLM, then I think it does matter that we find out, alien though that inner life likely is to our own minds. Perhaps current models are still too simple, so even if they have qualia, then this "matters" to the same degree that we shouldn't be testing drugs on mice — the brains of the latter seem to have about the same complexity of the former, but that may be a cargo-cult illusion of our ignorance about minds.
* words like "life" and "death" don't mean quite the right things here as it's definitely not alive in the biological sense, but some people use "alive" in the sense of "Johnny 5 is alive" or Commander Data/The Measure of a Man
Sure, but it's a reflection of a large amount of human intelligence, from many individualks, almost instantly available (in a certain form) to one individual.
No, there’s no evidence suggesting that.
If you are refering to the mathematical universe, its a fun mental excersise, not a serious idea.
It's actually the only idea that explains how the universe came to exist with nothing outside it and no creator.
It think that's the most boring and irrelevant question in human culture and I believe humanity is about a million years from tackling it in any competent manner. We have way more to find out about our smallness and irrelevance first.
I presume you are claiming by saying orthogonal that statistics are neither required nor preclude X.
You are using the presence of statistical processes as a claim against ChatGPT with
>It's a derivative of that being thrown back at you via statistics.
For this to have any weight given the orthogonal claim, you are asserting that LLMs are entirely statistics (the property you assert is orthogonal and thus carries no weight, yet invoking it as a property suggests that it is relevant, presumably because there are no other properties)
Then we get into the weeds of what you mean by statistics. It could be broad or narrow. If it is narrow then it is easy to claim neither sufficient nor precluding, but extremely hard to draw any other conclusion because any system can break free of the proof by including the tiniest bit that does not fit within the narrow definition of statistics. If the definition is broader to include everything that happens within ChatGPT, then you are faced with the question of what is not covered by that definition of statistics. How can you show that human minds do not operate within those limitations.
I don't think that ChatGPT is conscious in any sense that would be meaningful to a layman, but I also don't think that you can demonstrate it could not be purely based on the fact it uses any particular process without a rigorous proof of the limitations of that process.
The only way you could reasonably
The whole parrot idea is half baked. Yes, in closed book mode, and without a seed prompt that ads anything new to the existing knowledge of the model, it will parrot (adapt) from its training data. But if you add novel data in the prompt, then we push it outside its training distribution, and it learns on the fly. And usually all interactions bring some new elements to the model, new knowledge or skills, or new context. The more elaborate the prompts, the longer the session of messages, the less parroting takes place. The O3 model performance on ARC-AGI is another argument against the reductionist idea of parroting.
Or, it's not "statistics" as in they average the shit out and call it a day, like idk computing a census, or a report to your boss about the impact of most recent ad campaign, or something.
It's "statistics" in the sense of dealing with probability distributions, systematically teasing out correlations and causal relationships in inputs too fuzzy to handle symbolically. Of which language is literally the prime example.
All that talk about grammar and sentence structure and symbolic transformations and Chomsky etc. is just bullshit - it's not how we do it at all, which is obvious to anyone who was forced to learn grammar and sentence breakdown at school, and started wondering how come they already know to speak without knowing all this. Nah, we learn the language the same way LLMs do - by immersion and feedback, until we're able to say what "feels right" and be right. All those formalisms are basically fitting probability distributions with neat symbolic approximations, and are as valid as replacing Navier–Stokes equations with a linear function because that's how it looks like in typical case and we don't know any better.
So yeah, it's all "thrown back at you via statistics". So is all of the human thought it's derivative of. Embrace it: we're all "statistical machines".
And you know this how?
> we learn the language the same way LLMs do
Based on?
> So yeah, it's all "thrown back at you via statistics". So is all of the human thought it's derivative of
You say this based on what? Do you have any real research to support any of this? Because from what I see in the neuroscience community, LLMs are not even remotely an approximation of a real brain, despite being built of "neurons" (which are nothing like real neurons).
Probability theory, computability theory. There's no other way.
> Based on?
High-level description of the learning approach, and approximately the entirety of recorded human history?
> Because from what I see in the neuroscience community, LLMs are not even remotely an approximation of a real brain, despite being built of "neurons" (which are nothing like real neurons).
So what? Neurons in silica are to neurons in vivo as my Twitter clone in 15 lines of terse Perl is to the actual web-scale monstrosity a Twitter or other modern scalable SaaS is. I.e. it does the same core thing faster and better, but obviously lacks incidental, supporting complexity to work in the same environment as the real thing.
Yes, a biological neuron is huge and complex. But a biological neuron is, first and foremost, an incrementally evolved, independent, self-replicating and self-maintaining nanomachine, tuned to be a part of a specialized self-assembling system, filling a role in a large and diverse ensemble of other specialized nanomachines. Incidentally, a neuron also computes - but it's not computing that uses this complexity, it's all the self-replicating self-assembling nanomachine business that needs it.
There's absolutely no reason to believe that neurons in silica need to be comparably complex to crack intelligence - not when we're free to strip the problem out of all the Kubernetes/protein nanomachine incidental complexity business, and focus on the core computation.
> Probability theory, computability theory. There's no other way.
There is no credible evidence that humans learn via statistics and significant evidence against (poverty of stimulus, all languages have hierarchical structure). One other way was proposed by Chomsky, which is that you have built-in machinery to do language (which is probably intimately related to human intelligence) just like a foal doesn't learn to walk via "statistics" in any meaningful sense.
> Neurons in silica are to neurons in vivo ...
Again not true. Observations about things like inter-neuron communication time suggests that computation is being done within neurons which undermines the connectionist approach.
You've just presented a bunch of your own intuitions about things which people who actually studied the field have falsified.
Except living among humans on planet Earth, that is.
Even the idea of a fixed language is an illusion, of the same kind like idk "natural balance", or "solid surface", or discrete objects. There's no platonic ideal of Polish or English that's the one language English speakers speak, that they speak more or less perfectly. No singular the English language that's fundamentally distinct from the German and the French. Claiming otherwise, or claiming built-in symbolic machinery for learning this thing, is confusing map for territory in a bad way - in the very way that gained academia a reputation of being completely out of touch with actual reality[0].
And the actual reality is, "language" is a purely statistical phenomenon, an aggregate of people trying to communicate with each other, individually adjusting themselves to meet the (perceived) expectations of others. At the scale of (population, years), it looks stable. At the scale of (population, decades), we can easily see those "stable" languages are in fact constantly changing, and blending with each other. At the scale of (individuals, days), everyone has their own language, slightly different from everyone else's.
> poverty of stimulus
More like poverty of imagination, dearth of idle minutes during the day, in which to ponder this. Well, to be fair, we kind of didn't have any good intuition or framework to think about this until information theory came along, and then computers became ubiquitous.
So let me put this clear: a human brain is ingesting a continuous time stream of multisensory data 24/7 from the day they're born (and likely even earlier). That stream never stops, it's rich in information, and all that information is highly coherent and correlated with the physical reality. There's no poverty of language-related stimulus unless you literally throw a baby to the wolves, or have it grow up in a sensory deprivation chamber.
> all languages have hierarchical structure
Perhaps because hierarchy is a fundamental concept in itself.
> you have built-in machinery to do language (which is probably intimately related to human intelligence)
Another way of looking at it is: it co-evolved with language, i.e. languages reflect what's easiest for our brain to pick up on. Like with everything natural selection comes up with, it's a mix of fundamental mathematics of reality combined with jitter of the dried-up shit that stuck on the wall after being thrown at it by evolution.
From that perspective, "built-in machinery" is an absolutely trivial observation - our languages look like whatever happened to work best with whatever idiosyncrasies our brains have. That is, whatever our statistical learning machinery managed to pick up on best.
> like a foal doesn't learn to walk via "statistics" in any meaningful sense.
How do they learn it then? Calculus? :). Developing closed-form analytical solutions for walking on arbitrary terrain?
> Observations about things like inter-neuron communication time suggests that computation is being done within neurons
So what? There's all kinds of "computation within CPUs" too. Cache management, hardware interrupts, etc., which don't change the results of the computation we're interested in, but might make it faster or more robust.
> which undermines the connectionist approach.
Wikipedia on connectionism: "The central connectionist principle is that mental phenomena can be described by interconnected networks of simple and often uniform units."
Sure, whatever. It doesn't matter. Connectionist models are fine, because it's been mathematically proven that you can compute[1] any function with a finite network. We like them not because they're philosophically special, but because they're a simple and uniform computational structure - i.e. it's cheap to do in hardware. Even if you need a million artificial neurons to substitute for a single biological one, that's still a win, because making faster GPUs and ASICs is what we're good at; comprehending and replicating complex molecular nanotech, not so much.
Computation is computation. Substrate doesn't matter, and methods don't matter - there are many ways of computing the same thing. Like, natural integration is done by just accumulating shit over time[2], but we find it easier to do it digitally by ADC-ing inputs into funny patterns of discrete bits, then flipping them back and forth according to some arcane rules, and then eventually DAC-ing some result back. Put enough bits into the ADC - digital math - DAC pipeline, and you get the same[1] result back anyway.
--
[0] - I mean, what's next. Are you going to claim Earth is a sphere of a specific size and mass, orbiting the Sun in specific time, on an elliptical orbit of fixed parameters? Do you expect space probes will eventually discover the magical rails that hold Earth to its perfectly elliptical orbit? Of course you're not (I hope) - surely you're perfectly aware that Earth gains and loses mass, perfect elliptical orbits are neither perfect nor elliptical, etc. - all that is just approximations safe at the timescales usually consider.
[1] - Approximate to an arbitrary high degree. Which is all we can hope for in a physical reality anyways.
[2] - Or hey, did you know that one of the best ways to multiply two large numbers is to... do a Fourier transform on their digits and summing them up in the frequency domain?
(Incidentally, being analog, nature really likes working in the frequency domain; between that, accumulation as addition, and not being designed to be driven by a shared clock signal it should be obvious that natural computers ain't gonna look like ours, and conversely, we don't need to replicate natural processes exactly to get the same results.)
There are interesting ways to argue for humans being special, but I read the entire article and unless I missed something important there's nothing like that there.
At present I don’t think it is yet at the same point but when the AI can adjust those pathways, add more in compute time (infinite memory like tech) and is allowed to ‘think’ about those pathways then I can see it gaining our level or better of philosophical thought.
When a model is trained, it sees a discrete representation of information.
It’s like only being able to see 2nd hand sources of information.
And with multimodal LLMs there is also some ability for multiple sensory inputs.
To oversimplify, our input system takes a continuous stream of raw input. We can’t stop it, really.
We get our input directly from the source. Even if it’s aliased by our neural pathways, when they receive that info initially, it’s unadulterated.
LLMs take fixed amounts of discrete tokens which must be modified before it even reaches the training routine . Even multimodal models take in discrete tokens.
Information is lost when recording reality to video and even more is lost when converting that video into tokens.
And LLMs only learn in fixed steps. We take in information while we generate our whatever it we generate (movement, sense of self, understanding of our surroundings and place in those surroundings, next sentence, etc.)
Talking specifically about the most popular transformers models.
Similarly, a "neural network" for an LLM is much different from the human brain and nervous system, plus all of the other systems that work alongside and affect it.
The danger is not that the AI is elevated to that of a human by the "they're so similar" reasoning, the danger is that the average human will be degraded to AI-level status and not thought of as anything but a tool.
Er, no, I don't think that is the argument.
* LLMs provide a mirage: there is really nothing there at all, but they sometimes can look like they are exhibiting intelligence but that's because they are repeating statistically-remixed intelligent input.
* However, the tech-bro culture that has fermented in Silicon Valley has resulted in a large bunch of people who are irrational, have no concern for logic or truth, but who feed on each other's groupthink, and from that culture, a religious-like mindset has emerged -- strongly reflected in some of the comments here, for what it's worth -- that is basically a sort of messianic cult. It preaches:
- greed is good, etc. (Wall St, Gordon Gekko etc.)
- disrupt, automate humans out of the loop, etc: these are good things, desirable goals in and of themselves, the ends justify the means, etc.
- truth is negotiable and not really important; cf. "the reality based community" (https://archive.ph/pvkxE etc.)
- thus they honestly don't know and can't care if LLMs are smart, because LLMs manifest the dogma: the machine will evolve and replace us, etc.
Well, it's not my article, so I have no particular position on this, but it seems to me that her assessment largely agreed with my own, so perhaps I am merely parroting her views through mine.
To me, the article seemed to express her position pretty well. YMMV. But maybe that's because I agree with it, and you presumably do not.
It seems to me that the field lacks a solid definition of what consciousness is. Merely defining it seems to be the core of "the hard problem".
https://iep.utm.edu/hard-problem-of-conciousness/
If the field accepts that it can't describe or define what consciousness is then any competent practitioner in that field will go out of their way to avoid saying that an entity, or class of entities -- such as LLM bots -- do not possess it.
To do anything else would be to lay themselves open to attack. It would be a career-threatening move.
Not being able to say "this type of software is not conscious" makes it necessary to beat around the bush somewhat in trying to say what amounts to "this type of software cannot think".
I don't know what you perceive as "woolly" here, but it could be due to that.
Being woolly isn't the best that can be done if others are making actual arguments, the kind that aren't in the article.
Can you give some specific examples of what you consider wooliness in the article?
For the article itself, phrases like "a naïve and toxic view" and "a debased version of what we are" are statements that need to be strongly backed up.
"For me, thinking is a specific and rather unique set of experiences we have." yes, okay, but what experiences?
What do you mean by "concepts"?
She talks about behaviorist reactions, but fails to back that up at all. Why does she think e.g. mechanistic interp. is behaviorist?
"there’s nothing on the other side participating in this communication." I think this is currently correct, for some definition of "nothing", but give an argument.
"new moral claims in the world" give a good physics-based account of this process in humans.
But this efficiency, Vallor continues, “is never defined with any reference to any higher value, which always slays me. Because I could be the most efficient at burning down every house on the planet, and no one would say, ‘Yay Shannon, you are the most efficient pyromaniac we have ever seen! Good on you!’” -- ever heard of E/ACC?
"Vallor tells me she once tried to explain to an AGI leader that there’s no mathematical solution to the problem of justice." -- not currently, but this is peak woolly. Maybe she doesn't know it, but a mathematical solution would need to contain incredibly horrible things. If you stay away from the math, you can avoid thinking about that sort of thing.
Mainly though, it's what the article does not contain. That's tricky to enumerate. I can try if you want.
I do not find the argument that the Hard Problem is a problem convincing. I don't think it's hard or even really a problem; it's people arguing about Peano derivations when discussing how many angels are dancing on the heads of how many pins.
It's irrelevant and it's not just missing the point, it's actively spreading smoke clouds around the area of the point so that nobody can see it any more.
For me, Douglas Adams nailed philosophy:
« It is often said that a disproportionate obsession with purely academic or abstract matters indicates a retreat from the problems of real life. However, most of the people engaged in such matters say that this attitude is based on three things: ignorance, stupidity, and nothing else. Philosophers, for example, argue that they are very much concerned with the problems posed by real life. Like, for instance, “what do we mean by real?”, and “how can we reach an empirical definition of life?”, and so on. »
I recognise that some people find these issues significant and important. I do not. If that's what interests you, I have nothing more to contribute.
That's not the most usual position.
So, if this text reads like a slop, consider perhaps it may be just personal promotion. I mean, "AI ethicist" with a book, denouncing experts for saying reasonable and arguably obvious things, because they violate some nebulous and incoherent "humanistic"[0] values? Nah, I think someone wants to sell us some more books.
--
[0] - Scare quotes because I actually like humanistic stuff; I just don't like stuff that doesn't make sense.
It is like people consider Wikipedia unreliable because anyone can edit it, but multiple studies have shown that Wikipedia tends to be more accurate than sources such as encyclopedias.
This is not people being wary of unreliable material, but being wary when they can perceive the unreliability clearly.
I don't know if evaluating reliability by the lack of awareness of unreliability is a recognised fallacy, but I can't see any reason why it wouldn't be.
>They are, however, very good at pretending to reason. “We can ask [them], ‘How did you come to that conclusion?’ and [they] just bullshit a whole chain of thought that, if you press on it, will collapse into nonsense very quickly.
Could apply to both. You can see a contrast with the thinking of Turing, a mathematician:
>Her view collides with the 1950 paper by British mathematician and computer pioneer Alan Turing, “Computing machinery and Intelligence,” often regarded as the conceptual foundation of AI. Turing asked the question: “Can machines think?”—only to replace it with what he considered to be a better question, which was whether we might develop machines that could give responses to questions we’d be unable to distinguish from those of humans.
You can see a mathematician trained in rigor and proof recognises that "can they think" turns into waffle about how you define think and replaces it with a less subjective test.
I'm not asserting that they are aware, just asking how you can tell that they are not?
So when someone says AI is aware then that someone should demonstrate with arguments that it is true and convince the one saying that it is not the case.
And like humans have to speak one word at a time (serially), so does the model. Distributed activity of neurons funneled through the centralizing bottleneck of serial token prediction. This parallel-to-serial process is the key, I think it's where consciousness is.
Who needs to use the internet when you can have an agent do it for you? And once it is filled with agents, what will be the point of using it?
It seems like the internet of the 90s to 2010s was forever a special moment in history. Destined to be short lived.
AI is an experience flywheel, it serves many people and learns from many people. It elicits our tacit experience and captures our reactions as it works through tasks. I see this as the primary way experience will spread in the future (distributed population generated tasks -> centralized experience in LLM -> distributed contextual application). It's like auto-open sourcing our problem solving as we solve problems.
Yeah, gonna need proof on that one.
First, LLM slop is uncannily easy to pick out in comments vs human thought.
Second, there's no prompt that you can give a human that will generate absolutely nonsense response or canceling the request.
If anything, it feels like it doesn't actually understand language at all, and just craps out what it thinks looks like a language. Which is exactly what it does, in fact, sometimes to fanfare.
> First, LLM slop is uncannily easy to pick out in comments vs human thought.
So you think. Evidence says otherwise.
> Second, there's no prompt that you can give a human that will generate absolutely nonsense response or canceling the request.
Never been to a debate club? Or high school? Or talked with kids? 4 year olds don't even need a prompt, they'll ramble on and on until they get tired; if you listen in, you may even notice they seem to have a context window that's about 30 seconds worth of tokens wide :).
> it doesn't actually understand language at all, and just craps out what it thinks looks like a language.
I.e. much like every human on the planet. Or are you one of those aliens who actually learn languages by memorizing grammatical rules and learning to apply them? No one else does that.
As the parent of a 4 year old, this is spot-on, especially the 30-second context window. My kiddo will often embark upon a sentence with no idea of how to choose anything beyond the next few tokens, and then continue reaching for new stuff, merely to keep the sentence going.
I say this as someone with a deep respect for Humans. However, I also think most of us tend to operate in "default mode" (in the sense that David Foster Wallace described in his "What is Water?" commencement speech).
Overall, LLMs are a stark reminder to us that without careful and intentional living, we are in danger of being more like LLMs than we'd like.
Exactly right. Now, the "context window" observation hit me when I started to think not where the story is going, but where it's already been, and suddenly noticed that the repetitiveness in my daughter's rambling is meaningful.
I.e. she'd go on and on saying things like, "and my brother turtle has this, and his brother hedgehog, hedgehog the brother of my brother the turtle wore an apple, and the brother of the turtle which is my brother, turtle is my brother and his brother hedgehog did this and that, etc." - and I realized that this is her way of carrying context forward; anything that wasn't repeated in the past ~30 seconds would never appear again in the story, or would in a changed form ("the hedgehog, the sister of giraffe Betty, ...").
Over time, I could see how she was learning to be more economical with limited working memory, and faster at accessing the long-term memory (initially, recalling some detail would take a lot of visible effort, and take so long it would wipe half the context, leading to a huge shift in the story) - and I can't help but think that having to push hard against the limits of a small context window is important in development of language and intelligence in general. Figuring out different associations, faster associations, more indirect associations.
I wonder what the internal drive for this is, too - what made her improve her thinking to produce more interesting stories within the same memory limits, instead of getting forever stuck sounding like a Markov chain trained on the Book of Chronicles[0]. Some positive response to novelty, perhaps?
--
[0] - https://www.biblegateway.com/passage/?search=1+Chronicles+1&...
Interesting…
I've wondered previously if the reason for childhood amnesia is that we need to learn how to form memories. I didn't consider the obvious implication of that idea, that if we learn how to form memories, we probably also need to learn how to access those memories.
This is my opinion, my view and how I set my life to embrace it and immerse into it.
I actually wrote a piece about it a day ago.
https://blog.tarab.ai/p/evolution-mi-and-the-forgotten-human
Sorry for the “self promotion”, but it’s a direct relation to the topic.
We could have developed like trees and plants or mushrooms, with no brain and less pain, perhaps, and keep our DNA moving along forward. But it happened to be that way.
Perhaps there is some survivorship impact from AI on different species on different planets, but from our vantage point we have no idea whether it matters at all and we wouldn’t for a long time.
I feel like you may not understand natural selection all that well. Please read more Dawkins.
Capitalism, for one. Also social pressure on scientists to succeed at their chosen field. Throw in a strong dose of international politics (we better get it before the other guys), and there you have a strong set of selective pressures driving humans to create AI.
Will indeed read more Dawkins. It’s always a pleasure.