A Model of a Mind
tylerneylon.com
tylerneylon.com
Context for the article: I'm working on an ambitious long-term project to write a book about consciousness from a scientific and analytic (versus, say, a meditation-oriented) perspective. I didn't write this fact in the article, but what I'd love to happen is that I meet people with a similar optimistic perspective, and to learn and improve my communication skills via follow-up conversations.
If anyone is interested in chatting more about the topic of the article, please do email me. My email is in my HN profile. Thanks!
Too many people have written books about consciousness. There's much tail-chasing in that space, all the way back to Aristotle. Write one about common sense. Current AI sucks at common sense. We can't even achieve the level of common sense of a squirrel yet.
Working definition of common sense: getting through the next 30 seconds of life without a major screwup.
Others define it as "knowledge, judgement, and taste which is more or less universal and which is held more or less without reflection or argument", which LLMs absolutely do demonstrate.
What you ask for, "getting through the next 30 seconds of life without a major screwup", would usually, 99.7% of the time, be passed by the Waymo autopilot.
That speaks well of the Waymo autopilot, which does considerably better than that. Tesla "self driving", not so much.
Tesla may be the same or not, but they aren't publishing current numbers where I could find them, only really old numbers.
Not that the exact value matters, because so far as I can tell the only autopilot worse than your threshold was the Uber one that crashed into a pedestrian pushing a bike. Even Tesla's 2016 version beat that.
> In 1949 Time described it as "the closest thing to a synthetic brain so far designed by man".
It will be a good practice to see how deeply terms can be applied in order to combat this gap.
I can conceive of scientific experiments involving consciousness. For example:
Hypothesis: Consuming LSD gives me a hallucinatory experience.
Method: Randomized, blind trial. Every Saturday morning I consume one tab of either LSD or water, sit in a blank white room with a sitter (who does nothing), and record my experience.
Results: Every time after consuming water, I have no visual hallucinations and get bored. Every time after consuming LSD, I see shifting colour patterns, imagine music playing on the walls, and feel at one with the world.
Conclusion: Results strongly support hypothesis.
I'd be keen to understand those limitations of phenomenonology and how they apply to this specific experiment, if you have an opportunity to expand.
I'm not sure if that would apply directly to your proposed experiment, but my concern is that what we experience as memories can sometimes apparently be generated on the fly rather than accessing something that was there before.
In general, these kind of phenomenological studies suffer from subjectivity, and are very difficult to connect to explanatory mechanisms, without additional objective evidence. One relevant approach to deal with this is Dennett's Heterophenomenology[3], which seeks to combine the subject's own impressions with other external evidence - so in your case, instead of having the sitter do nothing, you may want to pay them to gather additional objective evidence.
[0] https://en.wikipedia.org/wiki/Confabulation
[1] https://en.wikipedia.org/wiki/False_memory
I suppose if the sitter confirms that they and I stayed inside the cell, and that nothing else came in or out for the duration, that should address that issue? There could be cameras and microphones set up to record the lack of weird trippy phenomena. And I could record my own perceptions in the moment, either by writing it down or speaking into a dictaphone.
The brain is necessary and sufficient cause for subjective experience. If you have a normal brain, and have subjective experience, you should have near total certainty that other people's reports of subjective experience - the millions and billions of them throughout history, direct and indirect - are evidence of subjective experience in others.
Any claim of solipsism falls apart; to claim uncertainty is to embrace irrationality. In this framework, if you are going to argue for the possibility of the absence of consciousness in others who possess normal human brains, it is on you to explain how such a thing might be possible, and to find evidence for it. All neuroscience evidence points to the brain being a necessary, sufficient, and complete explanation for consciousness.
Without appealing to magic, ignorance of the exact mechanism, or unscientific paradigms, there exist no arguments against the mountains of evidence that consciousness exists, is the usual case, and likely extends to almost all species of mammal, given the striking similarity in brain form and function, and certain behavioral indicators.
Cases against this almost universally spring from religion, insistence on human exceptionalism, and other forms of deeply unscientific and irrational argument.
I can say, with a posterior probability of certainty exceeding 99.99999999% that a given human is conscious, simply by accepting that the phenomenon of subjective experience I recognize as such is not the consequence of magic, that I am not some specially endowed singular creature with an anomalous biological feature giving me subjective experience that all others, despite describing it or behaving directly or indirectly as if it were the case. Even, and maybe even especially, if the human in question is making declarations to the contrary.
Consciousness is absolutely subject to the scientific method. There's no wiggle room or uncertainty.
Quantum tubules, souls, eternal spirits, and other "explanations" are completely unnecessary. We know that if you turn the brain off (through damage, disease, or death) all evidence of consciousness vanishes. While the brain is alive and without significant variance in the usual parameters one might apply to define a "normal, healthy, functioning brain", consciousness happens.
Plato's cave can be a fun place to hang out, but there's nothing fundamental keeping us there. We have Bayesian probability, Occam's razor, modern neuroscience, and mountains of evidence giving us everything we need to simply accept consciousness as the default case in humans. It arises from the particular cognitive processes undergone in the network of computational structures and units in the brain.
Any claims to the contrary require profound evidence to even be considered; the question is all but settled. The simplest explanation is also the one with the most evidence, and we have everything from molecular studies to behavioral coherence in written and recorded history common to nearly every single author ever to exist.
I find that disputes almost inevitably stem from deeply held biases toward human exceptionalism, rooted in cultural anachronisms, such as Plato's Allegory of the Cave. We could have left the cave at almost any point since the enlightenment, but there is deep resistance to anything challenging dogmatic insistence that humans are specially appointed to cognition, that we alone have the special magic sauce that make our lives important, but the "lesser" animals morally and ethically ours to do with as we will.
Whenever we look more deeply into other mammal cognition, we find structural and behavioral parallels. Given hands, facile lips and vocal apparatus, and a comparable density and quantity of cortical neurons, alongside culture and knowledge, absent disruptive hormonal and biological factors, there don't appear to be any good reasons to think that any given mammal would not be as intelligent and clever as a human. Give a bear, a whale, a monkey, a dolphin an education with such advantages and science suggests that there is nothing, in principle, barring them from being just as intelligent as a human. Humans are special through a quirk of evolution; we communicate complex ideas, remember, reason, manipulate our environment, and record our experiences. This allows us to exert control in ways unavailable to other animals.
Some seemingly bizarre consequences seem to arise from this perspective; any network with particular qualities in connective architecture, processing capacity, external sensors, and the ability to interact with an environment has the possibility of being conscious. A forest, a garden, a vast bacterial mat, a system of many individual units like an ant colony, and other forms of life may host consciousness comparable to our own experience. Given education and the requisite apparatus, we may find it possible to communicate with those networks, despite the radically alien and disparate forms of experience they might undergo.
If your priors include mysticism, religion, magic, or other sources outside the realm of rational thinking, this might not be the argument for you. If you don't have a particular attachment to those ways of thinking, then recognize where they exert influence on ideas and update your priors all the way; brains cause consciousness. There's nothing particularly magical about the mechanics of it, the magic is all in the thing itself. By understanding a thing, we can aspire to behave more ethically, that we can include all forms of consciousness in answering the questions of how to make life as good a thing as possible for the most people... and we might have to update what we consider to be people to include the lions, tigers, and bears.
I guess each of us, individually, has to, at some point in time, come to a version of an answer that we can live with. I find yours neither the worst nor the best I've ever heard, but I'm glad you have it.
I think he's telling you to do that work yourself, because he doesn't want to do it and he's also not the best equipped to do it either.
This sounds like just another way of saying "I don't like your conclusions, but I can't see a good way to refute them."
No, I don't think so. It sounded more like "your conclusions are not new and wrong, but to refute them would require more effort than I'm willing to invest in replying to a rando's internet comment," which is a totally legitimate stance.
It's sort of like coming across someone in a forum who claims to have invented a perpetual motion machine, noting that they lack fundamental knowledge, but declining to volunteer to be their Physics 101 teacher.
I think one weak spot is the assumption that the physical hardware for speech (e.g. lips & vocal chords) are to some extent necessary and sufficient for language. I think we know with pretty high degree of certainty (by arguments parallel to your other points) that <i>specialized brain circuits</i> are key to language use, and the physical means are secondary.
You may also want to consider why you implicitly draw a line at "mammals" and if this is really justified. The line might belong at "primates" or at "vertebrates" or...the whole idea of a line might be mistaken.
Also, kudos for inviting criticism. If more people did that, we'd all be better off (at least, I believe this to be the case).
Granting an animal the cognitive software of our culture means they only need speech to benefit from abstractions otherwise impossible to develop without humans.I focus on mammals due to the shared structure of our brains. Humans are conscious, and we must identify a specific phenomenon, network, circuit, or class of neurons responsible for consciousness. Since rat, human, and whale brains are incredibly similar, it's highly probable they have similar subjective experiences. We can't definitively rule out consciousness in different mammalian species due to our limited understanding of consciousness mechanics.
Some birds and lizards exhibit conscious and intelligent behaviors, suggesting different brain structures can serve similar functions. An Etruscan shrew has the smallest brain of any mammal, and there might be a threshold of capacity and architecture required for consciousness. If consciousness depends only on architecture, shrews might be conscious in a way we can relate to.
We don't fully understand consciousness yet; untangling the factors is a complex problem. Bayesian probability suggests that if your brain causes your subjective experience, other animals with similar brains are also conscious. Their behavior and artifacts, such as emotion, psychology, and intricate devices, indicate consciousness. Consciousness includes not just self-awareness or theory of mind but the second-by-second experience and interpretation of sensory constructs.
This idea challenges dualistic explanations. Religion, simulation theory, metaphysics, mysticism, and other explanations often arise in the absence of a clear understanding. However, one's conscious experience as a fundamental prior suggests a material basis for consciousness. Abandoning the bias of human exceptionalism is necessary for a cold, clinical view.
Consciousness seems to be a spectrum. Helen Keller described moving from a continuous, borderless blur of experience to concrete, episodic sequences with clear self-other delineation. Primitive humans, biologically similar to us but without language, would have experienced life differently - much like Helen's pre-language blur of the eternal now. Whales and other large-brained mammals might experience life similarly. If we discover their language, or a way of mapping engrams of their experience onto language, it will reflect a different way of experiencing things. The same applies to forests, giant mycelial networks, and colony insects, where we might find the computational circuits underlying consciousness and cognition.
We don't get to explore those things if we insist on staying in the cave and theorizing; I'd love for someone to point out I've wandered down a dead end, but everything I've read and considered so far leads me to believe that there's no value in distinguishing one's own consciousness from all other things scientific. Treat it as the evidence it is; if you need to update, update all the way.
> This is deeply incorrect; it's been a claim made for thousands of years, but it doesn't stand up in the face of Bayesian rationality.
1. In this case, how is "Bayesian rationality" implemented (literally implemented(!)...what are the inputs, what are the outputs, what is the source code of the algorithm)?
2. What is the meaning of "doesn't stand up in the face of"?
The point of solipsism isn't to actually reject other minds. I'd argue that's quite a useless belief to hold, it provides no predictive power and no additional ways to engage with the world around one self. But there are any number of other equally valid interpretations of the only evidence one has access to: the fact that you yourself are conscious and that you receive input from a remarkably consistent source of information.
Not all of those interpretations provide the same tools of thought, but some are equivalent in power: For instance you could reject other minds and instead posit that your perception of them is a sophisticated construct created by your own consciousness to simulate interaction with a complex and consistent world. Alternatively, you might consider that other minds are simply facets of your own mind, reflecting your subconscious processes. These constructs are similarly axiomatic, they're unfalsifiable.
A lot of the rest of your argument feels like a straw-man to me. I don't fundamentally disagree with you—I also accept the existence of other minds because it provides predictive power and useful tools of thought. But writing-off people who question this axiom as "irrational", you're just defining "rationality" to include the axiom of the existence of other minds. And "irrational" is a loaded term when used to describe a person or their beliefs.
Pinker advocates for a model of mind that has multiple streams of consciousness. In Pinker's model of mind, there are multiple agents constructing models of the world in parallel, each trying to predict future states from current state plus current input data. A supervisory process then selects the model that has made the best prediction of current state in the recent past for use when reacting in the current moment. The supervisor process is free to switch between models on the fly as more data comes in.
Pinker grounds his model of mind in curious observations of what people remember, and how our short term memories change over a period of sometimes many seconds as our mind switches between different agent interpretations of what's going on. Witnesses are notoriously unreliable. Pinker concerns himself with why and how witnesses are unreliable, not for legal reasons, but for how those unreliabilities might reveal structure of mind. Pinker's most interesting observation (I think) is that what we seem to remember is output from the models rather than the raw input data; and that what we remember seeing can change dramatically over a period of many seconds as the supervisory process switches between models. Notably, we seem to remember, in short term memory, details of successful models that make "sense", even when those details contradict with what we actually saw. And when those details are moved to longer-term memory, the potentially inaccurate details of model are what are committed.
Quoted from rom https://www.newyorker.com/magazine/2017/03/27/daniel-dennett...
Regarding the multiple layers: the most interesting thoughts I read about theories of a mind, are the books by Marvin Minsky, namely: The Society of Mind and The Emotion Machine which should be more widely known.
More on Minsky's ideas on “Matter, Mind, and Models” are mentioned in https://www.newyorker.com/magazine/1981/12/14/a-i
In particular, I think there's a nice overlap between my description of self-awareness and yours. You mention a model capable of seeing its own output, but I take it a step further and indicate model capable of generating its own simulated actions and results. Curious what you think, thanks!
There's a theory that real brains subvert this, and what we perceive is actually our internal model of our self/environment. The only data that makes it through from our sense organs is the difference between the two.
This kind of top-down processing is more efficient energy-wise but I wonder if it's deeper than that? You can view perception and action as two sides of the same coin - both are ways to modify your internal model to better fit the sensory signals you expect.
Anyway, I guess the point I'm making is you should be careful which way you point your arrows, and of designating a single aspect of a mind (the action centre) as fundamental. Reality might work very differently, and that maybe says something? I don't know haha.
Since the author here says this is not just a random blog post.
To me, this would correct the introduction's reference to "AI-based language models". Which is a bit too "latest fad"; But which I do find fascinating because the success of language models hint that a very large part of the basic human's intelligence may be simply internalized language (much of the rest being still more pattern recognition.); But which cannot merely erase all other prior work on what makes our minds.
It is just meta self-awareness.
We evolved to be aware, model & react and to opportunistically control our environment. Loop 1.
Then we evolved to be aware, model, & & react and to opportunistically control our selves as bodies. Loop 2.
Then we evolved to be aware, model, & & react and to opportunistically control those activities too, our minds. In the process of being able to model & react to ourselves modelling & reacting to ourselves we completed a self-awareness of self-awareness loop. Loop 3.
I think this is a good explanation because what else is consciousness but knowing you are conscious? Self-aware that you are self-aware?
And it makes perfect sense as an evolutionary trajectory using greater levels of neural representation of our practical relationship with our environment, including ourselves, to create higher level survival options.
So this definition is both a functional and developmental explanation of the emergent phenomenon of consciousness.
Just as we only have partial access to information and control of our environment, our bodies, we are also limited to the degree we are aware and can control our mind. Introspection.
The next level up is “theory of mind”, “empathy”, etc. our ability to model & interact with others mind’s as individuals and groups, reciprocally. Loop 4. That created society and culture, living information that extends outside us and grows and lives beyond each of us.
Infusing our thoughts, and progressively thinking abilities into technology, that in theory and not too distant practice could have access to all parts of their own minds’ states, operations and design, would be Loop 5. When deeply conscious beings start, things will get interesting in the Sol system.
When people start talking about quantum waves or smart rocks or universes in the context of consciousness I feel a little ill. People like to “solve” unrelated mysteries by merging them and shouting “tada!” (“We are puzzled about how they built the pyramids… but we don’t know if there are aliens either, so obviously… these mysteries solve each other!”)
I.e. qualia. Whether that is of a color, the cold, or how it feels to be self-aware.
I think that will become tractable with AI, since we will be able to adjust what their mind has access to vs. what it doesn’t. Experiment with various levels of information awareness.
Qualia would seem to be a functionally necessary result of a self-awareness dealing with information encoded at a lower level, with no access to that encoding.
Information is provided, the coding is inaccessible, so it gets perceived as … something which it cant decompose. So can’t describe.
We are made aware of a signal we perceive as red, but cannot mentally decompose perceptual “red” into anything else.
So the question is, how does differently encoded information provided to our self-aware level, without any access to the coding, get interpreted as the specific qualities we perceive?
Wild thoughts:
Once we understand how qualia emerge, what would the qualia space” look like? What are the rules or limits?
Will we be able to somehow analyze a creature and infer its qualia in a way that we (or an AI) can then experience directly?
Will designing qualia for AI be a thing? A useful thing? Or are they simply isomorphic to the type and relationships of data they represent? Or just a temporary glitch on the way to more fully self-aware, self-observable, self-designed life?
"I think, therefore I am"
I think Descartes, if he were alive today would accept that slight adjustment. Descartes would of course then go on to rewrite Meditations II (which immediately follows "I think therefore I am") to argue that there might be an evil daemon that writes log files so that they give the illusion that I'm thinking. But if evil daemons are able to forge log files then all knowledge is impossible, so we're doomed no matter what. So best to pretend that isn't a possibility.
It always irritates me a bit that people like to throw "I think therefore I am" around as if Descartes himself didn't immediately refuted in Meditations II.
More specifically than that: "My ability to read the log file proves my existence"
Before that you can look into the AGI conference people like Ben Goertzel, Pei Wang. And actually the whole history of decades of AI research before it became about narrow AI.
I'd also like to suggest that creating something that truly closely simulates a living intelligent digital person is incredibly dangerous, stupid, and totally unnecessary. The reason I say that is because we already have superhuman capabilities in some ways, and the hardware, software and models are being improved rapidly. We are on track to have AI that is dozens if not hundreds of times faster than humans at thinking and much more capable.
If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
Don't get me wrong, I love AI and my whole life is planned around agents and AI. But I no longer believe it is wise to try to go all the way and create a "real" living digital species. And I know it's not necessary -- we can create effective AI agents without actually emulating life. We certainly don't need full autonomy, self preservation, real suffering, reproductive instincts, etc. But that is the goal he seems to be down in this article. I suggest leaving some of that out very deliberately.
> it will actually out-compete us for resource control. And will no longer be a tool we can use.
I’ve never been convinced that this is true, but I just realised that perhaps it’s the humans in charge of the AI who we should actually be afraid of.
We should anticipate something like that if we really replicate humans in a digital format. I am suggesting that we can continue to make AI more useful and somewhat more humanlike, but avoid certain characteristics that make the AI into truly lifelike digital animals with full autonomy, self-interest, etc.
Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.
An intelligent entity will want to survive, and will realize humans are necessary cells for its survival. A big dog robot with nukes might not care, but I wouldn’t call that AI in the same sense.
I don't see how that follows at all. You could have an intelligent entity, in the sense that you can ask it any question and it'll give a highly competent response, that is nevertheless completely indifferent about survival. Biological organisms have been formed by selection for survival and reproduction, so they have they operate accordingly, but I don't see the justification for generalising this to intelligences that have been formed by a different process. You would need some further assumptions on top of their intelligence - e.g. that their own actions are the dominant factor in whether they survive and replicate, rather than human interests.
Or, you know, Alan Turing eating the apple. I think he was a pretty smart guy.
I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down.
AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.
In my opinion, this is easily noticeable when you try to discuss any system, be it political or economical, that spans multiple countries and interests. People will just revert to whatever is closest to them, rather than being able to foresee a larger cascading result from some random event.
Perhaps this is more of a rant than a comment, apologies, I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Just thinking out what could happen in the next 5, 10 and 99 years.
Absolutely. Happy to be part of it if you are able to set it up.
Could you expand on what you mean by this? Specifically, is it OK with you if progress in AI causes the death of all the original-type human people like you and I?
Positioning the animal known as 'human' as some God-like entity that must survive at all costs, is extremely arrogant if you ask me. Obviously I wish for humanity to thrive and survive, as this is self preserverance and a bit of pride or ego. But the notion that we are special in some way just rubs me the wrong way and doesn't help think ahead on a large scale and timeline.
It's in human hands, we can hardly trust the enemy or even ourselves. We already came close to extinction a couple of times.
I presume when ASI will emerge one of its top priorities will be to stop the crazies with big weapons from killing us all.
It would require a civilization to consciously bond with its capability to do so (in such a way that it enhances the survival of the humans serving it). Not sure this would be competition in the normal sense.
I think the bigger problem is that too many people are focused on short term things like personal wealth or glory.
The guy who make the breakthrough that enables the AGI that destroys humanity will probably win the Nobel Prize. That potential Nobel probably looms larger in his mind than any doubts that his achievement is actually a bad thing.
They guy who employs that guy or productionizes his idea will become a mega-billionaire. That potential wealth and power probably looms larger in his mind than any doubts, too.
This is a very strong argument. Certainly all the ingredients to replicate a mind must exist within our physical reality.
But does an algorithm running on a computer have access to all the physics required?
For example, there are known physical phenomena, such as quantum entanglement, that are not possible to emulate with classical physics. How do we know our brains are not exploiting these, and possibly even yet unknown, physical phenomena?
An algorithm running on a classical computer is executing in a very different environment than a brain that is directly part of physical reality.
This is wrong. You can even get a mod for Minecraft which implements quantum mechanics.
https://www.curseforge.com/minecraft/mc-mods/qcraft-reimagin...
> In the current paper, we make use of the recently published work in quantum information theory by Candela to have students write code to simulate the operation of the device in that article. Analysis of the device has significant pedagogical value—a fact recognized by Feynman—and simulation of its operation provides students a unique window into quantum mechanics without prior knowledge of the theory.
This is immaterial, however. It is a well known fact that BQP is in PSPACE and Clifford circuits (a subclass of quantum circuits) can not only be simulated classically, but done so efficiently. It is not controversial.
Indeed, the whole point of Mermin's device is to give a very simple illustration for how it is impossible to replicate the behaviour of two entangled particles using classical particles (with hidden variables).
Now is this specific characteristic of entanglement an absolute requirement for quantum computing speedups? Could we have similar speedups with probabilistic hidden-variable algorithms? Probably not, but it is a good question. It is true that if you spend time reading research papers in the field, it is still not clear what the edge is between problems that can be sped up by quantum computers and which cannot, or if there is even an edge at all.
P ⊆ BPP ⊆ BQP ⊆ PSPACE
BQP problems can be solved on quantum computers in polynomial time, some of these problems may be outside of P and BPP (Bounded-error Probabilistic Polynomial-time), so they may not be possible to solve in polynomial time in classical computers, even with probabilistic algorithms.
It is true that there's still room for BPP = BQP, that has not been disproven, but it is somewhat controversial to expect so, at this point many smart people have spent their lifetimes prodding at it.
The only way to simulate accurately on a classical computer is to use global state but this goes against the instruction that the devices must be isolated from each other.
> This is immaterial, however. It is a well known fact that BQP is in PSPACE and Clifford circuits (a subclass of quantum circuits) can not only be simulated classically, but done so efficiently. It is not controversial.
Yes, BQP problems are solvable and a "subclass" of quantum circuits can be simulated efficiently. But the fact is there are known aspects of reality that cannot be simulated on a classical computer.
No shit. Of course you can't take a simulation method that takes exponential running time in terms of the size of the thing you're simulating (two Mermin devices), then simulate each half (each Mermin device) independently. If you could split it up like that you'd have a polynomial time simulation method!
QC researcher here, strictly speaking, this is false. Clifford circuits can be efficiently simulated classically and they exhibit entanglement. The bottom line is we're not entirely sure where the (purported) quantum speedups come from. It might have something to do with entanglement, but it's not enough by itself.
Re: about mermin's device, im not sure why you think it can not be simulated classically when all of the dynamics involved can be explained by 4x4 complex matrices.
https://en.wikipedia.org/wiki/Soar_%28cognitive_architecture...
How would it intelligently do this? What data would you train on? You don't have trillions words of text where humans wrote what they thought silently interwoven with what they wrote publicly.
History has shown over and over that hard coded ad hoc solutions to these "simple problems" never work to create intelligent agents, you need to train the model to do that from the start you can't patch in intelligence after the fact. Those additions can be useful, but they have never been intelligent.
Anyway, such a model I'd call "stream of mind model" rather than a language model, it would fundamentally solve many of the problems with current LLM where their thinking is reliant on the shape of the answer, while a stream of mind model would shape its thinking to fit the problem and then shape the formatting to fit the communication needs.
Such a model as this guy describes would be a massive step forward, so I agree with this, but it is way too expensive to train, not due to lack of compute but due to lack of data. And I don't see that data being done within the next decade if ever, humans don't really like writing down their hidden thoughts, and you'd need to pay them to generate data amounts equivalent to the internet...
It's a fair question, and I don't have all the answers. But for this question, there might be training data available from everyday human conversations. For example, we could use a speech-to-text model that's able to distinguish speakers, and look for points where one person decided to start speaking (that would be training data for when to switch modes). Ideally, the speech-to-text model would be able to include text even when both people spoke at once (this would provide more realistic and complete training data).
I've noticed that the audio mode in ChatGPT's app is good at noticing when I'm done speaking to it, and it reacts accurately enough that I suspect it's more sophisticated than "wait for silence." If there is a "notice the end of speaking" model - which is not a crazy assumption - then I can imagine a slightly more complicated model that notices a combination of "now is a good time to talk + I have something to say."
All evidence points towards human reason as a fundamentally different approach, orders of magnitude more efficient at integrating and making sense of ridiculously smaller amounts of training data.
Marketing, and a bit of collective delusion by a lot of people having the "can't understand what they are paid not to" thing going on.
And all evidence actually points toward human reason as an incredibly inefficient and horrifyingly error-prone approach, that only got as far as it did because we're running 8.1 billion human minds in parallel.
While evidence suggests that human reasoning uses a fundamentally different approach, it remains to be seen whether human reasoning uses a fundamentally superior approach.
I have yet to see a single AI system that can learn to produce the word "mama" after doing fully self supervised training, and being fed only the cosine transform of the audio it produces and a few hundred hours of video/audio feed showing a mom saying the word and becoming very happy when the word is finally uttered. Did I mention the output must be produced using an array of mechanical oscillators, resonance chambers and bellows with unknown and highly variable acoustic parameters, that need to be discovered and tuned at runtime?
I have seen this "human intelligence training is wasteful" line and I think it is complete nonsense. The efficiency with which humans can acquire any language with barely any training data is unfathomably better than large scale statistical models.
For instance, the idea that we can neatly have the emotion system separate from the motor control system. Emotions are a cacophony of chemicals and signals traversing the entire body - they're not an enum of happy/angry/sad - we just interpret them as such. So you probably don't get to isolate them off in a corner.
Basically I think it's very tempting to severely underestimate the complexity of a problem when we're still only in theory land.
Very much looking forward to seeing continuing progress in all this.
https://kar.kent.ac.uk/21525/2/A_theory_of_the_acquisition_o...
Memory Organisation Packets might also deal with issues encountered.
https://www.cambridge.org/core/books/abs/dynamic-memory-revi...
When intelligent machines are constructed, we should not be surprised to find them as confused and as stubborn as men in their convictions about mind-matter, consciousness, free will, and the like.
Minsky, as quoted in https://www.newyorker.com/magazine/1981/12/14/a-i
What is missing from this picture is the social aspect. No agent got too smart alone, it's always an iterative "search and learn" process, distributed over many agents. Even AlphaZero had evolutionary selection and extensive self play against its variants.
Basically we can think of culture as compressed prior experience, or compressed search.
Although it's not spelled out in the article, I'm hoping that the feature of agency along with an emotional system would enable constructive social behavior. Agency is helpful because it would empower AI models to meaningfully speak to each other, for example. Human emotions like empathy, social alignment, curiosity, or persistence could all help AI models to get along well with others.
I think the contradiction you see is that the model would have to form a completion to the external input it receives. I'm suggesting that the model would have many inputs: one would be the typical input stream, just as LLMs see, but another would be its own internal recent vectors, akin to a recent stream of thought. A "mode" is not built in to the model; at each token point, it can output whatever vector it wants, and one choice is to output the special "<listening>" token, which means it's not talking. So the "mode" idea is a hoped-for emergent behavior.
Some more details on using two input streams:
All of the input vectors (internal + external), taken together, are available to work with. It may help to think in terms of the typical transformer architecture, where tokens mostly become a set of vectors, and the original order of the words are attached as positional information. In other words, transformers don't really see a list of words, but a set of vectors, and the position info of each token becomes a tag attached to each vector.
So it's not so hard to merge together two input streams. They can become one big set of vectors, still tagged with position information, but now also tagged as either "internal" or "external" for the source.
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.
If that was the case, people who were born blind would demonstrate markedly reduced intelligence. I dont think that is the case, but you can correct me if I am wrong. A blind person might take longer to truly 'understand' and 'abstract' something but there is little evidence to believe that capability of abstraction isnt as good as people who can see.
Agree that sensory inputs and interaction were absolutely critical for how the minds evolved, but model training replaces that part when we talk about AI, and not just the evolution.
Evolution made us express emotions when we are hungry for example. But your laptop will also let you know when its battery is out of juice. Human design inspired by evolution can create systems which mimic its behaviour and function.
No, because the mind of a blind person, even one blind from birth, is still the product of a billion years of evolution of organisms that had sight, sound, touch, smell, etc.
Not to mention, a person who has no sensory input at all (no sight, no sound, no touch, no smell, no taste, nothing at all) is unlikely to have a fully functioning mind. And certainly a baby born like this would not be able to learn anything at all.
Of course, the situation is not 1:1 by any means to AI training, as AI models do get input, it's just of a vastly different nature. It's completely unknown what would happen if we could input language into the mind of an infant "directly", without sensory input of other kinds.
Still, I think it's quite clear that humans minds are essentially born "pre-trained", with good starting weights, and everything we do in life in essentially fine-tuning those weights. I don't think there's any other ways to explain the massive input difference (known as the poverty of the stimulus problem in cognitive science). And this means that there is little insight to draw for better model training from studying individual human learning, and instead you would have to draw inspiration from how the mind evolved.
Hard disagree. Evolution made a bigger/better neural processor, and it made better/different I/O devices and I/O pre-processing pipelines. But it didn't store any information in the DNA of the kind you're proposing. That's not how it works. The brain is entirely "field programmable", in all animals (I assert). There is no "pre-training".
This is all somewhat hard to gage in human babies, as we take a relatively long time to become functional. However, it's clear when looking at many other mammals - baby reindeer or horses, for example, are able to run within minutes of being born; they can see, they can interpret the images they see as objects, they understand things like object permanence, they can approximate distances and speeds, they have a simple theory of mind and can interact with other agents, they can recognize their mother's udders and suckle at them for food, and many many other tasks that they have 0 training for. The only possible conclusion is that their brains are pre-trained, and they are only performing some quick fine-tuning based on experience in their first hours of life.
I think the analogy of the brain as hardware devices ("neural processor", "I/0 devices", etc) is misleading. I think I understand the very strict mind-matter dualism you're alluding to here. But so far attempts at using actual computer hardware to reproduce human-like cognition has gotten nowhere close, despite consuming order of magnitude more energy and data.
But of course we can be assured it’s not quite like that in reality. This is just another example of how our models for explaining the life are reflection of the current technological state.
Nobody considers that old clockwork universe now, and these AI inspired ideas are going to fall short all the same. Yet, progress is happening and all these ideas and talks are probably important steps that carry us forward.