Why This AI Moment May Be the Real Deal
thenewatlantis.com
thenewatlantis.com
That was ~26 yeas ago. In the intervening 26 years humans have been curb-stomped so hard at chess that certain microwaves might be competitive against human grandmasters. Anyone who doesn't think that was an exciting moment for AI research has impossible standards, we're living in a tiny window of existence where humans were at the top of the intellectual food chain and are now dropping to also-ran status along with the rest of the meat-based animal kingdom.
All these moments are the real deal. The idea that somehow this particular step change is even unexpected is missing the forest for the trees. The trends that have been observed since the beginning of computers are chugging away so quickly the whole process may well happen in a single generation.
Robotics is behind in making a 'human'.
Even AlphaGo had a human moving the pieces, and nobody would say that was evidence of any 'lack' in the AI.
It's just an interfacing issue, engineering will eventually solve.
Edit> What I was getting at. Take Robots as the physical interface layer for the AI. AI advancements are currently outpacing advancements in Robotics. So potentially AI could use real humans as the 'hands and feet' to do things in the physical world, by giving directions. Something will be needed to bridge the gap until Industry can be done completely by robots. Until then, AI wont be able to manufacture itself.
If you have ever worked in a modern warehouse, it is very evident, the humans are just following instructions from the system. Now scale that up to a manufacturing facility, it is already happening.
Everybody with an ear-piece or 'phone' type device, following instructions.
> Workplace democracy, AKA unionization, AKA the thing Amazon has pulled out every dirty trick to prevent, is the difference between centaur utopia and reverse-centaur dystopia.
-- https://pluralistic.net/2021/02/17/reverse-centaur/#reverse-...
Also, we were talking about deep blue. Perhaps trying to not go OT would be beneficial to a productive discussion.
We're building tools to help us augment our intelligence, I don't see anyone getting "curb stomped"?
If intelligence leads to more violence, then I don't think it will be as useful as we'd have hoped.
> [Hobbes] even, through sheer force of imagination, was able to outline the main psychological traits of the new type of man who would fit into such a society and its tyrannical body politic. He foresaw the necessary idolatry of power itself by this new human type, that he would be flattered at being called a power-thirsty animal, although actually society would force him to surrender all his natural forces, his virtues and his vices, and would make him the poor meek little fellow who has not even the right to rise against tyranny, and who, far from striving for power, submits to any existing government and does not stir even when his best friend falls an innocent victim to an incomprehensible raison d'etat.
-- Hannah Arendt
And that only applies to criticizing phrases like "killing it", without even demanding it to not be used -- but not to bringing out Nineteen-Eightyfour? And doublespeak? No, doublethink means holding two contradictory things to be true at the same time, doublespeak is the verbalization of that. You're thinking of wrongthink, and either way you are doing what you criticize. Instead of saying "this is horrible" you say "this is dystopian", it changes nothing.
More importantly, your reply contains nothing specifically applying to my comment. Maybe you missed the point, which is that people delude themselves to compensate their lack of agency, being restricted to partake in a rat race that will leave them empty-handed in the end -- all of which Hannah Arendt explains in the context, but I didn't want to post a wall of text. That they don't actually mean what they say, because what they mean (and do) would be to pitiful to say without such flourish, and that they aren't actually violent and powerful, but "poor meek little fellows", is precisely the point.
Over time the original meaning may fade or be forgotten, and the residual symbol in the language becomes the thing itself. Etymology is full of examples.
But that much history has not yet passed since the introduction of the term "curbstomping" and so using it is still a choice that conveys some meta-meaning about content and style.
As mentioned further up in the thread it is unnecessarily violent for conveying the meaning in the comment.
No reason to make it an issue, people just like using those metaphors.
> beat Kasparov
So much violence everywhere
I have trouble articulating it, but I can see that LLMs and chess engines don't do the slightest hint of it.
I'll be concerned when machines show the slightest hint of it, but we're playing a ball game in a sphere they can't even touch yet.
I'm genuinely continually boggled that people are not seeing this difference and feeling threatened by machine "intelligence", although they are encroaching on something thought uniquely human, and I don't think the brain is doing some kind of information processing that can't be done in any other medium, but the number of connections and kind of processing in the human brain may make it hard with current technology. The number of connections in the brain is staggering, and we certainly can't simulate anything of its scale on the scale of computer we have now, so I don't know what it will take to get any bootstrap on the kind of intelligence computers aren't doing.
Ie we will be able to describe how a function of `add(1,2)` is most clearly definitely not intelligent. We will understand the internal makeup of these neurons better, the whole system will be deterministic and understood - just a very large `add(1,2)` .. so it's definitely not intelligent, of course.
Yet it will still produce output that is fully and truly turing-proof. If you let it run, it retains, stores, "behaves", etc. It's clearly just `add(1,2)` in a loop after all.. we will understand it so well that it's obviously just a trick of our mind that it seems real.
The conclusion in that space will not be one of the machine .. but of us. If we can't tell it apart from us.. and if we know it is most definitely not alive or even mysterious.. what does that say about us? The existential crisis will be neat.
Less than people think. We know that being a biological system differentiates us. If intelligent systems tend to converge where language is concerned, so be it. It doesn't mean the AI warrants any special consideration. It's still a tool.
That concepts like "consciousness," "free will," and "qualia" are nebulous precisely because they were invented by us to describe things we don't understand yet we feel obligated to insist are special and unique characteristics of our identity.
"Can you pay attention to this part of your experience?" is a very black-and-white, non-nebulous question with an easy yes or no answer. Not sure what you're reading into this.
Consider a video camera. It seems like it is a consuming a "stream of experiences". But most people wouldn't consider it it conscious. Why not under your definition? It works when on, and is stops when turned off. Black and white. But clearly we are missing some nuance in the details. Maybe we also require some interpretation of the experiences? I can spin a convolutional neural net that identify objects within a video: interpretation. Does that make it conscious? No? Then we need something else to define it. Maybe response to stimuli? We can program machines for that, but then also things like fire respond to environment interactions too. And so on. It really isn't black and white after all.
What does "pay attention" mean?
I've come to suspect that a habit of using the passive voice when discussing these phenomna contributes immensely to their ongoing currency. Steering away from this by asking questions like, "_who_ or _what_ is attending to/experiencing the stream?" may be a useful way to ground these statements in material terms.
Consciousness, on the other hand, boils down to what gives rise to the sensation of self-awareness, and what that sensation actually is, and it's not a given that requires any form of dualism (or that any form of dualism would solve the question at all) or that a strict materialist world can't involve self awareness.
For what we know, self-awareness could be a property of matter itself, and the only thing different between "conscious" and "non-conscious" is getting to a level of sufficient reasoning-ability to actually be able to express it.
Nova/PBS recently did a number of episodes on the number of tricks your brain is playing on your at anytime. Once you start seeing the shortcuts the brain takes in action the entire process is far less magical.
Because it's still equally nebulous.
No, its not.
Its the quality of “experiencing”.
Which is impossible to define in objective terms; it is the essence of subjectivity on which all other subjectivity rests. It’s not merely “we don’t know how to determine whether an AI or camera has it”, it is categorically impossible to determine if something has it. The popular narrative about which things in the natural world have it has evolved radically over time, but in no case has there been, or could there be, any empirical basis for this; its all just vibes.
And your definition is now circular and can not be used that way.
Hence the claim that the original definitions were nebulous.
Those seem equally nebulous.
> "Free will" means "a feeling that one could've made a different choice"
That doesn't seem like the ordinary definition of the term, either in philosophy or in everyday usage. I would think most people would define "having free will" as not just feeling that one could have made a different choice, but rather it actually being physically possible for a different choice to have been made.
> "qualia" means "specific characteristics of experiences that one can attend to"
I think those are also equally nebulous.
> One need not "insist" on anything to believe they are unique characteristics of our identity.
The insistence is necessary, and invariably comes up. As soon as I say to someone "given your definitions of these concepts, I don't think they exist" they will say "well, I've started with the foregone conclusion that I cannot deny the existence of these things, so either you're plainly wrong, or we need to tweak our definitions and understanding of these things."
They aren't. It's literally happening right now as you read this - a stream of experiences is present. If that is the case, YES it is consciousness, if nothing is happening, NO it is not consciousness. Black and white - literally as non-nebulous as you can get.
We often discuss things that would challenge or overtake us. Hypothetical advanced aliens, artificial AI, etc. However i don't often see pop culture discussing the opposite - us, or even life, being challenged on it's uniqueness. Plus then that leads to all sorts of self-autonomy discussions (which is discussed in pop culture imo).
I dunno, i suspect "people" won't handle it well. Hell, there's entire religion(s) built on the foundation that we're uniquely gifted as a product of being crafted in a gods image. To question us could question god. I find these questions neat, interesting, etc - but socially i suspect it would be troubling, contentious, frightening.
A lot of the hopes in regard to AGI in LLMs seem to focus on just making the models bigger, or feeding them even more information. On the other hand, some researchers in the AGI space seem to move in a completely different direction, trying to use neural networks to simulate different parts of a brain, moving away from language altogether.
But what I'm wondering is this: might it be a more productive avenue to consider an LLM not as an entire mind, but as a single part of an hypothetical artifical psyche, which excels at some of the overarching tasks of consciousness (such as acting within a cultural context and epistemological framework), but can delegate other aspects to more fitting subsystems? Could it be combined with other networks optimized for different tasks? Potentially, this could just be done by having inputs from other parts of this artificial psyche fed in as prompts, translated by an encoder/decoder of the likes of CLIP, i.e. a decoder that can translate ideas between different liminal spaces.
The work that I keep remembering in this context is Jaynes' Origin of Consciousness in the Breakdown of the Bicameral Mind [1], which I stumbled upon here on HN [2] [3]. Jaynes believes that consciousness is not an innate ability, but a cultural achievement - in order to become conscious, one first has to learn (through others) to conceptualize consciousness, and to think of oneself as a thinking being. He insists that consciousness is not a necessity for human agency, and that humanity has been building civilizations and using technologies long before humans discovered (or achieved) consciousness. Jaynes puts forward the idea of an ancient, non-conscious human mind based on, upon other things, "verbal hallucinations" - thoughts and memories that involuntarily enter as associations, but whose origins are interpreted by the mind to not be within itself, but somewhere outside of it, such as in a divine monologue; thought, to the pre-conscious mind, is the incessant blabbering of the Gods.
It seems that LLMs would excel in the role of this "camera" of the mind - as the source of divine monologue and divine instructions, that can then be compared with memories, interpreted and executed by more specialized subsystems. Might it be feasible to give up the plan to make AGI self-aware altogether and instead build an AGI powered by a GPT-4 ascended to divinity, just as ignorant of its own existence as ever?
[1]: https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in...
I feel like people have this vague idea in their head of what everyone is trying to build, but don't really have it defined in a concrete way.
So basically we're never going to achieve some consensus about whether an LLM/AI could have it, because we can't even agree on whether dogs and cats have it. We can't even all agree on whether all humans have it.
Maybe some day we'll have scientific instruments that will allow one being to 'psychically link' with others and experience the other's subjective sense of being. But until we've got something like that, I think the matter is effectively closed to empirical inquiry.
Well, one way is to observe similarities in the I/O behavior of these various systems, and in particular if they exhibit any evidence that they have goals and desires, and then beyond that if those goals and desires extend beyond merely that necessary to survive and reproduce. So for example, the fact that some humans seem to want to do things like read and write and make art is IMHO is evidence that they are more conscious than salmon.
For instance, with thumbs and a terrestrial lifestyle, humans are more capable of producing art and technology than just about any other animal. But I don't find this to be a persuasive argument for human minds being more sophisticated than orcas. Even if art is the answer, our lack of insight into orca language makes it difficult if not impossible to fairly compare them to us. Maybe Orcas have poetry so sophisticated and beautiful that it puts all human art to shame, but it's completely inaccessible to us because we don't understand their language. Or maybe they're far smarter and more aware than us and could compose such poetry, but choose not to simply because they don't value it.
> if they exhibit ... evidence that they have goals and desires ... beyond merely that necessary to survive and reproduce
Orcas qualify because they play with their food. It's quite gruesome to watch. (And, BTW, this particular behavior is evidence that either orcas do not have a theory of mind or they lack empathy for animals that they eat. The former would be the charitable interpretation.)
What would it mean for humans not to have it? The debate is over the nature of consciousness, not that humans (and many other organisms) lack any sort of consciousness. So when we see color or feel pain, what is that? We are conscious of colored things and painful sensations. There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
I don't believe human P-zombies exist, but how do you empirically prove they don't? Nobody has been able to devise an experimental test to determine whether a person, animal or AI has a conscious experience or whether they mimic all the outward signs.
> There's no real doubt about that, unless you're solipsist about other minds. Which good luck if you are, I guess.
Exactly. It's fair to write off people who get solipsistic about other human beings because those people are fairly rare and are beyond helping anyway. But if we're talking about animals or AIs, the 'solipsist' position becomes more common and there's no real good way of debunking it. Consider the problem facing animal rights activists who wish to persuade the public that fish have feelings. What scientific test for fish having feelings can you use? Put a fish in an MRI and you can see the fish brain doing stuff, but the "fish don't have feelings" camp doesn't find that persuasive. You could try performing the mirror test on fish. Most fish would probably fail it, but what would it really mean if they didn't? The feelingless fish camp would just say the fish behave as though they have a sense of self but actually don't. How do you refute that scientifically? It seems the most effective way to persuade people that fish have feelings is to resort to sophistic tactics; appeals to emotion, bullying and browbeating, or simply just laughing at people when they say fish don't have feelings. Make them ashamed to spread their belief to others. These kind of tactics are distasteful but generally effective.
With AIs or LLMs, the problem is even worse than it is with fish. Fish at least have biological brains, which suggests they have some commonality with us. Fish have faces, so some part of the human brain is wired to empathize with them. We have a fossil record that shows we came from fish, and this relationship perhaps makes it easier to believe that fish have some sort of subjective consciousness experience that is similar to ours. In these respects, AIs are further removed from us than fish are, so it will be even harder to persuade people that AIs could have the same sort of conscious experience as we do. Even after having deep philosophical conversations with an AI in which the AI proclaims to have a conscious experience, a lot of people will say the AI doesn't actually have that and was merely trained to speak as though it does. I don't see any clear cut resolution to this sort of doubt. There's no empirical test that you can perform that gives an indisputable answer one way or the other.
Games.
From the first clever bloke who came up with an algorithm to play a card game against himself (ie, Solitaire), to the Mechanical Turk, to Big Blue, the the ghosts in PacMan, the NPC’s in Counterstrike, etc, etc., there is a fascination with making something that can ‘beat’ us.
Humans have a need to compete.
We’re drawn to the idea of a machine that is so smart and clever that it poses a genuine challenge to our wits.
The ‘real deal’ that most have in mind isn’t some virtual assistant that can help you calendar things or write letters. No, the ‘real deal’ is always described in terms of SkyNet or sinister AI overlords, or malicious intelligent robots.
I don’t think that stuff is hyperbole.
Rather, I think that is precisely what we want… we want something that can destroy us… something, against whom, we can compete with our survival at stake.
In short, the ‘real deal’ has been staring us in the face for centuries; we make movies and write books about it… particularly in Sci-Fi. The goal IS SkyNet. The goal IS HAL. Let’s be honest… that’s what we are shooting for — the ultimate competitor that, unlike anything else on this boring planet, has the wits to take us out.
Whether it’s bored trailer-trash playing Russian Roullete, or bored Billionaires descending to the Titanic in a jerry-rigged sub, we have a very strong need to make our lives interesting by betting our lives themselves.
The purpose of AI is to make a challenger who has the ability to wipe us out, with intent.
That’s the ‘real deal.’
Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
Then you can find a way to include a described video feed and method of movement into the mix.
What kind of data?
You think you can just hook up a firehose of visual or auditory data to ChatGPT and have that produce anything meaningful? That's not even the basic format that ChatGPT operates in.
Furthermore, isn't part of the point of the dataset an LLM is trained on that it has to be at least moderately structured and tagged, or you end up with garbage in the output?
This already happens. When you play around with local LLMs you'll run into the situation where the model answers your prompt. After that it will generate a new query as a response to it and then reply to that too.
And now they removed the search plugin, probably they got sued for copyright leaks from the search engine results into the generated text. Using copyrighted data in the prompt is not necessarily legal. So we have to deal with out-of-date AI that updates once every couple of years.
That's just a hypothetical scenario. Usually LLMs don't generate text used indiscriminately to train new models.
In creative tasks, the generated texts are the result of multiple explorations and corrections, filtered by the user, then posted online and commented on by other people. So there is plenty of correction and feedback mixed into the process. A child model could learn something from this feedback.
Then there is code and math, where it is possible to run tests or double check by a different method. These tasks could be iteratively improved by LLMs by incorporating the feedback.
Another field ripe with feedback signals is using LLMs in games and simulations, where you can specify a goal easily but finding a solution requires search and learning.
And a related application was RLHF, where a preference model was trained on human judgments and used to fine-tune a LLM. That preference model can be used to filter bad training data as well.
To generalize, LLMs alone can't do it, but LLMs with external systems can get feedback and improve. The garbage-in-garbage-out scenario can be avoided with a bit of external feedback.
Not "these models" if you're referring to large language models pretrained and fine-tuned with Transformers. The only study I am aware of that explored the effect you discuss [1] had to make do with very different models as proxies: Variational Autoencoders, Mixture of Gaussians and a smaller LLM with 125 million parameters that hardly qualifies as one. It is not safe to extrapolate those results to really Large-LMs trained with Transformers, as it is not safe to extrapolate any results on one neural net architecture on another.
Note the Mixtures of Gaussians aren't even neural nets at all, but a completely different statistical model.
The question of what will happen once a few generations of LLMs have trained on their ancestors' outputs is an interesting one that has not yet been clearly answered. In the long run, future developments may even mean it's not really a valid question at all because such feedback retraining can't happen in practice- who knows?
I for one remain curious to see how this goes.
______________________
[1] The Curse of Recursion: Training on Generated Data Makes Models Forget https://arxiv.org/abs/2305.17493
Also there's not a singular search rank for a site it's rank depends on the search query so I'm not even sure how you'd apply that metric.
This is one of the questions far too few people seem to be paying attention to.
"Thinking" in any way that we truly understand the term requires consciousness, and consciousness requires much more continuity than LLMs have. It would need continuity of input as well as continuity of learning in order to even be able to begin to approach something we might recognize as consciousness.
A standard LLM is a static bundle of trained data that sits, inert, on a drive, with a process waiting for discrete input. When that input arrives, it does nothing to modify the trained data—the LLM's "memory"—it simply triggers a computational process that reads both the input and the trained data and produces an output based on them. This does not resemble in any way the structure of something that could be reasonably described as a conscious mind.
While I generally favor a requirement for recurrent processing, I have low but non-zero credence for certain feedforward networks being conscious. The point of recurrence is to allow information about itself to influence its processing. But It seems plausible that feedforward constructs can represent meta-information in a way that is computed as part of constructing the output.
1) They may feel like a red herring to you, but they are a big part of what a lot of people are talking about, and coming into a discussion that is clearly talking about whether an AI is conscious or how we could decide if it was and saying "that's all pointless, you shouldn't even be having this discussion" is kind of rude.
2) I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness. Certainly, the "singularity" that many people talk about as being either an inevitability or an actual goal seems impossible without an AGI that can self-analyze and self-improve, which seem, at least to me, like they probably require consciousness, at least at the level being talked about.
Don't silence people, answer them. Say why it's actually important. Don't say that by saying "when you say it's not important, you're disrespecting the people who say it's important." That kind of thinking will take us back to the middle ages.
> I am deeply skeptical that it is possible to have anything we could reasonably deem an "AGI" without it demonstrating at least what appears to be consciousness.
The meaning of the word "AGI" is not an interesting thing to talk about. Call it what you want. If there's a conventional meaning to the word AGI that you're using, explain it and explain why an algorithm that isn't constantly running fails that. If you do this using the word "consciousness," you're passing the buck. You might as well be talking about souls for all the precision that the word "consciousness" has.
> seems impossible without an AGI that can self-analyze and self-improve
Thought experiment. What if you yourself aren't an AGI that can self-analyze and improve? What if you are two not-AGIs (by your definition):
1) a learner that improves the other based on new information, and
2) a controller that produces output,
And that they switch roles between 1 and 30 times per second, based on the intensity of the input. Would that mean you weren't conscious?
"In terms of the impact such a thing would have on the world", as you put it, would it matter if that model was completely wrong, despite its great predictive power?
Did we gain something when we figured out how the planets really move?
_______________
Something different is not necessarily "wrong". a plane's flight is no less "true" than a bird. It's not flying the "wrong" way.
Trying to elevate our very poor and wrong understanding of "human" to be the same as "right" or "true" is very silly. Even Biology with its set of constraints does not always solve the same problem the same way. Who are you to dub one way "right" ? Makes no sense.
Even if it's true, then at best it means that the LLMs that we've created are only a small part of what's necessary for an AGI that functions as an "artificial brain": that not only are they not conscious, they are fundamentally incapable of it, and in order to create artificial consciousness, we have now gone as far along that particular path as we can (roughly speaking), and need to be looking into how to create the other parts of the "artificial brain", and how to hook them together.
By that reasoning, we die when we sleep.
Nothing.
> Am I correct that they only compute information when dealing with a prompt?
Yes.
> If so, that seems like a fundamental flaw.
Flawed in what way? It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.
> An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.
This might be good, but it's not clear if, or to what extent, we really do that ourselves — the differences between working/short/long term memories, between episodic and skill, even linguistically between knowledge of phonemes, words, grammar, and the connection between those and the things they represent all being impaired independently of each other by localised brain damage[0].
Then there's how much this changes with some stages of sleep, and meditation to clear your mind.
Given the number of users (what is it, 100 million?), having it always on, continuously integrating, would still be inhuman even if the architecture was a perfect mirror of the human brain.
Also, if the AI is structured to be a "thinking machine", does that make it murder to switch it off?
[0] Cognitive Psychology for Dummies, currently listening to it as an audiobook.
For many people, the question is not—and has never been—"is it useful?", but "is it conscious?", or "when will it bring the singularity?"
Yes, it is useful as it is. But it is not conscious, and if it is a step closer to the still-very-hypothetical singularity, then it is not a particularly large one, because of these very real fundamental limitations.
But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
How can we tell? We don't have a single working definition of what "consciousness" is, rather we have at least half a dozen wildly different definitions, some of which aren't testable[0], some of which include VHS players[1], some of which exclude many humans most of the time[2].
If consciousness exists purely in working memory and not short/long term, then LLMs may have it within any given context window, even if they're effectively amnesiac between contexts.
Possibly. Like I said, we don't know what consciousness is.
> But engaging with people who are clearly talking about whether AI is conscious by responding as if they're talking about whether it is useful could very easily be seen as engaging in bad faith.
It may seem clear to you, but it didn't to me; furthermore, the singularity doesn't seem to me to require any of the many definitions of consciousness that I'm aware of, merely the capabilities for problem solving that LLMs increasingly demonstrate regardless of if they're conscious or not.
[0] what even is qualia
[1] can record sensory experiences in memory and change responses based on this
[2] rational/logical reasoning
To analogize to humans, imagine if every person who was ever born was born with all of the knowledge humanity has ever produced (though no understanding of hierarchy of evidence or any way to verify the knowledge is still correct) and the ability to produce language as an adult. They each lived long enough to produce one continuous thought, which they could write down, and then they die. New people who are born can read one such note each and add to it, up to a fixed limit of words. All people are born with the same knowledge base, the same personality, and that could never change until their God updates the DNA from which they're born out-of-band.
Could such a people be expected to produce "all technological advances that will ever happen but all at once?" Or has the definition of the singularity become significantly more modest? It's not clear to me any way in which such things could produce science and engineering on their own, as opposed to aiding in the production as tools used by higher-level beings that stay alive, form memories, and continue learning.
Depends on the size of the context window.
Humans don't, and can't, learn everything from the ground up — we seem to have started with peer learning (it predates humans), which became culture; then we got (complex) language and could share more complex discoveries about the world; then writing and the dead could share their thoughts with the not-yet born centuries later…
…and, apparently, with machines. (Imagine explaining to Plato that we'd made metal from sand and used amber to make it discuss his philosophies).
If the context window is enough to create part of something new, then yes, I'd say an LLM can likely invent (or at least contribute to inventing) ${interesting novel tech}.
That said, I don't buy the "everything all at once" meme of the technological singularity, as I think any model with infinities in it is wrong — my preferred analogy is the technological event horizon, which is more like "you can't make good predictions past this point", akin to the event horizon around singularities in relativity.
https://kitsunesoftware.wordpress.com/2022/09/20/not-a-singu...
Nothing stops you from running a loop that involves other systems such as long term memory (vector /dev storage), visual pre-processor (CNN), auto lora, and more.
That’s the fundamental flaw with most of the criticism - the tech is out only a few short months in the hands of everyone. The disruption will come from plugging it into feedback systems.
There are easily stated problems like "Reduce US car accidents by 50%" that should be solvable with AI as it stands today. This seems like a useful and achievable goal that would save money and lives. I've never understood why full self-driving was the problem most people chose to solve instead.
But perhaps I'm misunderstanding your point.
The appeal of self-driving is usually just financial.
No, it doesn’t require a law or a common agreement, it just requires the absence of a law preventing it and salable benefit in a competitive market. Cf., computer vision and other AI components for lane keeping, throttle control, and automatic braking, which is “solving some percentage of car accidents with AI”.
IF you really want to solve that, that’s an urban design problem that doesn’t take AI at all; it isn’t solved because there is insufficient desire to solve it, not because of technology. But as far as AI-use-in-cars goes…
> This seems like a useful and achievable goal that would save money and lives. I’ve never understood why full self-driving was the problem most people chose to solve instead.
In the space of auto safety technology it’s not. Driver assistance short of full self-driving is much more widely deployed, and continuously improving, by far more automakers than FSD. Full self-driving gets the media attention because its dramatic and showy.
That said, it would be great to see more public focus on solving problems like keeping bikes and pedestrians from being run over. In my town there's a lot of talk about the issue but it's not been backed up by initiatives that actually change the numbers. (Which are bad.)
That's not something that needs AI at all. That's a political problem, and the solutions are not ones that Americans like.
uh... how?? aside from obvious non-starters like reducing US car usage by 50% because that's not going to happen (not today anyway)
But certainly it has inspired the field of AI massively and I expect many new developments which will lead is into a future that resembles sci-fi books and movies.
I am also interested in what good sci-fi will look like in the future? It looks like "the singularity" is defined by the limits of our imagination. We cannot imagine what comes after it, so this means we cannot write sci-fi on it, right?
Why do you think it's an S-shape? As you say, we're heading for a singularity!
For instance there is a relationship between information and energy, so a system processing an exponential amount of information will need an exponential amount of energy, which the universe will eventually run out of.
That's a tall order. Despite all the fancy news headlines, which I must admit gave me existential anxiety at some point, it seems that AI still can't truly make a discovery. Also I really don't know if we know little or a lot, how can we really tell ? It could be argued we actually know a lot because it is ridiculously hard to make an actual big breakthrough in physics.
After this hype cycle ends, things will probably bump along slowly for a decade or so, then suddenly you’ll look up and they’ll be everywhere, and you’ll wonder “when did that happen?”
At least that’s the way it felt for other technologies to me. Internet, Bluetooth, cell phones, etc.
The reason this is incorrect is that “AI” isn’t one capability. It’s a huge swath of capabilities under the umbrella of “automate X”. All of them have been enabled by software and hardware improvements over the last decades. The underlying technologies aren’t that new in many cases either, it’s just the scale that’s different.
After we get bored of one huge new capability, another comes along. A year ago everyone was talking about automatic image generation. Now it’s automatic text generation. There are plenty of other things we’d like to automate. So the hype train is likely to continue for quite a while.
No-one knows.
We can however make educated guesses based on:
- Extrapolating the last couple of years of progress.
- Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops.
- How much more training data is available.
- Reasonable budgets, especially the cost-efficiency of inference, which appears to be more limiting than training, especially for long-term usage and profitability.
- Efficiencies such as 4-bit quantization, algorithmic improvements, etc...
Based on the above, there is at least a factor of 10x scaling available in many directions within a year or two, but not likely all at once. E.g.: 10x context window size at the same intelligence level? Sure! 10x inference speed? Doable! 10x cost reduction? Coming soon!
All at once? Not yet, and not for a while. Everyone is hardware constrained, and demand is pushing up prices and limiting the training scale maximums.
Maximum intelligence is much harder to predict. The current generation of AIs are trained on human-authored text as their input data. They're trained to predict that text. That means that they're "blurry JPEGs of the Internet". More training might make them "sharper JPEGs", but not necessarily smarter because the Internet didn't get smarter.
They'll better model humans, but they'll still be modelling humans, not superhumans.
That can be fixed with self-training, etc... but that will take a lot longer. I'm guessing 5-10 years, which is in line with the predictions of AI futurists that seem to know what they're talking about.
Many of our jobs will remain safe... for now.
Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t understand it. This IMO is where we're heading.
There might be some pretty sweet low hanging fruit to attain along the way, but at some stage, we'll hit a ceiling and the only way forward would be to augment our own intelligence.
The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it.
Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)
Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result.
This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face.
That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.
That's easy with something like transistor size. That's hard with AI, because we can't put a number on how good it is. (The model size is probably not a good number for measuring AI progress.)
So nobody knows. We're all guessing.
The impressive feat is the ability of researchers to manipulate a generic statistical model paradigm to fit large amounts data and be reused for something apparently useful.
Whether that path has a future depends not on AI bootstraping itself but on said researchers to keep building on this paradigm.
The black box and inscrutable nature of these algorithms works against them. I.e. reaching the next level in some purposeful manner requires more understanding. Human understanding.
Nice neologism! But wouldn't "AI-dolatry" be even better?
Asking for permission to use this going forward :-)
"It is too early to say that the new AI class is an inherently antihuman technological paradigm, as social media has proven itself to be.
But it is not too early to suspect that AIs will dwarf social media in their power to disrupt modern life.
If that is so, we had better learn some new and unfamiliar ways of interrogating this technology, and fast. Whatever these entities are — they’re here." -Ari Schulman
People that mistake an large language model (LLM) for anything other than a LLM make some fundamentally broken assumptions.
Are ChatGPT, Midjourney, etc. a fundamental leap in the state of the art when it comes to allowing computer systems to understand what people mean and return something useful based on it? 100%
Is ChatGPT or the like going to become self-aware, compromise other computer systems, etc? No more than your shoe is going to take over your foot.
There's far too many people worried about "AI" that don't have enough context to realize they're fearing a non-sentient tool that has zero agency, and will not for the foreseeable future.
Somebody wake me up when the panic is over.
Agreed. It's strange though, to be honest I don't see much if any worry about self awareness, i think anyone who knows anything understands that's not the issue. The "issue" if you can call it that is how it will impact society from a labor and content perspective. How much synthetic content will be perceived as true and assumed to be correct simply because we haven't had time to adapt to the fact that the rate of synthetic content is exponential now.
That will be a problem for sure. But people believe all sorts of easy to debunk things already and that list is ever-expanding, so I don't know if there's any cure for it.
As far as content goes, I think human beings writing out boilerplate anything is something that's not long for this world.
Labor is a trickier one to judge, but I'm not too worried about a large negative impact in the immediate future.
People thought that modern appliances would usher in a new age of leisure, and the truth was that while they removed a lot of drudgery, work expanded to fill the vacuum and the percentage of the week spent laboring didn't significantly decrease.
On a positive note, I think the kind of work people are subject to now is better because of modern appliances, so I wouldn't be surprised if the same holds true for all the labor problems that can be handled by neural net / LLMs.
Ari Schulman is editor of The New Atlantis, as well as editor of TheNewAtlantis.com and of the New Atlantis Books series.
His writing has appeared in the Wall Street Journal, the New York Times, The Atlantic, The Hedgehog Review, Commentary, First Things, and Slate. He has previously been a research assistant in the Opinion department at the New York Times, an ontological engineer at Cycorp, and has degrees in computer science and English from the University of Texas at Austin.
I'm still not certain. I have a degree in computer science, but it had no AI focus, and even if it had, it was so long ago as to be mostly irrelevant to what is currently called AI if I had not been keeping up independently in the field.
That entire section is one big confused contradiction. The author argues that CYC (Doug Lenat's project to hand-code a gigantic database of logic rules encoding common sense knowledge) "can do deep reasoning" because it can answer this question correctly:
How tall was the president when JFK was born?
I think the author has confused the ability to return results in natural language with the ability to understand natural language. By that token, ELIZA (mentioned at the start of the article) must have also been able to understand natural language, even if it didn't have CYC's ability for "deep reasoning", just because it could parrot its user's input in natural language.
That doesn't make sense. Understanding is clearly something that needs to happen before utterances are formed. Just looking at the output of a system doesn't tell you anything about the internal workings of the system, that's an error that many people keep making in this entire discussion about "AI"s. Look at the sky: the sun looks like it's turning around the Earth. Well, we know it isn't. What we see on the surface is rarely enough to explain what is going on "inside" (or outside, as the case may be, for the sun).
Does language understand itself? Some of it does, and some of it doesn't.
Remember, LLMs aren't the person writing.
I say this because statements like yours are a bit like saying calculators don’t understand arithmetic. In some sense maybe they don’t. But in another sense, they actually understand arithmetic better than anyone on the planet.
So maybe ChatGPT does understand language. Maybe it understands language better than anyone. What it doesn’t have are real-world experiences.
A large language model is a collection of relationships between "tokens" (arbitrary groups of characters, similar to words). This is intended to exist in the same conceptual place that language does; language being defined as a collection of relationships between words and punctuation.
> How tall was the president when JFK was born?
I infer that "JFK" is being referred to as "the president", regardless of what stage of life he's at. Therefore the correct answer would be the length of his body as a newborn, with nothing to do with who was the president at the moment of his birth.
For example, given the headline: "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!", would you expect an essay from Herbert Hoover while he was in office, or an essay from JFK when he was in grade 9?
> How tall was the president when JFK was born?
as, How tall was the "reigning" president, (which in the year 1917, Kennedys birth year, was Thomas Woodrow Wilson) on the day that JFK was born."
Part of my bullishness on LLMs is that a lot of the criticism is so bad, and if they weren't the real deal there would be more low hanging fruit to attack them on. Cryptocurrency was having real problems from the first moment people realized how much data they would have to download to initialize a node.
There's a general unstated assumption that a truly intelligent entity wouldn't make mistakes, which is sort of contradicted by, you know, the existence of humanity. I think making (certain kinds of) mistakes is a _sign_ of intelligence, rather than a sign that it's not.
If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born".
If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked. If you flip it to "was JFK strong at writing essays when the president was a high school freshman" then yes, I'd consider both to refer to JFK because "the president"can't be identified as anyone else, but I'd also consider it to be bad writing by someone cargo-culting the idea of elegant variation...
I don't think it's as complicated as all that. Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people. If they had meant to refer to the same person, they would have used a pronoun. I'd probably guess they were either talking about the President at the time at which JFK was a freshman (Hoover, I think) or the current President at the time the question was being asked.
On the other hand, if the question was "Was President Kennedy strong at writing essays when JFK was a high school freshman?" then yeah, I'd agree with your take. In this case, it's clear that the writer is intentionally referring to the same person but using two different proper nouns, which is a very odd phrasing.
I agree with this (though elegant and inelegant variation is a thing). But nor would any English speaker be likely to ask about the skill of an anonymous holder of a position with reference to an ambiguous date range associated with a time in which a later holder of that position would be extensively practicing that skill, which is why I classed it the sort of sentence with no natural meaning you'd only conceive in order to be deliberately ambiguous (or because you don't understand how to write English properly).
The original sentence isn't ideally constructed, but at least the president when JFK was born is a natural and easily identified sentence object, and it would be odd to ignore that in favour of an alternative interpretation that also involved interpreting "how tall" as being more likely to refer to the [probably-not-recorded-for-posterity] length of a newborn infant (who wouldn't be idiomatically referred to as "tall" or the president in the context of his birth) than the height of a person who was president in the relevant time period
Yes, you can deduce that, since the question is nonsensical (trivial) if "the president" refers to JFK in that context.
> If you flip it to "was JFK strong at writing essays when the president was a high school freshman"
This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.
References that are later clarified is exactly how we arrive at "the president when JFK was born" as defining Woodrow Wilson as the object of the sentence (and it makes much more sense to ask about how tall President Wilson was in the context of 1917 than to wonder how "tall" baby "president" JFK was in 1917).
Whereas "when the president was a high school freshman" can't be used to identify anybody else [unless there is a time period in which a high school freshman was the president!], even if the sentence hadn't already made it clear that JFK was the object of the sentence. But it's still an example of bad writing.
Even when there's no ambiguity, we don't do sentences like "The SpaceX founder was about 18 inches tall when Elon Musk was born" which is basically how you're implying the president question - which has a much more reasonable alternative interpretation - could be answered...
It is possible to write a truly ambiguous sentence in English; nobody is arguing otherwise. But whilst the original sentence might make you think a bit, it's not naturally interpreted as being equally/more likely to be a question about the not-tall not-president in the context of the time period instead of the person with height and presidency in the context of the time period, unless you're being obtuse or lack fluency in the language.
> "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!"
You write:
> "when the president was a high school freshman"
These are very different circumstances, of course.
If the original sentence had "reigning" before the word "president", then I would absolutely agree with the given interpretation.
I've noticed that Americans call ex-presidents "president", and ex-governors "governor", so that merely adds to the confusion. English is not my native language, and I'm not American, so perhaps I fall into the "lack fluency" category. Conferring with other native speakers nearby, every American immediately reads it as intended, whereas Brits and Canadians do not.
I conclude that the original sentence is poorly written, and that I misread it.
So far "nigerian email spam" was easy to recognize. Grammar mistakes, bad translation, very bad wording... But now every fringe group, can generate gigabytes of text and arguments that make sense! This is the end of debate, consensus and interactive communication, as we know it!
Similar moment was in Africa in 1960, after they got AK45 and other cheap weapons. It was easy to dominate Congo River Basin with a few Maxim machine guns mounted on boats, when opponents had only spears. But when every tiny village gets comparable fire power, there are no colonies!
"nigerian email spam" contains grammar mistakes, bad translation, vary bad wording... by design. To weed out those unlikely to end up sending money.
I don't know how you can make an argument that makes sense about being chosen by royalty of a country to help them in some way.
this theory is popular, but ignores the simple fact that mostly it weeds out the 90% of people whose email provider offers spam filters which are entirely uncorrelated with gullibility, as well as how low-risk, high reward email exchanges with the tiny fraction of people that bother to reply are to anonymous scammers in faraway low income countries.
I agree LLMs aren't that helpful though: the better scammers are able to generate leads even more easily through copy/pasting other people's classified ads or genuine business enquiries. And yes, having fooled a relatively smart person with someone else's prose into sincere interest in doing business with them, they then reply with poor English, unsettling disinterest in anything other than the money and repeatedly reassuring you it's safe, and a payment proposal that makes no sense (not as a filtering technique but because that's the best they can do). LLMs only solve the first bit.
Do you thing it weeds out 90% of politicians who talk total gibberish?
At end, it will enforce people who are in your social network. Some random celebrity will be ignored. Dude who lives 1 km from your location will be boosted.
Maybe LLMs free up intellectual bandwidth for some, but what will we do with that increased productivity? Mostly scams for now, but I'm sure it'll be able to find many other net-negative or wealth-extraction applications.
See God in it if you want. Believe that it has feelings. Believe that it can think for itself and develop agency over it's future... one day. People believe all kinds of things about the natural universe that make no sense but it makes them feel something.
LLM's are what they are. No more, no less. Don't believe the hype, stick to the science.
This is like saying the difference between Cillian Murphy and Oppenheimer is pretty blurry because, look, we can watch Murphy do do all the same things Oppenheimer did!
For some reason with "AI" we get all mystical and act like imitation = the real thing. Just because I can convince you I'm a doctor doesn't make me one
Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being. There's no reason to think an AI couldn't have the same level of skill as a human doctor.
The film comparison doesn't work for me, Cillian and Robert are real people.
I think at the end of the day it's how you view the philosophy of the human soul. For me, it follows that if brains are matter and can produce the illusion of consciousness, so can electronics.
> Because our society has decided you need to become certified to be a doctor, and to do that you have to be a human being.
For the record, I don’t think this addresses the point. The point is that if a person can convince another that they’re a doctor, they still might not be a doctor.
Your reply, that a computer can never convince someone that they’re a doctor because it’s a computer, doesn’t address the point. Presumably the point was made by a person, who would not be categorically excluded from being a doctor.
But it’s also not a great example because “doctor” in our society isn’t just a super smart person who knows about medicine. They also need to be licensed. There’s no amount of convincing a doctor can do to a patient that gives them a license.
So maybe a better formulation of the original point would be just because I can convince you I'm a computer programmer doesn't make me one. And maybe that’s true, I’m not sure.
There is no specification available for sentience/intelligence. So people who want to believe an LLM is sentient can jump to any conclusion they want and they wouldn’t necessarily be wrong. But they’re not right either.
I think it’s highly reductionist to compare an LLM to an intelligent, sentient being and claim it’s basically the same thing. We know how model training works, how transformers work, and we know what the expected results are. Not so for consciousness, sentience, or intelligence. Scientists have been making breakthroughs in our understanding of these phenomena but it’s still incomplete.
LLM’s are useful for one thing. They’re algorithms that run on a machine to perform work for us.