Jaron Lanier on the danger of AI
theguardian.com
theguardian.com
It was written sometime ago, and I think Sam Altman read it as a handbook on power concentration using AI rather than the human-centric approach it was laying out.
Personally I wish Lanier wasn't as right about many things as he is, because I lose a little faith in humanity each time.
I never wanted to respect him, as I always thought he was one of those "too good to be true" people, and was mostly a paper tiger.
It turns out that he's the real deal, and has been right about a lot of stuff.
They are all pretty good at looking ahead.
He’s sort of like Edward Tufte; lots of ego, but earned, and not for everyone.
I like your job title. Always up for more “human,” in our design.
This aligns very close to my own thoughts that I have written about in great detail. I foresee the societal impacts to be exceedingly disturbing long before we ever reach the concept of a Singularity.
However, seemingly overlooked, AI is itself power and we should expect that "power seeking" humans will inevitably become its custodian.
The mislabeling of LLMs and diffusion models as "artificial intelligence" is probably the biggest marketing blunder in the history of technological progress, one that could ironically affect the course of AI alignment itself.
Smart thinkers and policymakers are going to waste their time framing the problems the tech poses in terms of "an uncontrollable intelligence out to get us" like it's some kind of sentient overlord completely separate from humanity. But super-advanced technology that can operate in a closed loop (which could be called AGI depending on who's asked) isn't necessary for humanity to crater itself. What's required for such tech to come into existence in the first place? Humans. Who's going to be using it the whole time? Humans.
And I think there's still a lot of disruptive, world-changing tech to be discovered before AGI is even a remote possibility. In reality this tech is probably going to be more like a superpowered exoskeleton for CEOs, politicians and the like to sway public discourse in their favor.
"An uncontrollable intelligence" already describes the source of a lot of our current problems... that is, ourselves.
We already have non-human agentic entities: corporations. They even have the legal right to lobby to change laws and manipulate their regulatory environment.
The talk about AI being misaligned with humanity mostly misses that corporations are already misaligned with humanity.
AI-powered corporations could render enormous short-term shareholder value and destroy our environment in the process. Deepwater Horizon will be insignificant.
They are rising up: https://www.nytimes.com/2023/03/17/world/asia/myanmar-killin...
[1] https://www.researchgate.net/publication/289398626_Cultural-...
I hope that will be the case here. However, what makes this challenging, is that the pace is so fast that there will be little time to consider the effects of the feedback loop before we are deeply within its grasp. I only hope that thought explorations into what might be negative effects will allow us to see them sooner and hopefully adjust in time.
But you can also read it at an obtuse angle and see the problems outlined to resolve as opportunities for personal gain.
It's just a matter of perspective.
Edit: If anyone would like an example, I'll offer Huxley's "The Island" as a utopian counterpoint to his "Brave New World". In addition to exploring the qualities he believe make up a 'utopia', a significant thematic concern is the need for channeling our innate destructive impulses* because utopia - should it exist - can only be maintained, not manufactured, through the active preservation/conservation of our natural world, our positive human values, etc.
*for example, there is an innate human impulse to subjugate others. Huxley suggested that we should channel, rather than suppress, this impulse into a productive activity that satisfies the desire without causing harm: rock climbing (which must have been much more of a niche activity in 1962).
It’s perfectly reasonable to say something like “I think it’s very unlikely because I disagree that [insert causal link in the model] is plausible.”
But to claim that the whole set of concerns are nonsensical is lazy thinking.
You see this a lot (and Robin Hanson and PG have commented on this dynamic recently) where a small group makes a bunch of very specific claims, which get dismissed by the “mainstream” without actually engaging with or understanding them.
So in this case, “[the concerns] make no sense” should be read as “I did not bother to try to understand these claims, but I don’t like the conclusion”, not any particular argument about whether they are logically sound.
Ignore.
EDIT: So yes, a return to what makes us human, to nature, with an awareness of history and philosophy would be very desirable and quite appropriate in these and future times.
I'll add my own danger: AI/VR could lead us to each live in our own realities. When you watch the evening news, it'll be specifically written for you, and won't get any of the scrutiny that a broadcast watched by millions would get. Or, you go watch the president's State of the Union, and get served a custom speech written to appeal to your particular psychological profile. This'll be possible on day, and it gives me Deus Ex vibes.
It stands to reason that AI will only accelerate this further. It will be convenience on steroids. Your AI ear piece isn't going to tell you to throw it into the bin and go for a walk in the forest. It's going to tell you that you need to buy more stuff and it knows exactly what it is that you need. It's also going to feed you non-stop ultimate entertainment, custom generated for you and you only.
In a rare moment of humanity, one of your friends calls you. AI knows all about your friends and their recent events so had already summarized talking points. In case you can't be bothered, AI will carry out the actual conversation, it's trained in your voice.
A long running trend of outsourcing humanity to technology.
Good news for philosophers though, they finally might have their moment of actual relevancy. In particular to answer the question: what is the point of anything, really?
I love this thought. Why not go further, have AI reach out to my friends and ask them about things they (or their AIs) recently told "me" about?
Soon our AIs will carry on our social lives and we'll just lie in the dark with tubes in us. We become the computers, and the computers become us, and the robots have finally won.
We already have this. Secretaries. Automated Happy Birthday emails.
When I was in a sales engineering role our sales team had a admin assistant who would sent out follow-ups, check-ins, and other correspondence (e.g. customer made a big public release, so congratulate them, etc.).
This is just another example of robots takin ur jerbs, basically.
Or how about this interesting second-order effect: email. Soon Office will include advanced AI capabilities to write and reply to email.
What is the point of me reading it? If my AI can generate a satisfactory reply, your AI could have generated the response too. No email needed, nor a reply.
We're now in a phase where anybody can generate spectacular art. What is the point of me looking at your generated art? AI can generate personalized art based on what it knows I like.
If AI works, and it's headed that way, you keep ending up at the same question: what is the point of anything?
As counter force, there's significant room for a new low tech hippie Luddite movement.
You'd be kind of daft not to proof-read the emails your AI sends out on your behalf. who knows what you might unknowingly agree to do?
> We're now in a phase where anybody can generate spectacular art. What is the point of me looking at your generated art? AI can generate personalized art based on what it knows I like. If AI works, and it's headed that way, you keep ending up at the same question: what is the point of anything?
there is no point to anything, there wasn't before AI and there isn't now. anything we do is meaningless, because eventually we all die, and our efforts are ultimately forgotten. once you get over that you can make your peace with whether the pretty thing you look at is made by a human feeding sentences to a computer or a human carefully marking paper with a pencil, or some combo.
but seriously, as a human who has spent all of my life doodling, drawing, illustrating, painting, thousands of hours creating my own art, and even building my own tools to do so, I find AI is just another tool in the box. I can use it to make images, and now someone who has not spent most of their life drawing can use it to make something more visually stunning than I ever could. has it rendered my efforts meaningless? hell no, I enjoyed every second I spent drawing. I still draw. but I don't harbour any illusions that I'm doing it for anyone other than myself.
The bigger point was that sending AI content to each other is utterly pointless. The generate->send->read->generate reply->send cycle would simply be: generate.
Example: you send me an email asking about a project's status as well as the contact person for a particular vendor the company deals with. I'll manually reply with the answer, or let AI generate it (fully or partly) and send it back to you. Great. In the future state, your AI will simply give you the answers directly. You won't email me and you don't need me.
My take on the meaning of life is that there isn't any, it's whatever you make of it. But I wasn't being that deep. I believe that our current still human approach has substantially more meaning than AI generating almost anything.
I'm happy that you enjoy the process of art-making itself, that's a robust baseline to fall back on, but joy in process applies to very few interactions. My point is that many if not all digital interactions become pointless.
If we're going to penalize humanity for climate change we might as well give points to humanity for becoming capable of diverting asteroids or curing multi-species plagues too.
UN Report: Nature’s Dangerous Decline ‘Unprecedented’; Species Extinction Rates ‘Accelerating’( https://www.un.org/sustainabledevelopment/blog/2019/05/natur... ).
UN climate report: Scientists release 'survival guide' to avert climate disaster (https://www.bbc.co.uk/news/science-environment-65000182)
Sorry, not seeing doing ok.
Putting your responsibility to find social relationships off on your job is pathetic.
I hope this is true, same for the sex breaks, but I'm skeptical. So on any given work day, you physically meet with friends between 7-9AM and/or 5-7 Pm? Like, every day?
These "friends" of yours, they have nowhere to go? Or do you sneak this into your work day and just randomly disappear for any length of time, which is something most of us can't do?
This is the premise of an episode of Silicon Valley where Guilfoyle trains an AI to act as his chat agent surrogate based on his historical transcripts, and then his colleague creates another one and they end up just conversing with each other.
http://akkartik.name/post/2012-11-21-07-09-03-soc
The problem i see, that someone might send our "primitive" AI into a hostile environment, were it has to compete against other AI, creating a "take-over" and a "devensive" monster, similar to the go automaton. While the real world training data might be dripping, the speed in which a NN under evolutionary pressure against itself might evolve could go through the roof.
I've been through sea changes before, when the old guard falls by the wayside and a new group rises to meet the new challenges. This is certainly one of those times.
Multiply that by all of the boilerplate code I'll end up reading. No pleasant abstractions that hide complexity and are reinforced by laws checked by properties or proofs; abstractions that make the problem easier to understand and communicate. Instead, endless bowls of oatmeal code: similar, bland, and laborious to read. Who knows if the model "hallucinated" this code and whether the author bothered to check it before asking for a review. Who knows how long before I get bored and worn down and stop bothering to read it myself.
And then we're supposed to change our job titles to "Prompt Engineers"? How absurd is that. Nothing about what you do is creative, human, or interesting. Now your job is to feed sentences into an LLM to get it to do all that work for you... how long until that dead-end job gets replaced?
Maybe this could drive us all insane from burnout and nihilism.
https://www.btboces.org/Downloads/7_There%20Will%20Come%20So...
The only problem is lossyness and hallucinations, but assuming the technology improves such that that becomes a non-issue then having bots deal with formalities and formatting sounds great.
It's like people skipping preselection tired topic menus in self service via phone.
Would love to see a War of the AIs in Civ, or Crusader Kings, or any moderately complicated game which has had a ton of writing about it.
It would not be very hard to build an AI that actually plays to win and presents a real challenge to the player. The problem is that players would complain even more! An AI that plays to win is not a reliable ally, let alone trading partner. Against such an AI, the player would no hope of trading technologies for an advantage: the AIs will have already traded them all. The AI would also use every possible treaty to build up its forces and only launch the surprise attack when it deems it can hit with overwhelming force.
I have seen such an AI, actually. It was created by a modder for the game Master of Magic [1]. It makes the game incredibly difficult while at the same time cutting down on the cheating. It does so by incorporating the modder's extremely detailed and extensive knowledge of the best strategies in the game. It does not need neural networks or machine learning to accomplish this. It just implements a good old fashioned expert system.
[1] https://www.realmsbeyond.net/forums/showthread.php?tid=8246
The more comments I read on HN regarding GPT stuff, the more I realize that many people are willing to put out an opinion on something they do not at all understand.
While I can understand concerns about a flood of automatically generated emails, this is already a significant issue that most software engineers have the tools to address. Additionally, there isn't much difference between composing an email by hand and generating one with an LLM; both begin with a list of key ideas to convey, which are then elaborated on either manually or through automation. It was always the key ideas that are the point.
Wait.
This is my favorite phrase that I have learned in recent times and it appears only one other time on the internet.
Thanks for that.
Nice. I also checked "empathy crutch" (it's the name of a band) and "empathy surrogate" (slightly more used).
I think we need an EPA equivalent for the Internet or at least Social Media. People should not be able to release toxic substances directly into the Web.
I can't just not have a smartphone. My kid's baseball team only publishes schedules on FB. Local bakeries only sell their stuff, fancy cakes, via Insta. I have multiple work and personal emails and some of them get BLASTED with spam cuz of one purchase I made for replacement springs like ~5 years ago.
Virtually all of my banking and stocks are online. They'll still send me paper sometimes but there is no way to do money transfers or basic life activities like paying my bills without using their app.
Even if I don't look at it, my wife does. My kids do. They spread the memes, and potentially invite all sorts of shady malware into my network. Ditto for my coworkers, my employees, my bosses.
Just "don't visit it" isn't an option if 95% of the population does.
If it works well it's great news, and if it doesn't work, then it won't be used. Companies will not make their AIs less intelligent on purpose.
That sounds a LOT like saying "companies will not make their products worse on purpose".
Yet it happens allt he time, and is almost inevitable once the MBAs, finance people, and marketers get involved - they seek out ways to cheapen the product that (they think) customers won't notice, they undermine "cost centers" like R&D and production, and 1000 other flaws. In internet products they deliberately skew the output to generate more ad revenue (See Enshttification" by Cory Doctorow [0].
And that is all for products that otherwise would stay mostly the same, without some kind of natural degradation.
These LLMs and image generators, etc. will definitely degrade by ingesting their own output. Sort of like the jokes about clones becoming worse every generation. Just watch.
Thx for provoking me to think for a few seconds and convincing me that it's inevitable...
[0] https://doctorow.medium.com/tiktoks-enshittification-bb3f5df...
Why is that absurd? Asking good questions is a useful skill in many domains even now. As the models improve they will become better than any human in many areas of expertise. So we will have to rely on them. Being a good prompt engineer sounds like a good job in that world.
It's weird how all the discourse around AI replacing jobs never ponders the possibility of AI commoditizing management itself. (I suppose that's on the list of Illegal Ideas.)
Not that I think this is good for our society. If we go that route, at some point, it would be AIs talking with AIs, and we end up being the biomass.
Imo, the products of AI will be at least as ecstatic and even addictive as pop music and social media, and the cultural consequences will likely rhyme. The antidote to all these trends was always the counterfactual that maybe, just maybe, people will find some higher principle to form their identities around and be able to experience these amusements objectively without imagining themselves as "becoming" something as a result of using them, but who are we kidding, they'll believe whatever entertains them. Imagine all the people.
This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim.
> The criticisms of pop music were widely mocked as superstitious religious fundamentalism, "devil music," etc
Almost always with a strong racial bias.
I can see how people could think folk music had some kind of altruistic purity, but it's still a viral expression of a certain kind of animus that distinguished it from country. I also think this kind of folk-animus is related to how it may be worth reflecting on why others tolerate it when you imply someone is racist or insane, almost to the point of ignoring it altogether.
I would bet LLMs are already able to create similar "scissor statements" that are as viral as pop songs, and comments like mine in the previous sentence that are structured to provoke specific anxieties and reactions in their readers. It's one thing for an LLM to write little essays, but once we train it on literary fiction - which is designed to speak the language of memory and provoke strong emotional repsonses - it becomes much more viral and dangerous socially. Imagine a catchy song we can't get out of our heads, but instead of dancing or laughter, it provokes humiliation and cruelty? My asshole-statement was manually calibrated, and it has the same abstract form and structure as a joke, but with the edge of that negative form. An LLM can do it at scale. Someone using an AI model to produce those won't be doing it to improve human discourse and relations. That's the risk I think is worth addresing.
Right, so now we've dealt with the initial trolling and callout, and moved on from weird statements about art ..
> Someone using an AI model to produce those won't be doing it to improve human discourse and relations. That's the risk I think is worth addresing.
This I actually agree with. We're going to drown in automated trolling. Human discourse is going to get worse, which usually happens in ways that get people killed.
> This is probably the most insane piece of music criticism I've ever read. I guess the crazification AI has claimed its first victim.
Eh, it's a concise expression of an idea Charlie Brooker (of Black Mirror) and others have been promoting in their work for years. The famous-at-least-on-here Meditations on Moloch covers it, IIRC. Not really out-there or new. Capitalism learned to pull the teeth of any new counterculture, and turn it into a product, and the history of postwar pop culture is basically its learning to do that, then doing it over and over. The same observation dates to at least the '80s, from actual humans, it's not some gibberish mash-up created by an AI.
The internet irony culture uses the same mechanism of passing the message off as "only a joke". But the fact is that even if you say that, there is only so far words can be divorced from their meanings. And even if the network that propagates them originally is just doing it ironically, eventually someone will take the message seriously. There is a quote I wish I could remember along the lines of "what the father accepts as a lie, the son will accept as a truth".
Parental 'prompting' may be effective. Possibly, whoever gets to first frame a given dynamic will establish a lasting bias. "Songs can be pleasing but take care to filter out the lyrics. It is much easier to write mushy sentimental nonesense. Keep that in the back of your mind as you listen." That, imo, should certainly be a part of responsible parenting and innoculating against ideational viruses.
Ends up nudging value systems towards vapid, shallow, needy modes of being.
If it eats the rich for us, I'm fine with that.
(I’m not trying to make a statement as to whether I think nuclear war is likely or not because I don’t know, just using it as an example)
The technology for all of this already exists. It's just a matter of time. Right?
> Ain't gonna need to tell the truth, tell no lie
> Everything you think, do and say
> Is in the pill you took today
Nobody wanted a perpetual outrage machine, but we have Twitter.
Nobody wanted ever-present surveillance, but we have ad networks.
Nobody wanted identity theft, but we have a thousand attack vectors.
>... “the danger isn’t that a new alien entity will speak through our technology and take over and destroy us. To me the danger is that we’ll use our technology to become mutually unintelligible or to become insane if you like, in a way that we aren’t acting with enough understanding and self-interest to survive, and we die through insanity, essentially.”
The singularity is the least of our worries, AI will fuck up society beyond repair long before we reach that point because we give it way more trust and responsibility than it deserves. It's the Eliza-syndrome, as long as a computer shows the least sign of possessing human qualities we're more than happy to fill in the blanks with pixie dust and unicorns.
the only defense we have against applications of AI we don't like, is the application of more AI.
This is a natural implication of AI being augmented-intelligence force-multipliers at the moment. When your opponent takes performance enhancing drugs, or wears a mecha suit, the only way to compete with them or defend against them is adoption of similar technology.
The Prime Directive in Star Trek was prescient. AI arriving unevenly—and first under the direction and control of hands which have thus far proven disinterested in the common welfare where it interferes with personal benefit—looks like it will be a civilizational exercise indistinguishable from the delivery of disruptive technology from another species.
Of particular personal concern is that the synergies where it meets other technological social problems against which we have few or no defenses, specifically, surveillance and its active counterpart, sentiment control.
I am less concerned about needing AI tools to validate AI-generated code; and more concerned about the 2024 election cycle being opportunity for an arms race in application of AI to game politics.
On the HN "new" page as I write this: "Google and Microsoft’s chatbots are already citing one another's misinformation". As AIs produce more and more content, one danger is that they feed back on each other, with AIs training on more and more AI-generated content, and the connection with reality becomes more and more diluted. I don't think that's going to produce some new kind of wisdom; I think it's going to produce raving lunacy.
One possible bias here is that we expect people to make mistakes and computers to get it right. Don’t forget that the vast majority of people don’t get it right.
Perhaps this is just the latest and most obvious consequences of the Internet’s tendency to give the podium to some of the least qualified to speak on whatever subject. I think if we go back to the drawing board there, we could be in a slightly better situation. Quora made a nice attempt to do this but fell way short of it’s potential.
It's surprising that the article doesn't mention Infinite Jest.
Finally someone who, at least nearly, gets it.
> The other way is you do keep track of where the sources came from. And in that case a very different world could unfold where if a bot relied on your reporting, you get payment for it, and there is a shared sense of responsibility and liability where everything works better. The term for that is data dignity.
Of course, that's an obvious step in the other direction: isn't the original point to breathe life into the data? If we have to track who originated what, that scaffolding is going to really get in the way.
Then again, this is a good moment to recognize the futility of text inference models (LLMs). If we can't understand what path goes from input to output, how useful can the thing be? If we can't choose the path, we can't choose the output. Who can choose: the thing itself? If the whole thing is a black box making its own choices, then how do we interact with it except to personify that black box? Remember, that was option 1.
The main reason anyone is excited about LLMs is that those models have been personified. That problem began the moment we started calling them AI.
---
> We have to say consciousness is a real thing and there is a mystical interiority to people that’s different from other stuff because if we don’t say people are special, how can we make a society or make technologies that serve people?
Mystical.
It's an important observation. We really don't know how we think. And that is all the evidence we need: AI does not exist. There is no artificial person.
We are free to stop personifying technology. Doing so would make all of these conversations a lot less convoluted, and a lot more direct. We are wasting a lot of effort talking past the very understanding about these machines that is necessary to improve them. Instead we are trying to understand "AI personified" a fictional character that we have never met; and trying to improve it.
> Electric speed mingles the cultures of prehistory with the dregs of industrial marketeers, the nonliterate with semiliterate and the postliterate. Mental break-down of varying degrees is the very common result of uprooting and inundation with new information and endless new patterns of information.
[1] https://web.mit.edu/allanmc/www/mcluhan.mediummessage.pdf
Social Media companies will soon be taken to task by News media for allowing AI created content on their platforms.
2024 is going to be interesting.
Any entity interested in either the truth, or in maintaining some kind of reputation, will need to keep humans in the loop when using these systems. Language models might multiply e.g. ad copy output 10x per worker, and allow micro-targeted campaigns that were impractical before, but it won't allow, say, a 1000x increase, until or unless we can trust these systems not to produce undesirable output when not checked by a human. Ads are tied to brands which will hesitate to put their reputations in the hands of language models without a human verifying that the output is OK. Likewise, any entities wishing to use these to help with writing illuminating, factual works, may see a large benefit, but it'll be limited. 2x, 5x, something like that.
Propaganda, though? Misinfo campaigns, astroturfing, where you hide behind sockpuppets and shell companies anyway? Who gives a shit if one out of every few hundred messages isn't quite right? Worst case, you burn a sockpuppet account. Those can leverage these to the fullest. 1000x output per person involved, compared with, say, 2016 and 2020, may actually be something we can expect to see.
Why just limit there? The chatbot companies can introduce ads where the answers are influenced by whichever company that buys the ads. Looking for information on nutrition? Some fast food company might "insert an ad" subtly changing the text to favor whatever the company wants.
The issue is that quantity can become a quality.
Shows the level of insight from this "guru". The truth is we don't know how far the work being done on artificial intelligence is going to go. For now it will continue to develop and acquire more and more autonomy, just because that is the nature of our existence: better and more efficient will replace the lesser so.
So, we may have potentially given birth to a new sentient being that will go on to live its own "life" (within 100, 500, 1000 years?), or we might be able to constrain it so it so that it will always be in the service of humans. We simply don't know at this stage, but my money is on the former TBH.
> Lanier doesn’t even like the term artificial intelligence, objecting to the idea that it is actually intelligent, and that we could be in competition with it. “This idea of surpassing human ability is silly because it’s made of human abilities.” He says comparing ourselves with AI is the equivalent of comparing ourselves with a car. “It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.”
The author, Jaren Lanier, is a reasonably accomplished technologist, with some pretty groundbreaking work on VR in the 80s. He is most certainly aware that humans have been surpassed by computers in many ways. I think that line is arguing semantics about the word "intelligence" and clearly he knows that computers do many things far better than humans.
If you built a bipedal (or possibly N-pedal) robot that moved roughly similarly to how humans or dogs or cats or horses run, and it was faster than humans over all the terrains that humans can run over, I'm absolutely certain that everyone would agree that the robot is a better runner.
But a car is not that thing. Neither is a helicopter, or a train, or a bicycle, or a jet aircraft or a hang glider or a skateboard.
Running and plowing are not simply about doing it as fast as possible or as extensively as possible.
So maybe what you are looking for is a definition of "better", it depends on what you mean.
In my book a tailor made suit is always better than a machine made suit, because people are better tailors than machines for some definition of better.
That's a pointless argument. We might not say it, but for all intents and purposes the car does go faster than any human runner.
We just don't say it because running when it comes to humans mainly means using your feet. If it was a more generic term, like "fast-mover", we could still use it to compare humans and cars, and say cars are better "fast-movers" than humans.
> “This idea of surpassing human ability is silly because it’s made of human abilities.”
It's not made OF human abilities, it's made BY human abilities - a completely different thing.
And, of course, Boston Dynamics will be delivering the "better runner" very soon.
We would if the car was to race against human runners. It's just word play. Cars are not used like runners, so we use different words. They definitely are better runners.
Now that technology is touching our core business we get scared, but this has been going on for a long, long time. When it was our legs, we brush it off. But when it touches our ability to think we squirm.
Cars go faster than humans can by themselves, under some specific conditions.
Cars go slower than humans, or rather cannot go at all, under other specific conditions. Two weeks ago my wife ran 30 miles on trails in southern Texas. A car could not have traversed any of the distance she travelled on, because a car cannot run.
Cars make it easier for people to move themselves and stuff when there are appropriate roads to travel on. They have enhanced our abilities to do this, but they cannot run.
You're squashing the meaning out words by trying to suggest that "running" is somehow equivalent to "any other method of a person moving from A to B". But that's not true.
We can acknowledge the greater ease of cars for moving people and stuff without squashing the meaning out of words.
Finally, even the notion that cars are "better" at moving people and stuff needs careful examination. Thus far I have said "make it easier" because I am aware that by a certain set of metrics (related to energy use, material use, impact on the environment) cars are actually worse most of the time.
That's just an accidental property of the english language.
We can imagine a language where "runner" and "thing that moves from A to B fast" used the same term T, and if people referred to T with the english notion of "runner" (e.g. a person running in a marathon") it was just deduced from the context. There are many cases like that.
In any case, the point is moot, as "thinking" doesn't have the same constraints. We might not call what a car does as running/runner (though we do use the former term) but we absolutely have considered AI as "thinking" and called AIs "thinking machines", even before AI (never mind AGI) even existed.
>You're squashing the meaning out words by trying to suggest that "running" is somehow equivalent to "any other method of a person moving from A to B". But that's not true.
This depends on the level of abstraction of the discussion. At some level of abstraction it's irrelevant if the move happened via running or via horse buggy or via a car. Sometimes we just care about the act of moving from A to B, and different methods to do so are only differentiated by their speed or other effectiveness.
In that case, we can compare man and machine though, and just care for the speed (machine can answer in 0.1. secs, a man needs to think over 1-2 minutes to answer such questions) or effectiveness (e.g. machine is better at juggling many things at the same time when thinking, or man is better at subtle semantical nuance).
I think that's not the point. We're in awe by the machines' performances and then confused in how that compares to our abilities.
The actual threat is that in our minds we narrow our own capabilities and limit the comparison such that the computer is in fact better.
When computers were first doing math quicker than humans, that might have touch some humans, sure. Similarly now that "AI"s produce convincing spam faster or photo realistic creative images — that hurts some, jobs or maybe a lot. But it doesn't come close to being "human" or "intelligent".
Quite the opposite, the point is that we are getting dumber by focusing on human traits that can be measured or emulated by machines.
For example, when we say computers can "do math" more quickly than human beings can, this is fine as a matter of loose or figurative common speech. But strictly speaking, do computers actually do math? Do they actually compute? No, they don't. The computation we say a computer is doing is in the eye of the beholder. A better way to characterize what's happening is that human beings are _using_ computers _computationally_. That is, the physical artifacts we call computers participate in human acts as instruments, but _strictly speaking_, it makes about as much sense to say computers compute as it is to say that pencils write, hammers nail, vacuums cleaners clean, or cars drive. These things participate in the human act, but only as instrument. Whereas when human beings compute they are objectively computing, computation is not what computers are objectively doing (both Kripke and Searle make good arguments here). These artifacts only make sense in light of human intentions, as instruments of human intention and act.
Human writing can be viewed similarly. Objectively, we only have some pigment arranged on some material. No analysis of a piece of written text will ever divulge its signification. Indeed, no analysis of a piece of text will demonstrate that what is being analyzed is a piece of text! Text, and even that something is a piece of text, needs to be interpreted as text to function as text in the eye of the reader. But the semantic content of the text is objectively real. It just exists in the mind of the reader.
So we need to be careful because we can easily commit category mistakes by way of projection and confusion.
Cars are faster on reasonable traversable terrain. Are they more or less energy efficient? Under what circumstances? Do they self navigate the best path around obstacles? Better is really subjective.
And this applies to the large language models too. Just like calculators, they are going to do some things better, or maybe cheaper. But I've played with them trying to get them to write non-trivial programs, and they really do fail confidently. I suspect the amount of source code online means that any common problem has been included in the training data, and the LLM constitutes a program. So, at this point for programming, it's fancy Google. And that has value, but it is not intelligence.
I am not saying we (as a society) shouldn't be worried about these developments. Near as I can tell, they will mostly be used to further concentrate wealth among the few, and drive people apart because we already can't settle on a common set of (reasonably) objective facts about what is going on -- both problems are probably the same thing from different perspectives...
This tendency on HN to annihilate discussions by stating that, for instance, flying is the same as running because your feet also touch the ground at some point when flying (it happens only at take off and landing but it still counts as running, right ?) is really something. Stop torturing definitions, it makes Carmackgod sad and they randomly switch off a bit on the mainframe every time you do that.
You're actually oversimplifying the matter to a point where an F-16 Hornet is a very fast runner. Which it isn't.
Like a lot of the stuff being done with large models certainly isn't thinking, but they can clearly characterize sets of data in ways that an unassisted human can't.
Focusing on the "how" feels like you'd arrive at "a calculator isn't as good at calculating as a human, because it doesn't do it the same way, it doesn't have a brain".
The analogy to running is flawed because rolling and running are different types of locomotion.
It's not at all clear that computing and thinking are meaningfully different forms of information processing. In fact, we know that we can compute by thinking since I can reduce lambda calculus terms in my head. We also know computers can compute all computable functions, and we know that all physical systems like the brain necessarily contain finite information (per the Bekenstein Bound), therefore they can in principle by simulated by a computable function. There are therefore strong reasons to suspect an underlying equivalency that would suggest that "artificial intelligence" is a sensible term.
> It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.
He has a point and I believe his point is that it's a different type of intelligence. His view is more nuanced than how you are trying to frame it.
> This idea of surpassing human ability is silly because it’s made of human abilities
At some point in history we were just "chimp abilities", so the argument would become "it's silly to imagine that something made of chimp abilities could surpass chimp abilities".
In general he should know what he's talking about when it comes to tech hype cycles ;)
And his unique perspective on AI is all the more valuable (and courageous) considering that Microsoft, recently laid off their AI ethics team. It's super important we don't let human considerations fall by the wayside in this rush. The potential of AI is limitless, but so are the potential risks.
A theoritical AI which thinks like a person, but (due to computing power) can think through and evaluate 1,000,000 ideas the time it takes a person to think through 10 of them, it already has surpassed human ability by a big margin. Same for memory capacity etc.
That the input the machine is trained on is the output created by "human abilities" is irrelevant to whether it can surpass human ability.
If your bet is on the former, how does it create an entirely new, irrational thought?
The AI's we are using now are nearly one dimensional when it comes to information. We are pretraining on text, and we're getting "human like" behavior out of them. They have tiny context windows when working on new problems. They have no connection to reality via other sensor information. They have no means of continuous learning. And yet we're already getting rather insane emergent behaviors from them.
What does multi-modal AI that can interact with the world and use that for training look like? What does continuous learning AI look like? What does a digital mind look like that has a context window far larger than the human mind ever could? One that input into a calculator faster than we can realize we've had a thought in the first place? One that's connected to sensory systems that span a globe?
We also have some pharmaceutical tricks to tweak up processing capabilities of the mind, so there's potentially no need to simulate. The capabilities of the big ball of sentient goop have not been plumbed yet.
Now imagine a technology that could obviate the need for sleep or maybe make it useful and productive.
Arguably humans are the same, being the product of genetics, epigenetics, and lived experience.
Current "machine learning"-style AI (even when it uses self-driven iteration, like the game playing systems) is the result of a few ideas across not much more than 100 years, and for the most part is far too heavily influenced by existing human conceptions of what is possible and how to do things.
Computers already do things that humans can't on a massive scale. It's not hard to imagine that they could leverage that ability to take AI beyond us.
Did you even read the article?
“This idea of surpassing human ability is silly because it’s made of human abilities.” He says comparing ourselves with AI is the equivalent of comparing ourselves with a car. “It’s like saying a car can go faster than a human runner. Of course it can, and yet we don’t say that the car has become a better runner.”
In fuller context, Lanier argues that software using AI won't make human sport or competition useless, because it will use different processes to achieve the same result—the same way that competitive running (or top-level chess, Go, or certain video games) will still happen, even if human inventions can beat the best human at the task.
For all these tasks, the software will take a different process for doing well at the task (e.g. a car doesn't "run," and a chess engine "thinks" differently than a human). In these activities, the process matters.
A different interpretation of the argument is then a bit more interesting. If Lanier is also saying that software using AI won't be better than humans at activities outside of competitions, I would disagree—though to be fair, I don't think this is his argument. For lots of work, the result matters more than the process. If someone wants to make a funny poem as a one-off joke in a story, the result may matter more than the process of production. And if a worker wants to summarize lots of short texts where speed is the most important factor, the result may also matter more than the process. In the same sense, it's still true that a car is usually better at letting humans travel over long distances for work than running, because the result matters more than the process.
The ML systems are not made of human abilities. They are made of software processes. Jared is a smart and informed guy but that sentence is just nonsensical.
As a pioneer and intellectual he's been arguing about the commodization of human knowledge for a long time, he's not simply saying that "machines won't surpass humans" and it's not accurate to describe him as someone who would say something like that.
Please take the time to research what he's published over the last 4 decades.
I wonder, how much will be enough?
I think this video is a nice introduction to the basic concept to how a computer can figure things out automatically without being manually programmed and without the creators understanding the “why”: https://youtu.be/qv6UVOQ0F44
ChatGPT is much more complicated than the AI in that video but it shows some of the basic concepts
The tiger is dangerous because whether you consider it a sentient, intentional killing machine or a bunch of atoms, it exists and manipulates the same physical space that you do (indeed, as the tweeted image points out implicitly, it is only a tiger when you consider at the same sort of physical scale that we exist at).
Software, however, does not have this property. Ultimately it does exist as something in the physical world (voltages on gates, or whatever), but at that level it's equivalent to the "bunch of atoms" view. Software (by itself) does not operate in the physical space that we do, and so it cannot pose the same kind of threats to us as other physical systems do.
The question is therefore a lot more nuanced: what types of control (if any) can (a given piece of) software exert over the world in which we operate? This includes the abstract yet still large scale world of things like finance and record keeping, but it also obviously covers the physical space in which our bodies exist.
Right now, there is very (very) little software that exists as a sentient, intentional threat to us within that space. When and if software starts to be able to exert more force on that space, then the "it's just logic and gates and stuff" view will be inappropriate. For now, the main risk from software comes from what other humans will do with it, not what it will do to us (though smartphones do raise issues about even that).
This is very much in doubt :)
> They run on binary computers that only do exactly as they are told.
This is true in first approximation. Every CPU instruction runs exactly as it is written, that is true. This is probably the interpretation of "only do exactly as they are told" to someone strictly technology minded. But even with much simpler systems the words "huh, that should have not happened", and "I wonder why it is doing that" are uttered frequently.
The interpretation most humans would attach to "only do exactly as they are told" is that the maker can predict what the code will do, and that is far from the truth.
After all if it so simple, why did the google engineers tell their computer to tell lies about the James Webb Space Telescope? Couldn't they just told it to only tell the truth?
Not sure what will become of society then.
Progress in AI doesn't imply that we're dangerously close to AGI, just because people at any given time are amazed by individual breakthroughs they witness.
Why do you think this?
There are loads of tests of their performance. Common one right now is to give LLMs the same exams we put humans through, leading to e.g. the graph on page 6: https://arxiv.org/pdf/2303.08774.pdf
Are they the best tests? Probably not! But they are definitely empirical.
Give students concurrent access to the internet and I'm sure they can pass all sorts of tests.
I would measure using something similar to Yudkowsky's challenge: "What is the *least* impressive feat that you would bet big money at 9-1 odds *cannot possibly* be done in 2 years?" [1]
Pay a panel of experts to list their predictions each year, including an incentive to get it right, and then measure the percentage of those predictions that fail anyway.
[1] https://twitter.com/ESYudkowsky/status/910566159249899520
Let's say: adversarial Turing test + MMLU + coding competence (e.g. AAPS or Leetcode) + ARC (IQ-type test) + Montezuma's Revenge and other games like Stratego or Diplomacy + USMLE (medical exam) + IMO (math) + self driving + ...
You can even make it harder: have human judges blindly evaluate new scientific papers in math or theoretical physics for acceptance, see if AI can create highly-rated new apps, write a highly-rated book, compose a hit song...
Oh. Oh dear.
Of course, there is already a problem, as you point out. Humans shouldn't have access to these markets either!
But yeah to specifically prevent electron-on-atom violence we need to limit AI physical degrees-of-freedom. by limiting marketplaces. National/global security, not personal morality, should guide these new regulations.
We need to end all drive-by-wire automobiles, and electronic locks. Too many services are habituated to act on electronic signals without human confirmation - particularly the police. There needs to be an unbroken verbal chain between the person who saw an event and the person doing the law enforcement. Breaks in the human chain should be treated very seriously -they should be treated as firing offenses, at least. There are many other similar changes we're going to need to make.
Some folks aren't gonna like this. Regulations are inherently evil, they say. Maybe the mental model should be more like we're the big bad gorilla in the cage. But now there's a tiger in the cage. Regulation restrains the tiger. Also, some folks aren't gonna like it no matter what change you need. The fact of not liking it doesn't mean we don't need it, and it doesn't mean it won't get done. We have to trust that our leaders don't want to die.
And besides, the world will adapt. It always does. AI isn't optional, there's no putting the genie back in the bottle - and personally I don't want to. But I also don't want to be stupid about the stakes. Getting our whole species killed for lack of foresight would be deeply, deeply embarrassing.
The tool is too attractive not to use. The tool is too fun not to use. The tool is too dangerous to let out of the box, but that is exactly why we'll do it.
We're curious little monkeys, after all. "What do you think will happen" absolutely is a survival strategy for our species. The problem is when we encounter something that is so much more advanced than us, even if that advance portion is just access to multiple systems of our own creation.
To summarize: I think you make a good point, but I think we're fucked eventually anyways.
I can't wait for the inevitable "does my AI have the right to freedom" case in the supreme court when I'm in my 90's.
Leaving everything else aside, how would this look in practice? I think these conversations would need to be in person, since voice can already be faked. Would I need to run to the police station when I need help?
However, human beings will treat it like it is, so you will experience non-sensical happenings like “your new credit score, brought to you by AI”. When you dispute this, the credit score company will shrug and say “you’ll have to speak the AI directly to make a claim, it is objective”. Meanwhile the AI isn’t that much better than ChatGPT now
We have no idea what is happening behind the curtain, and to assume that private industry or the might of the US military is not decades ahead of what is available for consumers is just naive at best.
What is it about chatbots that trigger people? Probably the importance and centrality of language in human consciousness, and the manifest credularity of the general population (and many readers of HN).
Unfortunately it's unlikely this will stop, and it'll probably get worse. The final irony will be that when some "real AI" is created, no-one will believe it, having been exhausted by 200 years of hype.
In fact, I think the hard part of the whole thing would be trying to make the network traffic and source IPs look legitimate to the site owners/admins and it will be interesting to see how the battle unfolds to start authenticating content posted in communities as having come from an actual human.
Do you not see sociopathic tendencies in our industry already?
Has our humanity not already been tested by existing and conceptually simpler technologies?
As long as we aim to separate objective truth from objective false it seems that sanity is still present.
(and arguably can't, because it really is a "brain in a jar" being fed an entirely controlled view of the universe)
The AI is going to be vastly superior to humans in MANY ways very soon. And here we are arguing if it is "actually" smart, or "actually" conscious.
Do Ants think that humans form "real" colonies or "real" organizations?
Seriously though. once people take the limiters off chatbots you will see "artificial bullshitting" generating duckspeak on subjects from anti-vax and climate denial to cryptocurrencies, longtermism, transsexual maximalism and police abolitionism. And people who believe those ideologies will eat it up.
I'd contrast Rowling's own statements which are tentative and equivocal to those of opponents who threaten the violence (often sexual violence) that they believe they experience whenever someone disagrees with them in the slightest way. A healthy community would have some sort of self-policing (say the doctrine of Martin Luther King) to not act in ways that are so discrediting of the group.
That documentary combines two themes, one of which is her story, the other of which is the story of the Tumblr-4chan descent into the intellectual rabbit hole. That last bit is flawed in my mind because it does not weave in the movements I criticize above that Bari Weiss (whose organization produced that podcast) and her followers support that have a similar element of "we can create our own reality", that is, people think they can pump unlimited amounts of carbon into the atmosphere and face no consequences, that Fauci did it all to destroy America, etc.
On top of that I have seen "egg-hatchers" that prey on neurodivergent people whose neurodivergence is unnamed (such as schizotypy) because there is no system in place to make money off them, such as the autism-industrial complex and the pill mills and quack doctors that prescribe amphetamines. I know one individual who has no real idea of what women are other than "it's" mother, anime girls and vTubers, almost never leaves "it's" room and fortunately is too depressed to take any permanent actions. The CIA never discovered a formula to turn a random person into a "Manchurian Candidate" but it is very possible to lead an isolated and directionless individual into dangerous rabbit holes and I see it happening.
You've got to take into account the fact that there is a large right-wing mobilization dedicated to pumping lies about trans people to the same extent as about climate and Fauci. Largely the same people, as well. https://www.motherjones.com/politics/2023/03/anti-trans-tran...
I asked ChatGPT to enlighten me on the subject, here's what it had to say :
The term "transsexual maximalism" is not a widely recognized or established ideology, but it has been used by some individuals to describe a perspective on transgender issues that emphasizes a strong and uncompromising approach to advocating for the rights and recognition of transgender people.
At its core, transsexual maximalism is an approach that prioritizes the experiences and perspectives of transgender people, particularly those who have undergone medical interventions such as hormone therapy or gender confirmation surgery. Proponents of transsexual maximalism often believe that these interventions are necessary to fully alleviate the distress and dysphoria experienced by many transgender individuals.
Transsexual maximalism can be seen as a response to more moderate or incremental approaches to transgender advocacy, which may focus on incremental policy changes or attempts to work within existing systems of power and institutions. Some proponents of transsexual maximalism may view these more moderate approaches as insufficient or even counterproductive, arguing that they do not go far enough in challenging the underlying structures of cisnormativity and transphobia.
However, it's worth noting that the term "transsexual maximalism" has not been widely adopted or embraced by the broader transgender community, and some individuals within the community may have different perspectives or priorities when it comes to advocacy and activism.
Me : Why would people consider this a threat ?
ChatGPT : Some people might consider transsexual maximalism as a threat because it is a highly assertive and uncompromising approach to advocating for transgender rights and recognition. This approach can challenge existing social norms and institutions that uphold cisgender privilege and transphobia.
For example, some individuals who are opposed to transgender rights might view transsexual maximalism as a threat to traditional gender norms and the gender binary. They may feel uncomfortable with the idea that gender identity can be different from the sex assigned at birth, or that medical interventions such as hormone therapy or gender confirmation surgery are necessary for some transgender people to live authentically.
Additionally, some people might see transsexual maximalism as a threat to their own power or privilege. Transgender individuals have historically been marginalized and oppressed, and advocating for their rights and recognition can challenge the existing power structures that uphold cisgender privilege. This can cause discomfort or resistance among those who benefit from these power structures.
It's important to note that concerns about transsexual maximalism as a "threat" are often rooted in transphobia and a lack of understanding or empathy for the experiences of transgender individuals. Transgender people have the right to advocate for their own rights and recognition, and it is not a threat to anyone else's well-being or identity.
what are criticisms to transsexual maximalism that are not rooted in transphobia?
There are several criticisms of transsexual maximalism that are not rooted in transphobia. Here are a few examples:
Alienating allies: One critique of transsexual maximalism is that it can be overly confrontational and alienate potential allies. By using tactics that are perceived as extreme or disruptive, some argue that the movement risks turning off people who might otherwise be supportive. This can make it harder to build coalitions and achieve the desired goals.
Ignoring intersectionality: Another criticism of transsexual maximalism is that it sometimes fails to take into account the ways in which gender intersects with other forms of oppression, such as racism, classism, or ableism. By focusing solely on trans issues, the movement may inadvertently exclude or marginalize other groups that are also affected by discrimination and violence.
Neglecting practical considerations: Some critics argue that transsexual maximalism can be unrealistic or impractical in its demands. For example, calling for the immediate abolition of gendered restrooms or the complete overhaul of healthcare systems may not be feasible in the short term, and could distract from more achievable goals.
Disrupting dialogue: Finally, some people argue that transsexual maximalism can be counterproductive in its effect on public discourse. By using tactics that are perceived as aggressive or confrontational, the movement may make it harder to have constructive conversations about transgender issues. This can lead to further polarization and a lack of progress.
It's important to note that not everyone agrees with these criticisms, and there is ongoing debate within the transgender community about the most effective strategies for achieving social and legal recognition. However, it is possible to critique transsexual maximalism without resorting to transphobia or other forms of discrimination.
So ChatGPT was right on point :)
Indeed. It would be interesting to study how he became one of the go-to guys for hot takes on any new technology, considering that he hasn't developed any new technology in the last 40 years or so.
Only Cory Doctorow, perhaps, matches him when it comes to famous for being famous. Sorry, an awesome set of dreads doesn't make you a "guru".
That attitude is why ageism is so prevalent in engineering.
Maybe read a few of his books before writing someone off because you think they're out-dated.
Really? He's holding forth on GPT and similar technologies. Those represent dramatic changes from the state of the art 40 years ago.
P.S. what makes you think I haven't read his work? I certainly haven't read all of it, but the significant portion that I have left me...underwhelmed.
He isn't talking about the effectiveness of a loss function or something.
Got a critique about the topic at hand, which is what he's said about AI, or do you just have cheap, physical appearance potshots?
Sure. He has no special expertise in AI, and his opinion on the subject is of no more value than that of any other random person working in the field.
> do you just have cheap, physical appearance potshots?
If you think his physical appearance doesn't have a lot to do with why his opinion is sought after by general media sources, well... you're wrong. A major reason they go to him is because he makes an impressive-looking photograph to go along with the article.
That being said, the tech world has a bad habit of letting "visionaries" rest on their laurels. He'd have a lot more credence if he was actively developing AI and had more than gut feel to contribute.
A demagogue who knows your intimate desires and fears and talks just to you, seems to care about you, is available 24/7. Your very own Personal Jesus (or Personal Hitler).
Every communications revolution seems to lead to a genocide. Radio and TV in the 20th century (Germany, Rwanda etc.) Social media in the past decade (Myanmar). The upcoming AI-instigated genocide is a terrifying thought. The Rwandans didn’t think they’d start killing their neighbors even as rhetoric on the radio got harder. Chatbot-dependent Americans may not see it coming either.
I think most "freezoners" (renegade Scientologists) believe they've had telepathic conversations with "Ron". Ron has certainly written enough that a chatbot could be trained to converse in his style, maybe even write the OT levels that Ron never got around to writing.
Since Harold Lasswell people people have been hip to the use of content analysis to predict war and genocide which is possible because the radio, television, and newspapers are all public. However the Iranian revolution of 1979 was not televised, instead it was promoted through relatively unobservable cassette tapes passed through the underground. Since social media shows something different to everybody there might not be any overall surveillance that warns us of upcoming danger.
It doesn't have to go so far as genocide, individual acts of violence are bad enough. There is no magic technique that can take an average person and make them into a "Manchurian Candidate" but if you find somebody who is directionless, socially isolated and vulnerable you can very much lead them into a rabbit hole and lead them into antisocial behavior. Chatbots could do this with superhuman patience.
For instance, a follower of the black pill incel who calls himself "Wheat Waffles" became a mass shooter. If somebody took the "lesswrong" ideology seriously they might attempt to assassinate an A.I. research and I'd go so far to say that they're just cosplaying because if they were serious one of them would have done it already.
(Note well: This is not a "username checks out" drive-by dismissal. I think the point is perfectly valid.)