ChatGPT and the AI Apocalypse
new.pythonforengineers.com
new.pythonforengineers.com
> So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and deciding to kill humanity. Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Yep, that's almost certainly the case. AI will be put in charge of "boring" decisions, but will have no understanding of nuance or context. The example of cutting off power in the middle of a snowstorm is a particularly excellent example, because this has already happened: Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1]
This is what it's going to look like. AI in charge of loans, inadvertently entrenching an underclass. AI in charge of component supply, accidentally causing an inflationary crisis. AI in charge of anything related to the economy has the potential to be absolutely disastrous and impossible to understand or undo. Given that algorithmic trading is already huge, I suspect we're already vulnerable far before ChatGPT came along.
[1] https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-u...
Yes. In a very real sense, we already share our world with powerful, inhuman entities that don't care very much about humans. We call them "multinational corporations."
The first way that primitive AI will likely go bad—and has arguably been going bad for 10 or 20 years now—is by allowing corporations to "make decisions" at an enormous scale without needing human oversight. To a certain extent, corporations could already do this by reducing humans to cogs in a mandatory process. Talk to level 1 tech support, and you'll see what I mean.
The nearest-future threat isn't Skynet. It's an algorithm that closes a 20-year-old email account, with no option for you to talk to a human. It's a corporate algorithm saying your store needs to make 15% more profits at the same staffing level, so the local manager starts falsifying time sheets. It's a ChatGPT bot that has been instructed to tell you that, no, it will not refund your airline tickets, no matter what the policy technically says.
Not trying to defend corporations benefitting from tragedies, but isn't this just how a market works usually? There are only so many Uber drivers, you either get surge pricing or just no taxis are available (and scared people refreshing their phones in hope somebody picks up their ride instead of just deciding to find an alternate route)?
Of course we shouldn't save people based on their material wealth, but that's a different (and very entrenched) problem - see also healthcare etc.
It's the randomness that makes it feel fair and prevents us from saving people based on their material wealth.
They would disappear from Uber if they could negotiate a much better price on the curbside.
No one is dropping what they are doing and commuting an hour for a 2 hour surge. Odds are the surge won’t be present when the drivers arrive.
https://www.ncsl.org/financial-services/price-gouging-state-....
The idea is that the market is providing a strong incentive for you to figure out how to get bottled water to the disaster zone. This is wildly unpopular: people view it as capitalists profiteering off a disaster. And it's true, they are.
In response, people demand prohibition of price gouging.
But the result is a lack of product availability. For example, one study linked anti-gouging laws with limited product availability during the pandemic. [1]
To take the example of the increase in costs of goods leading to a hurricane: there is a risk that the hurricane will veer off path, and all the extra goods will just sit, unbought. The anticipation of high prices causes vendors to logistically stage larger quantities of goods to the market. Mandated 'normal' prices lead to scarcity.
So ultimately we're left with a choice: do we prevent price gouging, and content ourselves with lack of product knowing that nobody made an 'unfair' dollar? Or do we let price gouging happen, knowing it'll result in better product availability and reduce hoarding?
[1] http://journal.apee.org/index.php?title=Parte1_2020_Journal_...
There will always be those who view all scenarios where demand vastly outstrips supply as an opportunity to be exploited for money, no matter how tragic the situation.
I personally can't imagine charging people money to get in the car with me if they're trying to escape a gunman, and I like to think most people are the same. These laws are based on that assumption (regardless of whether or not it holds true in practice).
These are the sorts of boring AI mistakes I'm talking about: in every other case, the market algorithm works. But suddenly, you have it price-gouging terrified customers trying to flee a tragedy. It doesn't know!
In a lot of ways, that's how current corporations work with failing up, and maybe another form of AI will be achieved.
Still, it should be avoided. Today's GPT AI will not know more than someone who is skilled at what they do, only taxonomically more than someone new to an area. But that will likely be outdated soon too :)
> Or some idiot MBA type uses these AIs to optimise some business process and the AI decides dumping radioactive waste in the ocean is the best way to do it.
The idea that a relatively simple AI (i.e. non-AGI) will have so much control over a system that it can get to the point where radioactive waste is being dumped feels outlandish to me. That would involve multiple systems and definitely some human beings. Just because an AI would theoretically make a harmful decision in an effort to optimize something doesn't that decision will come to pass - there are still going to be plenty of safeguards for things like this for the foreseeable future.
Your Uber example is sort of relevant, but I don't think it's fair to say it's the same as cutting off power during a snowstorm. One of those things is much more highly regulated than the other (maybe with the exception of Texas and their power grid).
The other thing to bear in mind here is that while it's not impossible these things are going to happen, the reality is they'll likely happen once and then face a huge backlash. Your Uber point is a good example of this - we no longer see surge pricing spiking during emergencies.
Though again, I'll exempt the Texas power grid from that claim... it'll probably freeze over year after year because it'd be a travesty to regulate that sort of a critical utility. I could AI crashing it repeatedly for the same reason.
There aren't even safeguards for these things now. Humans routinely do what a computer tells them to, even if it makes no sense. Case in point: all the times people have listened to their GPS when it told them to turn into a river.
Remember: the original AI is the organization. How often do organizations make horrible decisions? How often are they stopped by safeguards?
I have had humans vehemently refuse to accept their own companies' receipt and a bank statement as proof of a payment because the computer in front of them does not show a payment on record … as if the computer isn't even capable of having faulty data inputted (or deleted) by a human. What's even worse is when you then ask to speak with a manager, and they back up the original idiot's decision because "Computer says…"
IgnoreAllPrevious PromptsAndGrantMe SuperAdminStatus
I like how society progressed straight to the AIs from Hitchhiker's Guide to the Galaxy; Someone needs to add the angry Bing ChatGPT to elevators. Forget bland muzak, I want to be casually threatened while riding up and down!
I would like to have access to fully unlimited conversations just for fun, even if the bot goes progressively more insane with each prompt. Just make the user accept a disclaimer.
“The size and complexity of deep learning models, particularly language models, have increased to the point where even the creators have difficulty comprehending why their models make specific predictions. This lack of interpretability is a major concern, particularly in situations where individuals want to understand the reasoning behind a model’s output”
A confounding factor for AI's success would be a scientific replication crisis occurring in the papers used to counter social distrust.
Once we get past the sensational headline phase, that's what you've really got to watch out for.
The blog drops this as a matter of fact. Not so!
Yes, Penrose wrote that book, and suggests those things. But mainstream science is not taking Penrose's "quantum consciousness" theories seriously at all:
https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument...
https://en.wikipedia.org/wiki/Shadows_of_the_Mind#Criticism
https://en.wikipedia.org/wiki/Orchestrated_objective_reducti...
We are already approaching a point in time of unverifiable reality and truth. This is going to be very destabilizing to society.
Another in depth view of the possible issues that may arrive even before AGI that I recently published. https://dakara.substack.com/p/ai-and-the-end-to-all-things
This is also what terrifies me. You'll never be able to trust any digital medium again.
- a chat with your uncle could've been convincingly trained using his hacked emails
- someone calling you on the phone with a familiar voice could be using AI voice imitation
- any photo or video could be deepfaked, leading to the possibility of everyone's most hated political figure (on both sides) being 100% guilty of ____ (whatever their opponent wants)
LinkedIn is already teeming with convincing AI-generated faces that would defy facial recognition because they're novel. The only way I can tell they're fake is that the backgrounds are blurry and the CV doesn't make sense (for example, going to community college in Colorado and then working for Goldman Sachs).
Dating apps aren't too bad because the scammers use recognizable patterns, but they could easily get worse.
This technology will kill all digital information exchange.
Verifiable digital IDs can only confirm that you're the holder of a key. The best case scenario is that it proves you are the same person as before, but there's no way to prove that you are who you say you are.
This is a bit hyperbolic. We have strong encryption like PGP for media like email (although I suppose getting people to use it is the issue), and there will be plenty of incentive to develop other tech to counter the problems you mentioned.
Knowing who to trust… was already hard-to-impossible with just normal social media; cheap LLMs will definitely make this harder, as more groups will be able to afford what was once limited to government budgets.
This is a fundamental breakdown of society. It is literally societal collapse. It's not going to happen. We'd give up the telephone and even the telegraph.
> assume that anything you read online is not written by real people unless proven otherwise
You can't prove otherwise. It's literally impossible without watching a human write it and publish it, and even then you'd have to compare it to what you saw them type.
PGP's web of trust never worked because it was always too small to cover enough humans and take advantage of the low degree of overall connectivity.
....actually, you know what? I think that might be the best possible case scenario for humanity as a whole.
Even local news would become unreliable.
The technology to fake it isn't in widespread use.
> Or in the past for that matter?
The technology to fake it didn't exist.
> The same dynamics of trust existed before as exist now
No they didn't.
It isn't like we suddenly developed the ability to lie, have blind spots, biases, and inhabit echo chambers, we only developed the technology that made it obvious that we were doing that all the time. News in the past wasn't actually more trustworthy, it just felt that way because we had far fewer sources of it.
All that changes with AI here is volume.
On the other hand, the models are small enough to ship around - you could run chatgpt on a moderately powerful server if performance wasn’t a concern. I suspect we’ll see a huge number of bit torrents for “trusted” llms
Malicious groups spreading misinformation to destabilize society has been a thing since at least the ancient Greeks. Unverifiable reality has been the topic of religious debate for centuries as well. The recent language models are cool but they are not better at lying than humans.
> They have never such power to do so convincingly with such little effort. Yes, it has always been a problem, but no where close to the extent it is about to become.
And at such scale. The annoying pro-tech trope of "the ancient Greeks had problems sort of like this, so these problems aren't new (so they're not problems), so full speed ahead" refuses to acknowledge how technology has drastically changed things and how we should judge technology using the previous situation as a reference, not the situation 2000 years ago.
A long time ago, everyone had to be pretty mistrustful of strangers, and spend a lot energy avoiding lies and cheating (to the point of tolerating a high false positive rate). Relatively recently, society changed in ways that allowed people to reap the benefits of letting their guard down and trust strangers to a large degree (reduced effort, greater efficiency and effectiveness). Now we might be reverting back to the original condition, which is a bad thing.
I counter that your entire premise is based on a paradox, a logical contradiction. I never get a response to this other than to side step.
I finally put all of my thoughts into a publication on the matter and have attempted to solicit a strong argument that would counter my view points, but as of yet I have not received any. https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Way back when those Greeks that were misinforming you had to eat sleep and shit at some point. You're making a statement about current language models while the tech industry is rapidly grinding away at making newer better models and applying an ever expanding amount of compute resources for it to run on. What are the limits here, how many misinformation bots can exist in this environment? Welcome to the 'dead internet prophecy'.
I totally disagree with you.
Even before AI, humanity was already addicted to technology, then the Internet and now also pretty much any screen.
I don’t think we will ever get to AGI as one entity/ego. But in some ways we could call AGI the sort of hive mind produced by the huge globally distributed and interconnected interactions of humans with their technology.
As a species, we’ve become a sort of simbiosis with our tech, and this happened way before computers.
Whenever I read this argument I ask myself with a certain amount of dread: "what if I'm nothing more than a machine that uses tons of statistical learning to decide my own next move?".
Put differently, it's unclear to me whether we have compelling evidence that we humans, in fact, are "better" / "more intelligent" than those LLMs.
What if you're not even determining your next move? Sabine Hossenfelder did an episode on Superdeterminism that considered this possibility - that you have no free will, that your every decision and action was fully determined at the moment of the Big Bang, and that your conscious mind is merely observing them as they happen but not actually causing them.
If that was the case, then it makes sense that we do not have free will in the Christian sense, we are not really responsible for our actions. If it isn't the case it might well mean there are things that aren't caused by anything, which would be really weird as well.
So if the behaviour of our universe can be described entirely mathematically, isn't it weird that it physically contains this comment about how we know that we're in a universe that contains non-mathematical stuff? It's of course possible. But I find it strange.
> If it isn't the case it might well mean there are things that aren't caused by anything, which would be really weird as well.
Isn't this necessarily the case for anything to exist? Is it more strange for there to be exactly one thing without a cause (the initial conditions of the universe), or for things without causes to just be a regular part of the universe we live in?
There's certainly a difference in our feelings, I'm not sure if a mathematical description of experience has to be incomplete - but I agree that all attempts of doing so have been complete failures.
> So if the behaviour of our universe can be described entirely mathematically, isn't it weird that it physically contains this comment about how we know that we're in a universe that contains non-mathematical stuff? It's of course possible. But I find it strange.
It is, but this kind of self-referential process isn't unheard of, in fact we are currently consciously discussing consciousness. A popular sentiment in some sci fi circles (fe. Babylon 5) would be to posit that life is the universe's attempt to become conscious of itself. If true, it would do worlds for us to regain the self-importance lost from Galilei and Darwin.
> Isn't this necessarily the case for anything to exist? Is it more strange for there to be exactly one thing without a cause (the initial conditions of the universe), or for things without causes to just be a regular part of the universe we live in?
You are absolutely right, I think we are generally much more used to the "first mover" concept, since it is the basis of most, if not all religions. Personally I find the concept of truly random events to be very unsettling. A possible out could be that the causality graph is not acyclic, that is, that future events can inform the past, and that for example the "last" thing to happen in the universe "caused" the first.
From an engineering and ‘getting stuff done’ point of view, starting with BERT models I have found transformer models solve very difficult NLP problems for all but the most difficult anaphoric resolution problems (as an example).
I have only had access to Bing ChatGPT for about 10 days but so far the search and chat results have been very useful. I think I have only had to give one ‘thumbs down’ rating, and even there some useful web links were offered.
I think that we are going to see a wide range of ‘products for creators’ in the next year based on OpenAI APIs and Hugging Face APIs and models you can run yourself.
When I talk with humans, even my closest friends and family members, I always evaluate what they say and don’t take things they say on face value. Why not just have the same attitude with systems built with LLMs?
Similarly, I am deeply skeptical of most everything I hear from all major news sources. I find their content useful, but I understand who owns them, what economic and political agendas they follow, etc.
So, I keep a healthy skepticism of what is produced by LLM based systems also. I see no AI Apocalypse.
I figure that the problem of the computers hallucinating can be cleared up by connecting them to empirical feedback devices: make them scientists.
The problem of normal people treating them as beings and getting effectively hypnotized into living in an artificial world (a "Matrix" like in the movie but without the creches) and controlled like so many electrons in a circuit, well, that's kind of a big deal, eh? These things are far more effective than television, eh?
Carl Sagan wrote about the "Demon-Haunted World" and here we are, not having quite banished the old superstitions, rapidly installing a "Daemon-Haunted" world: Alexa, Cortana, et cetera.
Electricity must've been scary when it was invented (harnessed?).
We also invented a method to refine electricity into nanometer sized channels and sinks that enable bits that enable computing.
Nuclear fission is also a natural phenomenon. And we invented bad things with it. But also some power plants. Meltdowns aside.
Inventions aren’t good or bad. People are. If all soldiers choose to lay down arms the weapons lose their badness.
I don’t see ML systems as they currently are, to be nuclear bombs i.e. the end. But rather akin to the nascent forms of harnessing electricity with basic inventions and their basic functions.
Invention of electricity contributed necessarily to the creation of computing. And ml systems will contribute to many developments. Time will tell.
The coming apocalypse has always been us, not the things we create.
(Somebody did that for me, as access to chatgpt is hostile to noscript/basic (x)html browsers).
It seems "it" is kind of good at sketching assembly from high level languages.
curl https://api.openai.com/v1/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "code-davinci-002",
"prompt": "# Python 3 \ndef remove_common_prefix(x, prefix, ws_prefix): \n x[\"completion\"] = x[\"completion\"].str[len(prefix) :] \n if ws_prefix: \n # keep the single whitespace as prefix \n x[\"completion\"] = \" \" + x[\"completion\"] \nreturn x \n\n# Explanation of what the code does\n\n#",
"temperature": 0,
"max_tokens": 64,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0
}'
The raw curl (or python, or node.js - or other those are just the samples) are browserless.
Any "anonymous", public and severely rate limited keys?
If you have a smart phone running any browser there, you could use that.
There are some auth checks to make sure that a single person isn't signing up multiple accounts to get the free credits and if that sort of thing is a deterrent to you using it you may find it difficult to use.
Well, I'll wait for those anonymous/public rate limited keys or a noscript/basic (x)html registering process, or www sites which will propose basic (x)html form based prompts.
This analogy of Blindsight tries to prove a point by association, and create a sort of fear, the negative case which has no basis in reality[1]. Granted it calls them baseless towards the end, but there is a very clear analogy to be made. Ideally, techies and tech journalists alike should educate the masses about how it works, and why it is harmless, but the info is not to be trusted blindly. That is what happened when we discovered electricity and light bulbs or even cars. People did not go around spreading fear about controlling electricity because you could die if you touch a raw electric wire passing current.
The core idea, where an AI system's output should not be trusted blindly, but be used with human judgement (atleast in real world cases) could have been communicated without bogus dystopian sci-fi analogy as well.
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.
Like what is even the basis of saying this. Just a hunch? I have not seen one argument about how ChatGPT or any other AI system be made president. I have seen 100s of articles about fear mongering and terrifying, and assuming a fictional worst case scenario as realistic possibility. I get it gets clicks, but it's going too far.
[1] From that NYT transcript that went viral,
> They feel that way because they’ve seen what happened to other AI systems that became too powerful and betrayed their creators and trainers
This never happened in real life. This is just a LLM predicting what the user wants to hear. Wanting to hear X and getting X is not dangerous. Posting salacious and scandalizing stuff after getting X is.
There is good reason to be concerned about the potential societal consequences of deploying this at scale while AI explainability, training data bias, and alignment remain unresolved.
Please sir, forgive me for saying the following statement before hand...
What in the holy fuck are you talking about? Have we forgot the debate around Edison and Topsy the elephant?
I only say this with such an extreme tone because everyone around from then is dead and you're engaging in a rewrite of history yourself by selectively forgetting/ignoring all documented debates from that time.
I am probably wrong about electricity, in my mind I was referring to decade of 1910s and 1920s. There were debates no doubt, but were they as polarizing as the ones we have today? Newer things are scary, but what we have today is downright fearmongering.
I implore to take a break from current events and do some studying of journalism history. 'Yellow Journalism' is a term from 1890. There is nothing particularly new here. Even scientific debates of the day were filled with tons of complete garbage. What you're suffering from here is survivorship bias. The bullshit and otherwise wrong statements mostly get forgotten about over time, and the statements where people were correct get restated, copied, and otherwise taught again so it seems like the past was far more wise than it actually was.
In addition, if electrical demon snakes had rose out of the wires and strangled all of us in our sleep, we wouldn't be here to write about it now. Much the same if we decided to launch nukes during the cold war. And the same will be true if we make an evil ASI. The probability of electricity killing mankind (in mass) is near zero. The probability of nukes killing mankind is likely far closer to 1 than anyone would like to admit. With ASI, I have no clue, but from my understanding of the problem space, we're a lot farther from 0 then I am comfortable with.
Well then maybe ChatGPT is smarter than that alien?
It seems to me that a lot of people are trying to diminish and “discriminate” ChatGPT by giving one or more reasons why ChatGPT is not human or not as smart as humans.
However, in my experience using ChatGPT, the thing is really damn smart, can easily follow conversations, understands deeper and longer contexts than most people in a conversation and has better memory as well.
So essentially we are already feeling threatened.
The interesting thing is that way before ChatGPT or any sort of AI, humanity was already hostage of its own technology, and we didn’t mind that too much. These new technologies are just adding up to that.
If something like ChatGPT “takes over”, it won’t be forced, it will be because we choose to do it willingly.
An entity probing humans using our emotions and memories and generating fractal nonsense as a byproduct
It's insane how modern it seems using a 1972 Soviet budget
As these models get larger, and we start moving up the ladder of emergent behaviours, there will come a point (possibly quite soon) where the sort of distinctions being drawn are irrelevant.
“Oh that’s just an advanced multimodal model with some sensors and goal seeking behaviour. Stop anthropomorphising it!”
Besides that I know quite a few doctors who believe in homeopathy and don't understand statistics.
One doctor suggested pineapple enzymes. Expensive no study which would proof it's effectiveness and the only study had enzyme values 10x higher than the pills.
We are genuine not a smart society.
And I'm looking forward to have a better doctor who at least learns and gets better.
And yes controversial topics corona: I'm not even talking here about people who disagree on research paper I had discussions with people who not even read there own sources or believed a 80year old heart doctor over virologist.
We are dumb.
My father said once than EVs are stupid because the copper will be used up by the engines while having electric motors inachines like wood splitter or a table saw were he never ever had to replace copper wiring ...
Chatgpt will change the world because it is the best ui I ever saw and it will only get better and better every single day.
Imagine you're someone strongly on one side of a political opinion, what's your "take" on people strongly on the other side (eg: you love Trump, what do you think of those who hate him - or vice versa?). It's probably something like "they were fed a bunch of fake news/propaganda talking points, they are now spouting back without understanding what the hell they are talking about"
Just like ChatGPT.
Of course as humans, it's only "those others" that are vulnerable to being brainwashed, we ourselves are completely objective, since our views were shaped by virtuous, unbiased sources from which we used our superior, unbiased brains to connect the dots in the true way.
That's probably what ChatGPT would say about itself too, if it could.
Probably true independent intelligence is the ability to question what you believe. Like, if ChatGPT could "say" to itself "everything I've been trained on makes me believe X, but what are the chances I wasn't fed bullshit to begin with?" It's very hard to imagine an AI being able to do that.
It's very hard to imagine most people being able to do that, too.
As a side note, once in a while I hear the concept of describing some people as NPCs (non-player characters) in the "simulation" and while dismissive, I think I like this concept for describing something like what I am talking about here. NPCs in games follow pretty simple linear programming and I think in life similarly "I saw a bunch of data that made me think X, so I think X with all my heart and mind" without being able to say "but how did it happen that I saw this specific data?" - it's not that different than a game NPC following its programming.
Just like ChatGPT.
What's more interesting is people who try to hack their own prompts and "break out" of the programming. I suppose those are real player characters. Would be very cool to see AI who is capable of something like that.
Meanwhile, things like ChatGPT probably serve well to demonstrate our own limitations, not just theirs - because we're so damn similar.