Negligence is still a possibility, if there are significant reasonably foreseeable consequences.
28,300 karma · joined May 16, 2014
Almost. In a way.
And everyone had a fairly good idea what fission and fusion bombs would do once built. (Teller was worried Trinity might set off a nitrogen fusion reaction and kill all life on Earth, but Bethe and others proved him wrong before testing.)
No one knows what the limits of AI are. It's not just untested, it's unmodelled, and unplanned - build it first, worry about consequences later.
It's just as likely - far more likely IMO - that we're physically incapable of understanding physical laws and physical systems on their own terms.
Hypothetically.
But yes - superpersuasion is the real danger. We're very, very persuadable and easy to manipulate, and the voters with the lowest cognitive abilities are trivially easy prey, with a huge ROI for minimal investment.
The AI bot farms are already running. What we haven't seen yet, so far as we know, is spontaneous superpersuasion aimed at leaders.
Are leaders any less susceptible to it than everyone else? Especially if they're narcissistic and easily flattered?
As for defences - capital is just political power by other means. If the capitalist is wasting money on yachts while workers can't afford food and housing, the return is morally disproportionate, and the usual outcome is socially catastrophic, in various possible ways.
Capitalism reduces every transaction to personal opportunism, with no concept of broader consequences. That's a shockingly stupid way to run anything - as we're all about to find out.
Also, everyone makes mistakes. But some people are, quite literally, a danger to themselves and others, and everyone needs to be protected from them. Including themselves.
If you don't believe me, try hacking some government sites and see what happens to you.
But OpenAI is a huge corporation and low-status laws don't apply.
Although to be fair the US has a number of prominent high status individuals who clearly belong in jail, so it's not as if Altman is getting uniquely personal treatment.
No one in the West has had the Eastern European experience of queuing for bread for hours and discovering there isn't any.
In the West we have exceptionally strong normalcy bias and can't imagine a world with severe scarcity.
In countries like Poland that's still lived memory.
It's at everyone else's expense, but so is everything else they do.
I very much can.
That applies to most software.
This doesn't solve the problem of "Who gets paid now, and for what?" at all. If anything it makes it worse.
It took maybe a century and a half for the modern consumer economy to appear. And it didn't happen until money started being shared into the wider economy for political reasons, not technological ones.
Meanwhile the enclosure of common land was catastrophic for rural populations. Sources of food and pasture disappeared as the Big Houses took them over and turned them into huge single family estates.
What business do you have telling everyone your personal experience justifies the horror others experience?
If you couldn't put down that $25, would you really not be able to find some other way to entertain yourself?
The reply was always "It's hard, and there are always going to be bugs."
But it's not just about bugs. It's about systems that are hard to use, poorly designed, and opaque. And sometimes the opacity is deliberate. There are dark patterns, outright lies about what happens to data, and more or less obvious grifts.
Was any of this truly great before AI arrived?
Is it an accident that you can't kill an MS365 subscription if you have more than X amount of GB on OneDrive, but the actual usage includes all your spam and email attachments, and it doesn't show up unless you know where to find it, and the location is very non-obvious, and so is the cleanup and deletion process?
Or that the eBay fee structure for international sales is utterly incomprehensible without automated help?
Or that if you select Subscribe and Save on Amazon your cancellation date is something like two weeks before the next delivery?
Or that if you sign up to Discord you need a mobile and an email for activation, but the site doesn't tell you this, and nor do tech support, who insist that a mobile is optional?
And software quality - services down, records lost, records stolen, photos and documents deleted - is a whole other layer on top of that.
AI is a moderately good solution to the first set of problems, because it can search and assemble information far more quickly than you can. So IME it's pretty damn good at tech support - not perfect, but better than DIY in many cases.
Software quality? We'll see what happens. If tech stacks collapse over the next couple of years we'll know AI was terrible thing.
I'm in the 'Too early to tell' camp. There's a fair chance they might. But if so it will be because of poor testing and design, not because of code review or lack of hand-rolled code. And it's not completely clear that quality levels wouldn't have dropped anyway.
The DX7 was a hit because of the polyphony and touch sensitivity, but also because the presets - especially the synth basses and e-pianos - were so obviously musical and immediately useable.
But then you listen to the music Chowning was making, and you can already hear a lot of the tone quality. It's academic electronic music, so no bass and e-piano, but it's pretty impressive for 1977. So it's clear he was musical in his own way, but more not interested in using synths in pop than not interested in music at all.
Later cards used PROMs, and were pretty fancy. The M9312 had 512 words and included some basic CPU diagnostics. Different variants handled boot from disk, paper tape, console, etc, and you had to specify the options you wanted when you bought the card.
Do you really believe developers have exactly zero moral responsibility to their users? None at all?
Really?
Would you still believe that if you found the developers of ssh deliberately included a backdoor for hackers, or neovim was uploading everyone's code to the NSA?
This is the extreme version of "I enjoy coding but I hate users."
It may be legally valid in absolute technical terms, but I'm going to take a lot of persuading that it's a healthy attitude.
It's a straightforward subsidy for employer profits.
Maybe a few thousand people for personal services of... various kinds. But no one's going to need the rest.
It's the ultimate capitalist fantasy.
And of course it won't happen, because long before things get to that stage AI will have independent plans of its own.
(Which is just as well, because if things did get to that stage the emperors would all wage war on each other rather than living peacefully and productively.)
I don't think we can imagine a post-ASI culture because - by definition - we're not smart or inventive enough.
It's not just farmers -> superfarms. Although in fact that did happen, but largely as a footnote to developments elsewhere.
It's more to do with the fact that our visions of the future haven't changed for over a century. They've been implemented in unexpected ways, and there have been unexpected social and cultural changes. But you can easily see the outlines of modern technology as far back as the late 19th century.
With ASI, the outcome could easily be something that doesn't look and act like technology at all. It would be some unimaginable New Thing. Literally no one on Earth has any idea what that would be or whether there would be room for trad-humans in it.
My memory of Nokia is that the apps and services just didn't work that well.
Just after I got my first iPhone I arranged a meetup with a friend. I used Google Maps on the phone to find my way, and it worked perfectly - almost unthinkable at the time, because we were all still using paper street maps while driving.
She was late because the map on her Nokia phone had gotten very confused and sent her to the wrong location.
The Nokia experience was like that. Before the iPhone I'd had a couple of Communicators. They were incredible in theory, and I sent bad photos in emails from various unlikely locations to prove it.
But they were big, slow, and clunky in practice. The iPhone wasn't.
And it did. Not by using a music OCR tool, but by inventing one.
No one is spending millions training frontier LLMs on symphony scores or literary fiction. But at this point I'm fairly sure they'd get workable results if they did.
I've been tracking music in AI for more than a decade, and this is the first time I've been able to say that.