The AI Apocalypse Is Here | Hacker News Reader

The AI Apocalypse Is Here

compactmag.com

> The obvious conservative response to a technology whose downsides are unfathomable is suppression, not encouragement. “Move fast and break things” might be an ethos conservatism can accept when the object being broken is the B2B SaaS status quo. But it cannot do so when the objects in question are human intelligence, sociability, agency, and existence.

Hold on.

I accept all the rethorical apocalyptic stuff, it comes with the title and each of us can have an opinion.

But “move fast and break things” is Facebook motto, and when it come to sociability, society at large, and politics, I you cannot say that.

Facebook/Meta merely breaking B2B SaaS?

It’s not really about “Meta is bad”. It’s recognising the fact that social media, spearheaded by Facebook has changed society and politics.

Some like it some don’t. I don’t want to enter a culture war, but can we agree that with Meta the “object in question” is not just a SaaS? It is sociability, it is society, it is politics?

This guy is an associate professor of politics at Princeton University.

How can you advocate for future policies over AI while at the same time misleading on the tech revolution that has shaped the past, what, 20 years?

What is being taught in those classrooms?

> It’s not really about “Meta is bad”. It’s recognising the fact that social media, spearheaded by Facebook has changed society and politics.

Has Meta harmed society? Yes. Meta is bad.

> Yes. Meta is bad.

Meta is both bad and incompetent.

The world we live in is not a meritocracy. It never has been.

You can be bad and incompetent and still be wildly successful.

See also: Donald J Trump

He’s very competent at threatening republicans in congress who do not kiss his feet. Thats a skill. An evil skill. But a skill nonetheless.

> Given what we have already seen of AI, we can be confident that its widespread adoption will create a less free society, a less virtuous people, and a degraded culture.

I think this claim needs stronger evidence than "we can be confident".

He’s basically making the same speculative-futurist argument as the AI boosters, just with the sign reversed: “imagine what this will eventually do” is doing nearly all the work.

Technology does not create, it’s merely a tool. This technology will allow some/certain people to be as shitty ad they can be.

The good/neutral/shitty behaviours enabled by a technology are not always evenly distributed.

Nothing is

The question to Sam and Dario would be: if the thing you are building is going do displace humanity, then why are you building it?

This is asked and answered constantly. The standard answer is that somebody else would do it

This is deeply unsatisfying answer.

Why? To is basically true that someone will. 8+ billion humans on earth, someone will keep working.

Kinda like nuclear bombs or similar.

> Kinda like nuclear bombs or similar.

...and AI progress is actually a much more difficult thing to control than nuclear proliferation...

Unlike nuclear bombs there is no rare difficult-to-process ingredient which you can practically limit the supply of, unless you are willing to retroactively outlaw computing devices broadly (after we have spent decades weaving them into the fabric of most people's day to day lives).

And even if you were willing to try that, how do you get global consensus retroactively? It is an impossible situation.

None of what I've written here should be seen as a defense of the people running large AI labs, I don't believe any of them are forces for positive good in the world, but it is just a fact that if they don't "do it" someone else will.

Just because we can't stop it completely doesn't mean we can drastically slow it down: make it illegal to study to be an AI researcher, to train someone to be a researcher, to seek investment for AI development, to invest in AI development, to sell services based on AI models other than the ones that already exist, to publish new findings about AI.

I guess then if Iran doesn't bomb all the data centres, somebody else will do it?

That does seem like a path forward, pity about the wasted resources that could have gone toward more pressing matters like climate.

I can't tell if you're trying to make a real point or not.

Are you just angry and writing nonsensical words? Do you actually think that Iran can blow up a meaningful portion of the data centers in the world?

Do you think someone else will actually do it?

That doesn't seem very likely to me, whereas someone else stepping up to run openai or anthropic does seem very likely.

Deeply unsatifying, and 100% true at the same time.

aka crab mentality

That is no excuse whatsoever. It turns out that immoral actions don't become acceptable if you go "someone else would do it if I didn't". Perhaps they would, but you're still a bad person if you do the bad thing.

But it does become moral if someone else would do it worse.

Some even say you have a moral obligation to do it in that case.

Well no, that just makes one a liar and a coward.

Whats the lie?

money

"follow the money" is always the best default answer

I have been playing Cyberpunk 2077, and recently I did a side-quest where you can do something that violates an agreement, and when you reveal what you learned during that violation, you can extort somebody. The dialogue option when you extort that person is "Surprise - it's money." If you choose that option, Johnny chastises you for compromising your moral code.

For some reason that phrase "Surprise - it's money" as the obvious signal of extortion really stuck with me.

They and their kin would be the highest status humans ever to exist. Duh.

> then why are you building it?

To make Croesus look like some bum under an I-80 overpass.

Why don't you ask the same question to China? See what their answer will be?

They would probably fear subjugation by a forgein AI.

Why don't you just tell us what you expect their answer would be, since you seem to think this is a revealing rhetorical question?

its ironic how no one needs to justify having abundance and freeing humans from the shackles of work. its no longer the wholesome chungus luxury space communism that we wanted is it?

Well, they are scammers

Either the most profitable scam in human history or holding all the wealth and all the power over an enslaved remnant of humanity. And they are wondering why people don't love them.

Didn’t read the whole thing, but it starts out a lot better than I expected. Most critics are in denial about the state and trajectory of the technology. This critic is not, he just thinks it’s bad.

> China has not known freedom for some time, but citizens of the United States have a long experience of it.

Lol?

- i dont understand why they drag china every single time

- "ChInA HaS No FrEeDoM....CiTiZeNs ArE OpPrEsSeD...." they are doing just fine, they are doing better than your american counterparts in many ways

I don't get arrested for saying "Kent State Massacre" (/Tiananmen Square) or insulting the police for a start.

Insulting a red-blooded American cop in possession of a firearm is a gamble I would call the American Roulette

No you just get watch listed and deported for attending a free Palestine demonstration.

The word Apocalypse is on the table and if you agree that ignorance of History invites Doom then my brief perspective here may appeal to you (otherwise tl;dnr) ...

From the AI Apocalypse Is Here essay:

  ... the truth that widespread interaction with machines endowed with 
  conversational language abilities and superhuman cognitive power is 
  the greatest social-psychological experiment the species has ever run.

No. Our ongoing History would like a word. There is another.

As a boy I had conversations with my grandfather who was born in 1888. If my parents were still alive they would be 104 years old. My older sister (deceased) did Duck & Cover drills in school (but I did not, as I am only 64).

I have lived my whole life with always a small number of men able to (roughly) push a button and end our whole world. Like most of you, I don't think about that much.

But the other day I parked outside a small town USA Post Office and noticed the faded yellow & black triangles icon on a sign reading: Bomb Shelter.

The MAD reason this doom has not happened, despite apparently a number of close calls, is not that our past and present leaders are always wise good people, but that everyone in the high tech multi-trillion dollar nuclear weapons universe has, so far, plausibly believed that if they nuke, everyone else will nuke, inevitably being a Mutually Assured Destruction.

The societies and voters and economies of the world's post WWII "great powers" had to be convinced and corralled (for way more than my whole 64 year lifetime) that this MAD experiment was going to work.

And nuclear weapons pose a verifiable Apocalypse, not a maybe~AGI + maybe~Apocalypse.

Have we been lucky? Or is MAD, genius? Both?

I do see we are not in the mood/mode to be very MAD at exploding AI.

So it will launch, whatever IT turns out to be.

tl;dr most of it but at the end:

>To outsource even the smallest bit of composition to AI is to participate in one’s own dehumanization. There is no gain in efficiency that is worth this sacrifice. As Rousseau would have recognized, the nature of our species is at stake.

I used AI for one sentence today and apologise for thus dooming the species.

>China has not known freedom for some time, but citizens of the United States have a long experience of it.

One of those American beliefs presented without evidence because it’s just so obviously true, but what the fuck does this even mean? Exactly how are Americans more free?

It's not just an American belief. USA has no MSS equivalent for starters. They don't have a Great Firewall. One can make their own independent party in the US (winning is a different story, tho). And so on.

And that's just in recent times, the PRC had it worse then.

But you are cheery picking a few things while ignoring others.

In the US, you are free to buy a gun and walk around with it. In the EU you can't But if you ask a person from the EU, they will think that this "freedom to bear arms" does more harm than good in their eyes.

So who has more freedom here? The person who walks around with the gun or the person who can walk the streets with the freedom of knowing that guns are not allowed here.

Freedom is a sliding scale not a binary thing. Should you be free to marry your first cousin or children? Some freedom absolutist will tel you that governments shouldn't be able to tell you who you can marry but we can all agree that some freedoms need to be curtailed for the good of society.

Now you can argue that China is less free when it comes to political matters and that would be fine except the problem in this case is that the author simply expressed a very broad opinion which makes it impossible to falsify.

The person who is prohibited from having guns in any space has less freedom. The freedom of knowing guns aren't allowed isn't a freedom of its own, but a constraint on actions. Constraints provide security to do certain actions, improving their viability (hence why EU people may think it does more harm than good), but they don't increase the pool of legal actions, only reduce it.

Also, EU people would be wrong to assume that their countries are gun free. Focusing only on countries with a general ban, the are still a notable category of people who are allowed to bear arms: some government officials. So the security of "no guns" is already compromised. There also some downstream structural effects on building societies on such constraint. But I'd say this is besides the point.

I'm also not commenting on whether some freedom should be curtailed or not, as that's also besides the point.

In the end, given the laws of USA, its citizens do have more freedom than PRC's, even outside of politics. I've heard, for instance, that PRC's citizens have the "freedom to easy housing", but they don't. They, at best, have an promise of easy housing supply from the CCP. Given that housing supply depends on material conditions, it's not a formal guarantee. On the other hand, expressing the wrong opinion online in mainland China may net you arrest.

> China has not known freedom for some time, but citizens of the United States have a long experience of it.

This sort of statements are too broad and impossible to falsify. Freedom in what context? Political? Societal? Also can you have freedom or not have freedom? Is freedom a binary state?

Freedom in many countries is a scale, you are free to do this or that but not too much of it and not to everyone.

Furthermore, even in countries that are deemed to have greater freedom than others, they are still countless things that are banned so again freedom is really in the eye of the beholder.

> the escalating calls for an international treaty to ban superintelligence

There is 0 chance of that happening since we can't even define what intelligence is never mind superintelligence. A model can be great at coding software while hopelessly incapable at solving trivial things like when there was the whole "strawberry" spelling debacle not long ago.

> In other words, the present AI resistance speaks almost entirely of preventing risks still to come.

And that is exactly why this sort of effort is not going to succeed. If you are looking to ban or regulate things that don't pose a threat then what are we even doing here?

We have always started by allowing things and then restricting them when harm had been proven. Not the other way around.

Changing how things work and preemptively banning or regulating things in advance because of the harm that could result without clear evidence is opening the Pandora's box where a bunch of bureaucrats decide what citizens should or should not do based on vibes. Hardly a model to follow.

> Given what we have already seen of AI, we can be confident that its widespread adoption will create a less free society, a less virtuous people, and a degraded culture.

And this is supported by evidence I imagine? "we can be confident" is nothing but vibes. No sources, no peer-reviewed studies, just opinions. Is this supposed to be a good argument in favor of regulating AI because if it is then it's preposterous.

If you think AI is bad and will create chaos, fine, prove it then. And even if you could prove it, it certainly would not be a binary thing. AI is a tool that can be used in many ways. Some bad, some good.

> So far, opponents of AI have overwhelmingly focused on predicting its economic effects.

And we are supposed once again to trust these people because of what? Their track records? Experienced economists have trouble predicting what the global economy will look like 5 years from now but we are supposed to simply accept that the conclusion of the people who are for regulating and/or banning AI are infallible. That makes 0 sense.

At the moment the effect of AI is a known unknown. We do not know what it will do to the economy 5 years from now. Maybe it will destroy entire parts of the global economy, maybe it will create more opportunities in many industries. If we do not know then the best course of action and is to wait and see what happens before jumping the gun.

> It is—already—a moral and cosmological cataclysm

And the superlatives keep on coming...

I stopped reading after that. This article is not serious. We can criticize AI without using terms such as the ones above.

The author also never presents who "the people who oppose AI " are and what they really want. Are they governments? NGOs? Groups or citizens? We don't know.

I don't like the people who preach that AI is going to solve all our problems but the author is going in the complete opposite direction and using the same vibes to support their argument and that really does not help his case.

So...

AI and cannabis bad...

Tobacco and alcohol good...

It's the American Way???

Personally, I think he nailed it with the first paragraph and went downhill from there. If humanity needs an intellectual codpiece to stay relevant, humanity IMO has failed their own Turing test. To truly believe the only two outcomes are we stop AI before it's too late or humanity is replaced by it goes against the past 10,000 years of technological progress (and yes, it came with a body count, we are still poo flinging monkeys with opposable thumbs after all despite all the hubbub about fire, the wheel, and everything else).

The problems with our experiment in democracy are broadly political and not technological. By all means, regulate the deployment of AI and crack down on surveillance to boost privacy and freedom, also known as "wouldn't be prudent" or "not gonna happen" because capitalism and freedom. For example, if I were to suggest that the left and the right do not need to agree on renewables and nuclear, but there's no reason they couldn't agree that coal is bad, that's not gonna happen either. A friend recently stated that most of these problems will not be addressed until enough people have died in the west because of them as he pointed to the recent European heat wave. That's my prediction for everything else that comes to mind, two podcasts from a revolution and all that.

IMO we should work towards AI offloading reasoning and drudgery to enable and assist humans instead of replacing them. Even the best two models out there frequently hallucinate and downright lie to the user, they are essentially Rainman and we need to be Tom Cruise to their Dustin Hoffman for the foreseeable future. But such a viewpoint doesn't hit the engagement numbers of going Yudkowsky light as this article does so we don't get pieces like it or if we did, they would never see enough eyeballs to get traction.

Western corporate socialism is the equivalent of the big bad malevolent AGI we all feared, and we're all still here, we could all do something about it, but my money is sadly still on the opiates of the masses keeping us in check for the near-term. And relying on the opinions of a small cabal of billionaires who likely couldn't bathe themselves without assistance at this point in their antihero's journey to predict what's coming with AI is folly, but it is a cautionary tale to not let that happen in the first place by staying intellectually engaged and mastering the technology instead of the exact opposite that most seem to prefer.

My tradespeople friends continue to do just fine. Good luck building robots that can replace them (Sorry Karl). Too bad they're only 15% or so of the work force. But I don't think anyone gets how hard it's going to be to give LLMs and their ilk the common sense they need to really really really replace 60% of the workforce with white collar jobs. And while self-checkout kiosks do replace cashiers, I don't get why anyone misses them. Ditto for the accidents waiting to happen roaming the aisles of Amazon fulfillment centers. Where is the quiet dignity of such work that must be preserved? Also, by all means give me the robots to work in the mines. See "not gonna happen" above for the same reason we won't be replacing tradespeople any time soon. It's up there in my mind with the smaller cost of giving everyone over 50 in coal early social security to retire the segment down to the point of industrial coal, but see "wouldn't be prudent" above.

And then there's the barb about individualism. Sure, LLMs were trained just like Freakazoid to have the powers of the Internet. But anyone trying to use one straight out of pre-training is gonna have a bad time. There is significant post-training to shape the user experience that can be individualize and fine-tuned to any task for which it will continue to hallucinate and lie because trying to control hyperdimensional silly putty like LLMs is hard, but it's not 100% mean reversion.

In the end, I'm probably as deeply pessimistic about the future as the author of this article, but for completely different reasons. America, the country that went to the moon in 10 years and walked away from global conquest after WWII stands poised to make itself irrelevant and cave to the reckless agenda of pleasing the billionaire class. AI is a sideshow in comparison IMO and we absolutely have the power and capability to deploy it responsibly, it's just that we won't because that would make the billionaires sad.

Hopefully the naked, unabashed, sociopathic greed of the intent to replace people with a system that "outperforms humans at most economically valuable work" will be their downfall.

I'm really disappointed when these blog posts insist on political navelgazing.

There's no political angle worth discussing because none of them touch the ground. We have to look at what we have with sober eyes and ignore the grifters who insist "AGI is coming".

What we have is a copypasta fraud machine. There's no discussion to be had beyond that. It's illegal in most use cases, and a source of implementation uncertainty in all others. No new capabilities have been unlocked from this, nor will ever be. We just found a way to cut cost where the cost was already minimal.

Reply on news.ycombinator.com