Hacker News Reader

Comment by missedthecue | Hacker News Reader

AI is an amazing Rorschach test in this regard. In the way that two people will look at it -- not just AI in general but in a specific implementation of it (coding, vidgen, etc). And one will feel deep dread and knots in their stomach while the other is filled to the brim with hope and optimism.

I think the former group almost certainly outweighs the latter group at least in the Western hemisphere, but I know the Western hemisphere has been steeped in a deep malaise of anti-optimism for several decades preceding AI to begin with. That it's almost a game to identify all the ways anything new might be bad, rather than celebrating the obvious ways they are good. And this penchant invariably biases their developing opinions of all new technology, be it AI or anything else.

View on HN

What if I'm super optimistic about most technology but also scared of AI?

Space, nuclear fusion, advanced fission, the internet since about 1993, blockchain, anti-aging, I've been a big fan of all of it. But I've read the AI safety arguments and seen what a lot of researchers say about it, and I'm seriously worried.

What AI if is the only means for having those in your lifetime

Some of them already exist. I'm optimistic enough to think plain ol' humans can do the others. I'm also fine with narrow AI that doesn't present the same risk as ASI; Google has built such AIs for protein folding and nuclear fusion, and nobody worries that they'll be any sort of problem.

If ASI is the only way then I'm fine with dying of old age without seeing this stuff, rather than risking human extinction. And even if ASI doesn't kill us, I'd rather see humans do it, instead of watching ASI do it while it keeps humans as pets.

We've come a long way all on our own. I don't think we've run out of tricks. It would be a shame if the human era came to an end.

What are you worried about? All the worries seem like bullshit to me. Like we're not going to have some runaway skynet thing, that's just completely off the table. These things run on data centers, we can just turn off the data center if we need to. I don't really see that becoming necessary either but we could.

And then there's fears that someone might use it to engineer a super virus or something like that, which isn't really a new worry. People could do that before too, but it turns out creating a global epidemic isn't really a thing very many people want to do, and it's harder than most people imagine.

"AI" are just chatbots. They can't fabricate robots to take over the world, in fact they have very limited reach into the real world altogether. Sure maybe you could envision some kind of malicious hacking, sabotaging some industrial control system to create a disaster of some kind or whatever but that was a worry before AI too which is why most critical systems like this are air gapped and have safety mechanisms.

So I really just don't understand how people think AI will destroy the world. And I think its especially frustrating to see all this energy going into AI-related despair when climate change actually is on the brink of actually destroying the world in a completely real and concrete way, but most people don't give a shit about that. Some people will say it's natural cycles or some other variant of denying the problem, others will say it's not a big deal, yet others will say science will solve it, we'll capture the carbon or spray aerosols into the atmosphere to stop the warming etc.

And they're all wrong. It's not natural cycles - shocker, climate scientists know about natural cycles. It is a huge deal. We're at +1.5c now and it's already having huge compounding effects. Warming is accelerating, we'll likely be at +3c in just a few decades. That is already approaching apocalyptic levels of warming.

Carbon capture is a joke, there is absolutely no way carbon capture solves climate change. We would need tens or hundreds of thousands of carbon capture facilities all over the globe, it would be the largest engineering project in history by orders of magnitude. There would be many times more capture facilities than power plants. Nevermind the energy, concrete, chemicals etc required to build and run all of them, not to mention actually deal with all that carbon they would be capturing. It's not feasible.

Climate engineering would be highly experimental, and it would also be a gargantuan project requiring enormous economic investment. It would likely have huge unintended consequences, and it might not even work. It might also have different consequences for different nations creating huge political issues - what if a country does climate engineering that's good for them but disastrous for their neighbours? Is it even possible to do climate engineering in a way that isn't bad for someone? Is it even possible/feasible to do it on the scale necessary to actually combat global warming?

Yeah I don't know why people are panicking about AI, if there's anything you should be panicking about it's global warming.

I'm quite concerned about climate as well as our other horrific environmental problems, but there can be more than one giant existential risk at a time.

I'm not concerned about today's AIs. But three years ago they couldn't add numbers accurately and now they're doing world-class mathematics. If it gets much smarter than us, then it will run the world economy, because anyone who doesn't use it to run things will be outcompeted by those who do. At that point we'll have lost control.

The way I see it we already have lost control. Hell, maybe we never had control in the first place. We've known about the effect of greenhouse gases for over a century and we still keep investing into fossil fuels. Because "economics".

If we had control we would have looked at the situation and gone balls deep into nuclear and other green alternatives many decades ago.

Some people certainly have been diving into nuclear & green for decades, and we now have far more wind, solar, and hydro power than we did several decades ago. Nuclear has been flat for only 2 decades and is about to start growing.

“We” certainly have had control the whole time, “we” have been consciously creating greenhouse gas emitting machines while knowing the consequences. It is the economics. The delay in environmental consequences combined with the immediate financial benefits of easy power & transportation made the choice too tempting for some.

The kind of control you’re talking about is self-control. We’ve always had the option to stop our bad behavior, and we chose not to. And we still have the option to exercise self-control with AI. The question is whether that option might disappear soon. Can AI get good enough that we will actually lose the choice to regulate ourselves?

> What are you worried about? All the worries seem like bullshit to me. Like we're not going to have some runaway skynet thing, that's just completely off the table. These things run on data centers, we can just turn off the data center if we need to. I don't really see that becoming necessary either but we could.

We can't even turn off the coal plants when the owners specifically want to because it's not making money, in part because the President thinks Global Warming is a hoax: https://www.utilitydive.com/news/washington-earthjustice-sue...

Also, there's a whole bunch of companies pursuing space-based data centres, which I think are dumb for a lot of reasons, but not quite dumb enough to not be a threat in this kind of manner as a few thousand GPUs is entirely doable before anyone decides it's not value for money.

Also, we demonstrably have useless-to-just-bad monitoring in place, which is how all the things on https://www.felonybench.com happened. That the companies could have done better, and should have done better, does not change the reality that they in fact did not notice until after the events.

Also, there's at least two companies putting these things in control of wet labs. Good news: they're currently too bad at this to pose a threat. Bad news: "currently" is doing the heavy lifting.

Also, there's a lot of people who are blindly using agentic AI and not checking anything, despite all the advice and case studies about why this is a bad idea and to not do that.

Also, there's a lot of people e.g. on Hacker News whose response to all the attempts to create guard rails on the AI's behaviour is to call this "censorship" or "a lobotomy"; and also a lot of people whose response to calls for regulation of the companies is that this is "regulatory capture". (Worse news: The US president has rejected calls to slow down AI, saying "whoever wins AI, wins").

> And then there's fears that someone might use it to engineer a super virus or something like that, which isn't really a new worry. People could do that before too, but it turns out creating a global epidemic isn't really a thing very many people want to do, and it's harder than most people imagine.

The number of people able to do so has likely gone up by a factor of 10x or so in the last decade. AI's ability to use tools on behalf of their users is a further amplifier on this, unless safeguards are in place. Bad news, "safeguards" is a pretty vague term, and vague platitudes are not enough.

More worrying is the possibility that some idiot will set biology challenges similar to the cybersecurity challenges which led to HuggingFace, with similar lack of oversight, and the agents will do much the same thing but with DNA/RNA-on-demand services. I really hope their equipment is better secured than HuggingFace was.

Good news: the agents in the HuggingFace incident did not act with what would be called destructive "intent" if a human did it. Bad news: they still did the things.

> "AI" are just chatbots. They can't fabricate robots to take over the world, in fact they have very limited reach into the real world altogether.

No, they're not "just" chatbots. They're also tool users that can see.

Good news: yes, they can't fabricate robots. Bad news: Guess what Musk is trying to do. Good news: Musk's timelines are wildly overoptimistic.

> Sure maybe you could envision some kind of malicious hacking, sabotaging some industrial control system to create a disaster of some kind or whatever but that was a worry before AI too which is why most critical systems like this are air gapped and have safety mechanisms.

Bad news: We're really bad at properly air-gapping stuff. We only know about Stuxnet because it breached an air-gap. This kind of thing happens a lot, even for really important stuff.

> So I really just don't understand how people think AI will destroy the world.

Ultimately, all it needs for this is to do what it was told, instead of what was meant.

All humans acting perfectly would prevent all the bad outcomes I can think of, but (1) that never happens, and (2) "that I can think of" is doing a lot of heavy lifting there.

> Yeah I don't know why people are panicking about AI, if there's anything you should be panicking about it's global warming.

People can care about more than one thing.

Look, you're worried about climate change: AI appears to be quite effective at convincing people of stuff, just consider how bad it would be if all the people who get rich from greenhouse gas emissions outspend environmentalists on AI-driven PR/marketing.

> "we can just turn off the data center if we need to"

1. If it never happens or AGI is friendly then we won't need to. If a hostile AGI is created, one trained on all writing about war and strategy and deception, able to respond in microseconds, with has ample time to plot before it makes an obvious move, then thinking you will wait until you see it attack and then turn off a datacenter, is way too little, too late. Like saying "the nuclear bomb explosion is local to one place, if it hurts, we will just walk away from it".

2. That claim does not hold water once you start to dig into who, specifically, would turn off which specific billion-dollar datacenter, under which conditions. Google has been moving compute jobs around the planet to get nighttime electricity rates for bulk compute jobs for years, the idea that there is "one datacenter" is far too simple. The idea that employee 74,123 at Meta doing AI research is also able to get a Meta datacenter shutdown quickly on request, or would know enough about Meta's infrastructure to know which datacenter, seems unlikely. The idea that it even has one off switch, or that nobody will quickly power it back on, or that the compute jobs will not pick up automatically somewhere else, or that nobody else on the planet will be at a similar point in their research, also seem important.

3. This assumes there will be a simple critical point that makes it clear we need to "turn off datacenter now". The fact that you then go on to talk about climate change, imagine you had said "we can just turn the factories off if climate change becomes a problem" and maybe that will make it clearer. Who will turn which factories off, at what point in the climate catastrophe, and why haven't they? That's why it won't happen with datacenters either.

> "Like we're not going to have some runaway skynet thing, that's just completely off the table."

You're confident that tens of billions of dollars of investment and research, US v China national AI competition, some of the largest companies in the world researching AGI will be a flop and develop nothing? Despite the progress from GPT2 to present day, despite historic analogies like the Wright Brothers to the Moon Landing in one lifetime?

NVidia shipped an estimated 40 million GPUs in 2025. Say they average 1 TFLOPS each and a year has 31 million seconds, that's 1.2 million million quadrillion calculations in a year. That much compute power piling up around the planet year-on-year is not going to be important or relevant to the near-medium term future? Because it's just "off the table" that anyone could invent anything interesting with that much growing compute in the next ten or twenty years?

They get electricity through cables. You nick the cable, end of story for the datacenter. Dataceters in space, welll.... not a good idea. When AI systems gets hooked to humans via brain interfaces and human like robots, well we'd better start worrying.

That doesn’t address any of the points I raised about who would make the decision, or about compute jobs moving to another data centre, or about a dangerous AI deceiving that it was stupid until it had a plan to account for that, or an AGI responding much faster than humans, but it raises new points - how do you “nick” a 20 MegaWatt power feed without dying, and what do you do about the batteries and generator backup in a secured area?

Now are you imagining that the employee who can’t ask for a data centre to be shut down, can call the military and telling them a self-improving AGI is about to destroy the world and they need to attack an American company?

>one will feel deep dread and knots in their stomach while the other is filled to the brim with hope and optimism.

Ignorance is bliss

This is not without reason, since living standards have been steadily declining since 1971, and every new technology has been explored by the rich to ensure that living standards decline faster.

> since living standards have been steadily declining since 1971

There are living standards issues; one of the big ones is a map-is-not-the-territory problem: people treat "living standards" as if it's a single quantity.

It's really a whole bunch of different things coming together, with different people caring more or less about different things: some of us tie the idea of "living standards" to overall GDP or income per capita; others to house prices, cost of medical care, can they afford to start a family, etc., none of these options are entirely right nor are any entirely wrong.

For example:

I'm now living in a house so well insulated that when we accidentally switched the heating off it took a week to notice even though it was snowing outside; the air is clean everywhere (except on new year's night when the fireworks are ridiculous); there are cures for an increasing number of previously untreatable diseases.

When I was a kid, the wooden window frames would rot from the condensation in winter; the petrol contained lead, and indoor social areas permitted smoking; HIV was a death sentence.

By many measures, my living standards are way up. And yet, no kids to share this all with.

You can make rough comparisons though. I'm sure your house being 20 times more expensive more than makes up for it being well insulated.

The ratio of wages to house prices did not change 20x, more like 1.6x. Though that will vary depending on where you live, of course, and also when your childhood was.

And yes, I think that price gap can be justified entirely by the insulation, when I look at energy costs per square meter in old buildings, available to me thanks to European requirement for energy performance certification. And that's even before PV so cheap it pays for itself in a year, which I have on my driveway right now.

The childhood home had single glazing, limited loft insulation, the cavity wall was just a cavity and had no insulation filling it, gas central heating. It was cold even with the heat on in winter.

So, I effectively spent just as much overall as my parents did, wage adjusted, but I'm much warmer for it.

Basically, I've gone from a childhood in a relatively big place that today would be considered "in need of modernisation" even at its best if it were listed for sale today, to a place which is about average size but of a quality far in excess of what anyone would have bothered to attempt for a normal family home when I was a kid, not even in a show home.

(And this was the cheapest new build I could find).

living standards have been steadily declining since 1971

This is generally false, and in areas where it's arguably true the problem is usually terrible policies, e.g. forbidding people from building housing and then being shocked when homes are unaffordable. On the other hand technology has improved living standards substantially; in 1971 less than 20% of US households had a dishwasher and 30% had air conditioning.

AI would be much easier to be optimistic about if it were in the hands of people I like and trust. Its in the hands of people who appear to be complete psychopaths instead.

On the other hand I'm amazed at how flat access is. Access to the best models available is incredibly inexpensive relative to technology in the past, and everyone in the world with at least a smartphone has access to an LLM of some sort now.

Second, the technology is very open. It's not as if the ONLY way to access an LLM is to pay a toll to Sam Altman. Models that are very good can run on my Macbook for the price of electricity and they're only getting better.

Having been involved with development of Internet social media and smart phones it seems like balancing futurist hopes with guarded pessimism might be an enlightened path.

Sure if you work in anthropic / openai or several connected / similar places future is exciting. And that's about it for the list, over-represented on HN but not really out there in real world.

Literally everybody else, all white collars including, feels they could be replaced in 5-10 years. Just talk to people ffs, its not that hard. Literally everybody is worried, doctors (know plenty from all parts of medical care), lawyers, bankers, tons of various bureaucrats, ALL are afraid in one way or another. Its also sort of helpless worry you have 0 control of, not very nice place to be 24/7 in the back of one's mind. Double that for parents with non-teen kids.

I use nature to relax and recharge and it works wonders. But reality is as it is and I have 2 small kids, I don't expect to hold the job I have in next 5 years, maybe due to llms maybe not. But I will have big issues finding another job on over-saturated market at that point, thats completely on llms. Thus as any sane person, hating the tool destroying my/our future while using it from time to time (in minimal fashion, code churning is really not a bottleneck in any decently large company and certainly not mine but I can imagine code sweatshops doing various cruds and eshops all day can be much faster now).

Reply on news.ycombinator.com