Hacker News Reader

Comment by Avshalom | Hacker News Reader

>AI had a 1-in-10 chance of shutting down all power generation on earth.

Here's the thing: I do actually kind of believe that. Not because I think that AI will be super intelligent and will do so out it's own volition. I think that despite AI being barely functional people will put it in charge of power generation to save a buck and it will just fail.

View on HN

You mean an industry with so much inertia and hesitancy they still run mission critical systems on tech from the early 2000s?

You think they’re dumb enough to risk throwing barely functional AI in there on the chance it’ll save a few bucks when it’s overwhelmingly more likely to cost 100x more in failures and repairs?

I'm not specifically worried about power generation I am going along with the example.

I do think that that level of shortsightedness is absolutely likely in any industry, any company.

I do, yes. The thing is that the power company as a whole is not a decision maker. Rather, individual people with short-term horizons are. If you can save the company some money, make a bunch of money yourself that way, and then get out, you'll have done well for yourself and you can ignore the consequences, or you can convince yourself that you'll be able to ignore the consequences, and that will be enough for plenty of people to want to do it.

>You think they’re dumb enough to risk throwing barely functional AI in there on the chance it’ll save a few bucks when it’s overwhelmingly more likely to cost 100x more in failures and repairs?

Now that you say it that way....

...Enron....

...Boeing....

Their technology conservatism is either because they want to use old reliable tech, or because they're unable to do hard things. If it's the latter then we have a problem.

GPT4 has been a great teacher, I love it, but I think a lot of people and industries are already "faking it" and they'll be happy to turn any decision over to AI as long as it makes them look good in the short term. We won't see this from the outside either, like the lawyer who let GPT secretly do his work, others will do the same, but won't get caught like the lawyer did.

This is the same industry that's literally burning down the state with the world's 4th largest economy.

Due in large part to their laziness with equipment... which is to say, they'd probably take up AI if they thought it could save them on the salaries of the people who might tell them they equipment should have been replaced 30 years ago.

Already in the works. Two recent examples:

https://developer.nvidia.com/blog/exelon-uses-synthetic-data...

https://stanton.house.gov/2022/9/house-passes-stanton-s-dron...

> Here's the thing: I do actually kind of believe that.

So... what's your answer to the question then? Are you just assuming we are likely to be annihilated and chill with that?

unions and liability reform.

Honestly, sounds good to me.

Reply on news.ycombinator.com