This oft-repeated idea seems clearly absurd to me. Even though I personally think concerns about AGI killing everyone are baseless, the company was in fact founded on those concerns and its top folks have been consistently concerned about it for a long time, way before they sort of randomly blew up and became a tech giant with a massive product.
It seems exceedingly clear that they have been concerned about "oh so dangerous AI" for many years before they had anything to hype.
Sibling replies are arguing that they're right to be concerned. But there's no need to be concerned yourself in order to see that they probably, in fact, are.
It's clear they want to make boatloads of money. It's also clear they operated as a research lab for 7 years before they made a single product and have talking about AI dangers since 2015.
To OpenAI's credit they still haven't taken down their original statement of founding [1] : "Because of AI’s surprising history, it’s hard to predict when human-level AI might come within reach. When it does, it’ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest.
We’re hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We’ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies."
All the stuff about going closed source, fiduciary duty, competing, avoiding "undue" concentration of power, etc for 'safety' came in their new charter in 2019 [2] after they created a for profit entity, partnered with Microsoft, parted ways with Musk, and so on.
It is also about reducing competitors. Like Nuclear Weapons, the first one rules the world. After you had them, you made sure that nobody else will get them. Or at least they did their best.
Except for the people saying this who are among the most expert in the world on this topic. But obviously it's fruitless even making this argument to you. If they're an expert making this argument, it's marketing. If they're not, then they're clueless.
If that Catch-22 doesn't set off alarm bells regarding your own thinking on the topic, you are quite literally beyond reasoning.
The same "experts" on AI couldn't foresee what would happen when they decided to suddenly fire their CEO so it's not entirely reasonable to expect them to have some magic ability to tell the future (particularly when all of their messaging on any given product is specifically designed to entice investors).
Another great example of faulty alarm bells.
I'm all for approaching these claims with skepticism, but that's not what you're doing here. You are categorically discarding claims you already disagree with.
> 100% of the people making wild claims about AI risks are worthless scammers and grifters with no scientific backing.
I mean this is demonstrably untrue: Sutskever, Suleyman, Hinton, Russell to name a few.
I have a serious trust issue with expert opinions based on beliefs, extrapolation, and gut feelings, instead of facts. Especially when they have a vested interest to ignore immediate issues at hand (privacy, power concentration etc) and tend to focus on hypotheticals that are supposed to come after higher order effects come into play. And especially when these opinions enter a feedback loop with an uncanny resemblance to a religion. Experts can convince themselves of anything. (source: being one in my area)
Nobody - nobody - has facts about the future. Don't kid yourself into thinking you're doing more than extrapolation and gut-feeling.
Of course. Gut feeling and some basic experience on how people function is almost the only thing I have here. However, I'm also not the one making those predictions.
OpenAI's charter [1], for example, is so insanely naive and idealistic, and at the same time doesn't seem to be just an empty marketing piece (they expressed the same ideas for many years). Whoever wrote this thing is clearly either has absolutely no clue on how the society functions, or is trying to hide malicious intentions behind the bullshit. In either case, I have a really hard time believing that someone who wrote this can make useful predictions.
Do you believe AI is a species level threat? What would you suggest we (gov't/society/etc) do about it?
It honestly amazes me the way that rationalists have reinvented every aspect of religion. They have scripture, a prophet, an apocalypse, and even a god who will torture you for not doing its will. And to cap it all off, of course, their own non-profit dedicated to preventing the apocalypse. Donate now!
https://slatestarcodex.com/2015/03/25/is-everything-a-religi...
Rationalism does not require that kind of distortion. The parallels are strikingly obvious; I don't have to torture Yudkowsky into a prophet, or the Sequences into scripture. Yud literally predicts the future and tells you to give him money to make it better. When rationalists write litanies and gather for solstice celebrations about how great rationality is, I'm not sure comparing them to a religion requires quite that stretch.
Or, to take a more conciliatory tone: Maybe he's right! But either way there's probably a spectrum, and rationalism is way closer to being a religion than, e.g. fans of the New England Patriots--who can only have a minor apocalypse on an annual basis, and lack scripture entirely--and further away from it than Scientologists.
There were experts calling out safety concerns. But they largely weren’t the ones parading them. This was made obvious when those calling regulation started seeing draft regulation and then threw their toys out of their pram (Altman).
I’m making a stronger claim. Practically everyone making public statements was doing so dishonestly. When push came to shove, none of them had workable ideas beyond a pause.
Well, it took a while to figure out asbestos is too deadly for humans to stick it into every wall. To really see if AI is dangerous I suspect we have to stick it in every place possible and find out, which is kind of sad.