OpenAI was working on advanced model so powerful it alarmed staff | Hacker News Reader

OpenAI was working on advanced model so powerful it alarmed staff

theguardian.com

55 pointsfergie46 comments

This sounds like some fabulist horse manure or maybe just marketroid speak. The only way to "risk the future of humanity" is to intentionally build in something that makes it dangerous. Anything less and it's just an expensive and somewhat alarming toy.

Besides that, if it has an off switch, it's not nearly the danger to humanity that one might claim it is.

Discussed yesterday https://news.ycombinator.com/item?id=38386487

This sounds exactly like PR to paper over the mess of the last few days. Not a bad thing necessarily, as it sounds like some adults are back in charge instead of the teen drama we've been seeing.

Any speculation on what Q* might refer to? What comes to mind are A* and QKV. Perhaps some way to more efficiently calculate QK using some kind of differentiable search.

maybe something with Q learning.

Qanon is going to have a field day with this :D

Can't gpt4 already do "basic math"?

It can't,as it relies on external plugins to make it work.

External plugins on the backend that are hidden from users? How does it work? Something like an api call to wolfram or is it another self hosted model?

When you get high on your own supply.

Been following the story, mostly on twitter don't know if any of this is true but at this point it is just pure entertainment reading some of the speculative theories:

https://twitter.com/YourFirstAI/status/1727661862781612215

If Q* is what I think it is… this is absolutely BONKERS

Q-learning has existed for decades already. It’s just a basic reinforcement learning algorithm. A* is also fairly old- it’s a heuristic-based path finding algorithm.

In typical engineering fashion, they may have found an intersection of the 2 and named it Q*. This is total speculation, but if this is a “breakthrough” that means OAI built an algorithm that can feed a highly efficient heuristic into Q-learning. That is MASSIVE.

to tone down your enthusiasm a bit, heuristic tree search and q learning were already mixed years ago. Thats what Alphago and Alphazero were.

Which Ilya was involved with...

Every article I come across on AI in the BBC or The Guardian seems to be a sensationalized scare story about AI taking over, compromising our data, or other alarming scenarios. They often quote someone they've dubbed the "Godfather of AI" to lend credibility to what feels like subpar journalism. I struggle to recall any BBC or Guardian piece that focuses on the positive potential of AI in fields like the creative arts, science, technology, and healthcare. It's consistently skewed towards negativity.

Negative articles get sometimes as much as 1000X the clicks of positive articles. When your business model is delivering ads to eyeballs, you want to maximize negativity.

Works in social media too. Social platforms promote the most divisive, triggering, or negative content to maximize engagement.

The Guardian does not use an ad supported model.

> Every article I come across on AI in the BBC or The Guardian seems to be a sensationalized scare story about AI taking over, compromising our data, or other alarming scenarios.

One thing all these have in common is being in, and by, the press:

https://quoteinvestigator.com/2017/11/30/salary/

Is this entire article based on the Reuters news that came out yesterday? The news that amounted to "Two guys in the alley told us some stuff, but we can't verify any of it" This feels like laundering news; all the "We weren't able to confirm the truth of these statements"' have been removed.

https://www.reuters.com/technology/sam-altmans-ouster-openai...

It's a great headline that feeds right in to the rest of the OpenAI drama, which is undoubtedly why it's all over the place. But I'm with you - where are the details?

We don’t have very much journalism anymore. We have click bait sites that reprint stuff they got from social media and comment sections.

It’s funny to talk about details because the entire OpenAI staff used vague media reporting to get their way when Altman was ousted and yet no one really has any concrete details about that. So nice they can cause a huge frenzy and then walk away in silence after it’s over because they got a payday. Suddenly any information is none of our business.

From the Reuters article you linked, the only hit from "ctrl+f confirm" now says:

> Researchers have also flagged work by an "AI scientist" team, the existence of which multiple sources confirmed.

Maybe it firmed up in the 6h-ago update, and so other sources started publishing more widely? I'm realizing I need to get in the habit of re-reading source materials, instead of going off memory on an evolving story like this :)

Yeah, the article is like 3 times longer than it was yesterday around 6pm ET when it was posted. It was barely 5 paragraphs then.

> It was barely 5 paragraphs then.

It has been a very bad trend on mass media to not properly inform about the different versions of the same article. Sometimes their entire narrative changes with no indication of what the article originally was about.

They've confirmed that there is an "AI scientist team" at OpenAI??? Sounds like a lot of journalism happened in between those updates.

I'm getting tired of these news outlets regurgitating stories. In this case all these other outlets are regurgitating that one article from Reuters.

My Gell-Mann Amnesia Effect [1] radar is beginning to make worrisome beeping sounds, also. I've seen so much far-reaching speculation over the last few days, I think the best thing I can do is intentionally focus on more interesting things (just learned about Lagrange Points [2] for example) and stop feeding the OpenAI hype machine.

[1] https://www.epsilontheory.com/gell-mann-amnesia/

[2] https://science.nasa.gov/resource/what-is-a-lagrange-point/

Gotta keep yourself in the hype cycle OpenAI. Good job.

AI under copyright and closed source is the biggest threat; all this bulshitting about AI risk is absolutely ridiculous.

Please enlighten me. How do you know that?

AI is just a tool; there is ultimately nothing special about it—it simply makes our lives a bit easier. As with any tool, it can be used for good or evil, and it's very hard to separate the two. In fact, the distinction is often arbitrary and, ultimately, useless.

The main problem arises when a technology is proprietary, benefiting only a portion of the people. This is literally the only present (and real) problem with AI; all the rest is just scaremongering.

AI is indeed powerful, but if it's controlled by a tiny elite instead of being accessible to everybody, problems and risks arise. The same is true for every piece of technology.

Let me give a short list of technologies I wouldn't want accessible to everybody:

- Thermonuclear warheads

- Designer biological viruses and bacteria

- Tools to remotely compromise / brick all computers in the world

... and so on.

We don't really know where AI is on the danger scale, and if it's one of the scarier corners, we might not know before it's too late.

What this episode did reveal is that whatever checks-and-balances OpenAI has in place are not adequate. It almost gave it's whole business to Microsoft. The board accused the CEO of being less than candid, while it itself was less than candid.

OpenAI seriously needs serious, enforceable frameworks for things like transparent governance. The level of opacity here is not okay. At this level, we should all know what's happening inside (whether or not we agree that models can be dangerous, or have access to those models).

The first two, at least, are already within reach of anyone with sufficient budget.

I doubt that there's any entity that's rich enough and wants nukes that doesn't already have them. Even international pariahs like North Korea have them.

AFA the second one goes, high school kids are doing genetic engineering in class these days. There's nothing in principle that distinguishes what they do (e.g., inducing resistance to an antibiotic, because that's easy to test) from something extremely nasty (e.g., making it resistant to all known antibiotics).

Nukes are not within anyone's reach, or instead of teen suicides and school shootings, we'd have nuclear craters in place of some cities. Most of these people:

https://www.forbes.com/billionaires/

Could not acquire a nuke.

The prevalence of genetic engineering is new, is a species-threatening issue. Unless we figure out how to bring these sorts of superweapons under control, we will be extinct as a species within our children's lifetime.

> Nukes are not within anyone's reach, or instead of teen suicides and school shootings, we'd have nuclear craters in place of some cities.

I didn't say "anyone". I said "anyone who's rich enough", which excludes virtually all "teen suicides" and "school shootings".

> Most of these people: https://www.forbes.com/billionaires/ Could not acquire a nuke.

I disagree. Strongly.

If there are people willing to sell nuclear technology to the likes of Kim Jong Un, there are people willing to sell it to the Walton family heirs.

There is something special about current-day AI and absolutely something special about Q if the claims are true.

> The main problem arises when a technology is proprietary, benefiting only a portion of the people. This is literally the only present (and real) problem with AI; all the rest is just scaremongering.

This is just silly, akin to claiming bottled human knowledge and intelligence can't be used to commit huge harm at scale (or atrocities) by randoms. Just the other day we had idiots from 4chan hacking nuclear personnel records and will probably ultimately dox them for the lulz. Imagine how much more effective these adult children (or just actual children) would be at causing actual harm and chaos if they had expertise bottled in the form of AI.

And yes, it's also a problem if the elite have it in their hands. That doesn't mean it's not a problem if it's available to society at large.

Yes and: Technological advances cause social disruption, with winners and losers.

I've adopted author Ted Chiang's viewpoint, paraphrasing: Concerns about AI are really concerns about unconstrained capitalism.

That sounds nice and comforting but it's not true. It's also concerns about enabling shitty (but stupid or ignorant) people to cause harm at scale. Enabling hackers to disrupt essential institutions, enabling warmongers to build WMDs, and yes enabling companies and governments to push down with the boot HARD.

if I have an answer-box that tells me the answers to whatever I want to hear, i'd like to be able to vet whether or not it has biases that were not honestly disclosed.

that is to say that if I bought an Amerurissan made answer-box I would expect that it gives me unbiased answers that were becoming of an answer-box, but if I was unable to know how it worked then the only way that I would be able to determine whether or not the Amerurissan answer-box was giving pro-Amerurissan advice for its' creators own benefit would be through statistical analysis and long-term effort.

If were making the case for making tools do all the work in the world, it's better that the people stay firmly connected to how and why those tools are being made, rather than trusting a singular ideology to get it right the first time for all of humanity.

The opposite of "open" isn't necessarily "closed". It could be "classified".

There was classified public key encryption before there was open public key encryption.

The difference is that classified public key encryption was developed by a public, albeit secretive, government agency. The risk that the OP is worried about is that a private entity would control a powerful AI. If AGI is as powerful as people say it is then it would give a private company an unprecedented amount of power, that is perhaps a greater concern than the dangers of AI itself.

A government agency would have greater means, motive and opportunity to misuse AGI than would a private entity, which is more subject to countervailing influences, including the government. So I would think that a classified government AGI is more dangerous than a proprietary AGI.

But in any event, I think that AGI dangers will depend strongly on how AGIs are connected to systems to gather information and how they are connected to systems to take real world actions on their own.

AI risk is like any other totally unknown risk. Eg. "The risk aliens come and murder us all".

The risk could be huge or tiny - but humans don't have enough information to know, nor really any clues.

Great attempt to pump the commercial entity value before an IPO. Well done.

What IPO?

I misspoke. They are going to have some sort of stock sale so employees can partially cash out.

I'm glad to see that ChatGPT has reached the level of intelligence of a a "spirit board" (Ouija board), which has scared the bejeezus out of kids and adults for decades.

https://en.wikipedia.org/wiki/Ouija

Reply on news.ycombinator.com