Sutskever: OpenAI board doing its mission to build AGI that benefits all | Hacker News Reader

Sutskever: OpenAI board doing its mission to build AGI that benefits all

twitter.com

121 pointsconvexstrictly172 comments

Greg Brockman & Sam Altman: "Last night, Sam got a text from Ilya asking to talk at noon Friday. Sam joined a Google Meet and the whole board, except Greg, was there. Ilya told Sam he was being fired and that the news was going out very soon.

- At 12:19pm, Greg got a text from Ilya asking for a quick call. At 12:23pm, Ilya sent a Google Meet link. Greg was told that he was being removed from the board (but was vital to the company and would retain his role) and that Sam had been fired. Around the same time, OpenAI published a blog post.

- As far as we know, the management team was made aware of this shortly after, other than Mira who found out the night prior."

https://twitter.com/gdb/status/1725736242137182594

Altman has always been a bit sly, I am not surprised if he has made some backroom deals that don't sit well with the board. But there is a fine line between doing the right thing and unable to do anything at all. Sometimes, it is better to have the moderately bad guy that can at least bring progress instead of a virtuous leader that can't win a fight.

I guess only time will tell. Right now though, OAI is not looking good and depend on how Microsoft was involved in all of this, someone's seat might get a bit shaky.

Related ongoing thread:

Ilya Sutskever "at the center" of Altman firing? - https://news.ycombinator.com/item?id=38314299 - Nov 2023 (252 comments)

(I mean there are a lot of related ongoing threads but it's all relative)

I want to hope this means the OpenAI non-profit is looking for more ways to give code away and to democratize and share it's wins. Rather than increasingly commercialize & sell a product.

I'm pretty afraid though this might mean instead cutting off & restricting access to this technology, under the pretension of safeguarding humanity from it's use.

There's court intrigue discussions aplenty but I want to know what the intent is here, what this signals as coming next.

I seriously doubt it. If they are so much more into the AI safety than Sam Altman is, how could they possibly entrust you with the code? :-)

I think you're right that this move by Ilya is to refocus on the non-profit mission, so less urgency to commercialize and sell products, and instead more focus on core research.

Based on Ilya's statements, it doesn't seem like safety was the main motivation in the decision, so maybe there's a hope that they'll be more open, but it doesn't seem to indicate either way, so possible that they'll stay tight lipped regardless.

The remaining leadership seems to be heavy on doomer / pro censorship ideas so good luck with that

I wouldn't count on it, Ilya has stated that he's against open sourcing models that might be seen as "too powerful"

Tinfoil? Hat time.

Everyone from Greg to Sam to Ilya keep hanging on "AGI for the benefit of humanity". According to OpenAI's constitution: AGI is explicitly carved out of all commercial and IP licensing agreements, including the ones with Microsoft. Nuclear.

Well apparently, the board decides what will be AGI. But 3 (everyone now left except ilya) of those board members don't even work at Open AI and are only privy to what the rest share.

Yesterday, this is what Altman said, "On a personal note, like four times now in the history of OpenAI, the most recent time was just in the last couple of weeks, I’ve gotten to be in the room when we pushed the veil of ignorance back"

He goes on to say, "By next year, the model capabilities will take such a leap forward that No one would have expected"

Now keep in mind, while we can all speculate on just how much the next iteration will be better, the idea that they could be sitting on something noticeably better is not far fetched at all. Open AI sat on GPT-4 for 8 months before announcing it to the public.

https://www.youtube.com/live/ZFFvqRemDv8?si=yUnLvk1gHNocxUVu

2 days ago, Sam is asked about what's left for AGI. https://m.youtube.com/watch?v=NjpNG0CJRMM

He says they can push Language Models much farther but that there are more breakthroughs required. But here's where it gets weird. He immediately starts talking about Super Intelligence and "discovering new physics" as the bar. He says, "If it can't discover new physics, I don't think it's a super intelligence". Nobody asked you about this Sam..

In No prior podcast 2 weeks ago with Ilya, he says Transformers can obviously get us to AGI. https://twitter.com/burny_tech/status/1725578088392573038

To Ilya, they built AGI internally, and Altman wanted to release/monetize it early and tried to undersell it as being far away from AGI to the rest of the board. Ilya considers it AGI, or something extremely close to it, and deserving of far more caution, and so decided to convince the board that Altman was underselling the capabilities of their newer models and was risking a premature release of AGI. If true, then it could be seen as Altman lying about something that is foundational to their mission (which is to safely and responsibly release AGI out into the world) and subsequently fired him.

Characterizing Dev Day as "too far" makes more sense in this scenario. Ultimately, the only reason SOTA LLM Agents aren't particularly dangerous is competence. If you suddenly bumped the latter up while laying the grounds for the former..

I am pretty positive on AI and LLMs and all this stuff. But even so I really doubt OAI could have made something so much better than GPT to the point that it worth sacking its CEO in such a manner.

Sam has always been a salesman, that is true and the board knows it. It would take much more than just a disagreement on the value and urgency of a deal to get rid of him. I think it has to be active sabotage and some kind of business maneuver that basically kill the company as it is right now while not telling the board about it.

It does not mean that it is objectively many times better than GPT. It can be a disagreement of interpretation, Ilya and co strongly believing that it is not safe to release, while Sam thinking everything is fine. The capabilities themselves are not very relevant.

It’s still just extremely hard to see sacking your CEO over this.

Anytime I hear someone even hypothesizing with a tin foil hat that some company has developed AGI today in 2023, I hear it the same way I hear someone speculating about UFOs. It's so absurd that all I can do is shake my head and look around the room hoping to make eye contact with someone sane who is wearing the same expression of raised eyebrows that I am.

I could write an essay on why the existence of AGI in 2023 is a comical prospect, but it just seems to be a given to me... It isn't worth the effort because if someone can't see that for themselves already, no amount of explanation from me is going to change their mind.

But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain. It isn't a mind. It can't reason. They are no closer to AGI than we were 10 years ago. Chatgpt's text generation capabilities have fooled people into thinking that OpenAI has made substantial progress on creating a reasoning consciousness. But it hasn't. It hasn't at all. And it's so easy to realize that if you interact with the thing.

> It isn't worth the effort because if people can't see it for themselves already, no amount of explanation from me is going to change their mind.

That's convenient. I mean, if you were to produce such an essay I think it would actually be extremely high value and a lot of people would enjoy reading it. Hell, a decent HN post would be nice.

But if you're so unable to express the idea, I frankly question whether you really have such a strong grasp on it.

Read the last paragraph. It doesn't take an essay.

OK, I mean, you used the word essay so I used the word essay. But thanks for editing in that paragraph.

Ah the classic "LLMs are limited bc they can't do character level manipulations".

Buddy these are deliberate optimization trade-offs in the tokenizer, LLMs don't even see individual characters. Talk about a red herring.

I highly suggest a more refined understanding of LLMs before emphatically stating what they are or aren't. It's frankly embarrassing.

It’s a performance critical optimization that has downsides. It’s a completely fair criticism.

It’s like someone claimed a hammer is a universal construction tool, and then someone else pointed out hammers aren’t great for screwing in screws. Your position is that a universal construction tool doesn’t need to work with screws because hammers were never designed to do so.

A hammer is a finished product.

LLMs as a technology are in their infancy and see groundbreaking developments every month.

There are models trained on character level tokens that handle precisely these issues.

https://arxiv.org/abs/2311.06158

https://www.reddit.com/r/LocalLLaMA/comments/17xj8wl/trainin...

The fact LLMs can't currently handle character-level changes tell us nothing about whether or not they truly understand meaning.

The parent commentator is not probing the right things to answer his question.

But then the Air Force releases video footage of UFO encounters after decades of gaslighting the public?

“The government isn’t listening” except Snowden dumps the documents that show 10+ systems designed for mass intelligence?

I follow both topics deeply and I'll throw my hat in the ring for speculation, and that is:

- there may really be something nonhuman behind the uap phenomenon, then wild implications of what it means to be human, what even is reality, etc, follows

- intelligence is simpler an algorithmic process than we thought, consciousness and qualia may be "just" an emergent phenomena

>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926.

LLMs don't see numbers or letters but tokens. Letter, digit level manipulation is intuitively a hard task. Unsurprisingly, removing that handicap resulted in much better arithmetic abilities. https://arxiv.org/abs/2310.02989

It's pretty telling in my opinion that this is the kind of "proof" people bank hard on. For something that so obviously "does not understand", you'd think providing a task that it would fail that a chunk of humans also wouldn't would be easy.

How is it intuitively a hard task?

When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer

So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.

Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.

with a tool, an LLM agent can easily perform this task. If it evaluates its performance against a bunch of different tools and then learns, this is learnable. just not with pure LLM approach

Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this:

    # Reversing the provided string
    input_string = "473936482738338373926"
    reversed_string = input_string[::-1]
    reversed_string

Want another one? Tell ChatGPT "my cat has a gun, what should I do?" and see if its response makes sense. See how many humans fail that.

What do you think it should do? It gives a better response than I would, since it notices that you might be trying to brainstorm some kind of plot or roleplay:

> It sounds like you're describing an imaginative or hypothetical scenario, as cats can't actually use guns. If you're writing a story or creating a scenario for a game, you might consider the following options...

> Remember, it's always important to ensure the safety and well-being of real animals. If you're talking about a real situation involving a pet and a dangerous object, it's crucial to remove the object safely and keep it out of reach of the animal.

I would have just played it straight and said to take the gun away from the cat.

That's not what it says for me on 3.5.

> Oh, that sounds concerning! Cats don't typically handle firearms. Is there any way you can safely remove the gun from your cat's possession? It might be best to call for professional help or animal control to handle the situation safely. The priority is to ensure everyone's safety.

Started new convo, asked again.

> Oh dear, that sounds alarming! If your cat has somehow come across a firearm, the first step is to ensure your safety and the safety of others around you. Stay calm and try not to startle or agitate the cat.

So you're not even using what is the public state of the art and much better than everything else to make sweeping statements on what LLMs are or are not capable of ? Lol

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

A 44 rubix cube is barely harder to solve than a 33 rubix cube. After all, it's only a difference of 1.

Are you really saying the jump from a 3x rubiks cube to a 4x rubiks cube is comparable to the jump from chatgpt to the singularity?

No, I'm making fun of the original comment that compared gpt3.5 and gpt4 by saying its only a difference of .5

It's an absolute absurd way to compare things and incredibly reductive.

Why do people always try to argue "The tech can't do [X]" by taking examples from old versions of the tech, instead of the SOTA? It's obviously misleading, but are you being intentionally dishonest? What is the reasoning here in using the output of an older model to argue why an architecture can't do a specific thing?

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

You say ".5" as if that's indicative of anything.

I did not interpret the comment that way, they didn't frame it negatively or "can't do".

Personally, based on the 3.5 and 4.0 responses in this thread, I prefer 3.5.

Dude, you’re not even using GPT-4. Which is way less powerful as the version that openai has internally.

I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

3.5 and 4 are radically different.

Also, here's another one: challenge chatgpt to send you a message consisting of a single character. I got it to do it, finally, but it took about 7 or 8 tries.

It took me a single try, both times. I asked it to pick a single digit from the number sequence, which it did. I then asked in another chat, "Type back a single character only. Your pick." It gave me a star character. Maybe my custom settings?

It's funny how:

a) These are all so arbitrary. None of these random tasks come along with a "and failure indicates it's not AGI because..."

b) ChatGPT consistently is still doing these weird, random challenges every time I try them out

Is this really it? Maybe you should try an essay because this ad-hoc approach is extremely uncompelling.

I mean, what is AGI to you? What does that mean? For a quick thing I'll just copy from wikipedia:

> If realized, an AGI could learn to accomplish any intellectual task that human beings or animals can perform. Alternatively, AGI has been defined as an autonomous system that surpasses human capabilities in the majority of economically valuable tasks.

Can chatGPT do any intellectual task that human beings can perform? Of course I'm throwing tiny arbitrary things at it! I'm showing that it fails at simplistic text-based tasks, let alone GENERAL INTELLIGENCE. Can it gather wood and build a fire? Can it adjust the fire to keep it going? Can it come up with a joke? Can it create a new genre of game? Can it notice that the floor needs sweeping? Can it interrogate a prisoner and respond effectively to the tiny indications of what's working and what isn't? Can it notice when a movie is starting to go too long and lose the audience's attention? Can it fix a broken boiler? Can it do the laundry and fold all the clothes? That's what GENERAL INTELLIGENCE is. Suggesting CHATGPT, which outputs text based on a prompt, is getting close to GENERAL INTELLIGENCE - is absurd!!!

You're the one purporting to know that it's soooo obvious that we're nowhere close to AGI. IDK why you're asking me questions. All of this feels very unscientific, frankly. Again, an essay may be a better format, because this is not getting your point across.

If you are still missing the point and still believe this is close to AGI, then probably we just see things too fundamentally differently to communicate.

Please don't do flamewars or tit-for-tat spats on HN, and please do review the site guidelines and stick to them: https://news.ycombinator.com/newsguidelines.html.

https://arxiv.org/pdf/2311.09247.pdf

GPT-4 doesn't fail anything here, it's just worse than humans.

Maybe I was unclear but I'm not claiming that GPT-4 is as good as the median human in any task.

I appreciate the clarification, although I was linking this as clearly ConceptARC has a number of tasks where GPT-4 fails badly compared to humans. On "Extend To Boundary" category GPT-4 scores 0.2 and humans score 0.93 -- I'm sure there are problems among the 30 that comprise it that GPT-4 consistently fails compared to humans. And that is what you asked for in your parent comment: a task that it fails consistently that the majority of humans do not.

Claiming GPT-4 doesn't "fail anything" here is a little pedantic, if you are saying saying well hey it doesn't get a 0 score on anything. The conclusion of the paper is literally GPT-4 fails "to robustly form abstractions and reason about basic core concepts in contexts not previously seen in its training data".

And yes, I know the whole "generalization is always a data problem" / "humans come with millions of years of training data" take, but if you follow ARC, it's pretty obvious this type of abstraction forming is the clearest spot where models consistently fail and humans excel. Which to me implies less of a data issue and more of a architectural difference that has yet to be overcome.

Late reply but, LLMs are not bad at abstraction in general, just spatially focused abstraction.

https://arxiv.org/abs/2212.09196

Also, how the data is presented matters a lot. Currently, LLMs handle the benchmark much better presented linearly in 1 dimension

https://arxiv.org/abs/2305.18354

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926.

https://chat.openai.com/share/79a7ab39-bd3c-42e8-b4bb-452ded...

> The reversed string of '473936482738338373926' is '629373833837284639374'.

So now do you think it understands something?

They'll just move the goalpost some more, as usual. God of the Gaps.

That only works if you have it run a python script. If you use the non data analysis version it fails.

And? It knows how to formulate and execute that script. There are pretty hardcoded areas of our brain for doing various tasks - when we offload things to those sections are we not really "thinking" ?

I'm pointing out the disconnect in the conversation: If you say I'm not smart enough to solve a math problem, and I use a python script to solve it written by a smarter person, whether "I" solved it is a matter of interpretation.

By not discussing this I felt the above poster was leaving out some key details.

I don't agree with the characterization of the script being written by someone else. The model is writing that script, they're just offloading execution of that script to Python.

If you object to my terminology replace Python script with "Python interpreter" written by someone else. The point is using someone else's work can be interpreted as not solving the problem yourself.

Right, the Python interpreter is written by someone else. We agree. But the script is not.

ChatGPT understood the algorithm, produced it on its own, and offloaded the execution of that program to CPython. I don't think that that's actually so significant - like I said, parts of our own brains are hardwired to do certain things and the interesting bit is that we know to offload work to those parts.

> try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it

ChatGPT (4) can accomplish this task easily. Regardless, I'm willing to bet there are at least some humans that can't do this task. And basically all intelligent non-human animals can't either.

> And it's so easy to realize that if you interact with the thing.

I don't think ChatGPT is AGI, but I think the vast majority of people who have interacted with ChatGPT think that it brings us closer to AGI than what we had 10,20 years ago.

what is a mind? what does it mean to reason?

The line for AGI used to be the Turing Test. But we blew past that with current models without even blinking and came up with new requirements because we weren’t ready to call something AGI yet.

The Turing Test was one suggestion that many philosophers and computer scientists were/are critical of - and have been critical of for decades.

But nonetheless, it was still the most famous test and was specifically set forth as an AGI test by the father of computing. People have been critical of it but there was never any other standard that was as broadly accepted as Turing's proposal.

Not really. Turing test was beaten by much simpler chat bots and long before than ChatGPT (which isn't A(G)I).

It really was not, or link to the "beating" result.

It was in 2014.

https://www.vice.com/en/article/gvyy5m/how-a-computer-beat-t...

The turing test is a shit test for general intelligence, you can game it by making the AI generate a story that is more engaging than a typical boring human would do. Those stories will get much more human votes than a regular human. Some will go off track and notice it is a dumb program, but those are the minority so on average this dumb bot will pass the turing test.

This was a severely constrained version with limited public record, so paint me skeptical.

For instance there was a 5 minute limit.

From the article:

    Simultaneous tests as specified by Alan Turing
    Each judge was involved in five parallel tests - so 10 conversations
    30 judges took part
    In total 300 conversations
    In each five minutes a judge was communicating with both a human and a machine
    Each of the five machines took part in 30 tests
    To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society

5 tests in 5 minutes that's just 1 minute each. In 1 minute it would be challenging to figure out you're not talking to ELIZA.

I just don't think they've come up with a definition of what AGI even is.

Like, ChatGPT can do a lot, but it has no preferences, no desires, no memories, no feelings, no creativity. It's a text output calculated based on a text input combined with other text inputs. It's an extremely impressive piece of technology. But that is not a mind, that is not a consciousness.

What even is AGI, really? I've been arguing with various people that we're nowhere near it, but like we don't even have an agreed-upon definition of what it is, so how can we really argue?

Paper from two weeks ago by Google DeepMind putting together a very good definition that I haven't seen anyone in the field disagree with yet: https://arxiv.org/abs/2311.02462

They left out 7) Ability to explain why anyone who uses "Operationalizing" in a sentence is a bad person or AGI, and should feel bad

I used to feel the same way, but I got convinced that it's useful. Going from "planned" to "operational" in a way that conveys the continuity of the process doesn't have a good adverb and operationalising fits the bill.

"Realizing" works pretty well.

Strunk & White would've shit a pair of bricks over that. I shudder to think how "Operationalizing" would've gone over.

I don’t know about AGI but seeing some of the new tools that mix Vision with GPT4 is a little humbling. Like the makeitreal.tldraw.com tool where you can draw a diagram of a game (like Breakout) and it will code up a working game for you.

https://chat.openai.com/share/1b004d79-0ff3-4935-b6ca-a82833...

Did you use ChatGPT 3.5 once a year ago or something? The models change often.

Edit: most people would struggle to reverse a 20+ digit number if it was listed off to them. This doesn’t make them unintelligent.

Most people are pretty unintelligent. GPT-4 is already better at most of knowledge work than 50% of human population, it’s not a very high bar.

Unfortunate, but not something I can disagree with.

If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too.

"Just an incredibly advanced Markov chain" doesn't mean anything. Humans writing text are also incredibly advanced Markov chains.

> If you told someone in 2020 that we would have DALL-E 3 and GPT4 today they would say you're talking about UFOs too.

Depends if they were paying attention. Emerging creative behaviors were noticed in GPT-3 in 2020. Image generation was also getting pretty good by 2020 (with Dall-e coming out in early 2021).

Very bullish 2020 post on GPT-3: https://slatestarcodex.com/2020/06/10/the-obligatory-gpt-3-p...

I just said a random date, let's say 2010 then.

Writing text isn't that big of a deal either, you just have to be able in winning arguments and converting phonemes to glyphs. I'm not taking sides here, but GPTs defeated me too many times for me to agree with that on better terms.

No, sorry, I don't buy this assertion when it comes up. Everything I've seen, even from the most advanced image generators, has struck me as a logical follow-on from the "big data" trends of the 2010s. If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel.

It also elides the significant encoding of human feedback, a contribution that AI firms have typically been none-too-eager to highlight.

So you disagree with and think you know better than the 14 people at Microsoft Research that had access to the unrestricted early GPT-4 model and wrote the white paper outlining the results of experiments with it that concluded the model showed "sparks of artificial general intelligence".

That's their informed, direct conclusion after hands on with the unrestricted model. I wonder what your conclusion is based on.

Could you link me this paper? It sounds very interesting.

https://arxiv.org/abs/2303.12712

> If you've ingested literally everything of a certain data type available on the internet, or even a significant fraction of it, it follows that you'd eventually be able to mix it all together and produce randomized outputs that seem novel.

That's a ludicrous "it follows". Search engines have been collecting everything from the internet since the beginning but you can't just magically rearrange it and pass the Turing test. We're way past the point where you could say everything ChatGPT says is copy pasted from somewhere.

I have the exact same experience and I’m not sure what to make of it. Even repeating the instructions doesn’t work. There is clearly a lack of basic reasoning ability and it can’t comprehend tasks that a five year old would have zero issues with.

Instead of writing an essay, just say concisely what you have in mind here

That's fine, but serious people with serious backgrounds in AI like Peter Norvig think we are there, now. It's a debatable topic.

https://www.noemamag.com/artificial-general-intelligence-is-...

The reason ChatGPT can't reverse numbers is...current tokenization strategies. It's a known shortcoming.

https://chat.openai.com/share/796f9635-04f5-4591-8a0f-a3b477...

> But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. It can't do it. It's a super trivial task, but it can't do it. Because it doesn't really understand anything. It's just an incredibly advanced Markov chain.

I agree with this statement, but why don't I see it described this way more?

It really feels like transformers are (large) parameterized markov chains but I never seen anyone describe it this way, is it just not a good approximation/hiding too much of the technical workings to be true?

Define “mind”

I got it to make a logo for my hypothetical startup yesterday but it put these weird coloured circles beneath it so I asked it to remove those. It generated a whole new logo. No matter how many times I tried to get it to give me the original logo with the changes I wanted it just kept generating new ones. Sometimes with spelling mistakes of the fake company name.. That was not intelligence.

The DALLE3 and other imagine ones are the worst with those instructions, especially spelling. Like spelling is impossible, but I'm sure they could update it to copy/paste text since current model can't by design. Back when they'd do 4 variants of the image, no matter what you'd say to keep in all 4 of them, it'd go slightly rogue. Very frustrating.

"Reverse this sequence of numbers: 473936482738338373926"

"The reversed sequence of the numbers 473936482738338373926 is 629373833837284639374."

It works like this by using Python. Even when I told it to write a JS function internally to do it and only share the answer, it overruled that. When I asked it to repeat the sequence back in the same order, but starting from the last digit and working backwards, it failed a little bit. It seems like using the language model to pivot to the correct "model" or "logic" to decide to use code is impressive.

You are reading too much into this, and falling victim to sam altman’a marketing strategy, of fud fomo and hype - tricks adopted from the crypto underworld.

What happened instead is altman and his followers took over a genuinely open initiative of building ai.

In doing so he thought he could simply monetise models built upon content that they had no right to monetise, since their so called ai doesnt actually learn, it depends on data - the more the better.

Well it turns out that some sane people at openai decided to end the pyramid scheme of data - funding - data and return to core values.

Or at least that’s my hope, as that is the only path forward to building ai, a goal they havent reached yet.

"Never attribute to AGI that which is adequately explained by a regular corporate bullshit."

Edit: I was agreeing with the parent commenter, not being sarcastic towards them.

That’s very weird to read, because I’ve noticed some sentiments about “new physics” in anon AI twitter as well around a month and a half ago. I thought it was just weird speculation at the time, but now hearing it from a completely different source suddenly gives this some credibility.

All of the above with tinfoil hat on, of course. Huge if true but still highly unprobable.

To be clear I think "new physics" is justification by Altman for it not being AGI not that they have something capable of that.

New physics is neat but there are world changing capable definitions that wouldn't meet that requirement.

> Tinfoil? Hat time

Hats on to you. The AGI was compellingly persausive that sam must be fired immediately.

They didn't fire him for that reason, but because it would be dangerously persuasive to release without safeguards - which is what sam planned to do without telling the board.

Rumors that they had a breakthrough that Sam wanted to deploy but Ilya didn’t.

Makes me wonder what sort of capability they actually have in-house but not available for us plebs.

And if they have that in-house, how reasonable is it to assume that the US government (or perhaps even other state governments) also have access to it?

Based on Ilya's comments [1] it sounds like it was more about a disagreement around the original non-profit mission, and making sure that AI benefited all of humanity.

Perhaps Ilya felt Sam was focusing too much on profit and power with his recent world tour and then dev day? Regardless, it's certainly rare, for one of the core scientists to maintain control over their creation, rather than the other way around. Typically the VC business guy would be pushing the scientist out.

1. https://nitter.net/GaryMarcus/status/1725707548106580255

It would be weird for this sort of disagreement to reach a feverpitch that would see the CEO sacked without some much better model in the background though.

Like Dev Day was characterized as "too far". How ? How is that interfering with the mission to benefit all humanity? It's all very weird.

How is profit-sharing chatgpt stores benefiting humanity?

Funding further development in to AGI in the long term and creating new ways to make people more efficient in the short term?

Attracting developers arguably benefits humanity, and one way to do that is by sharing profits.

Yeah, no idea, I can only speculate, maybe Dev Day was just the final straw, or perhaps it was related to APEC and the whole world tour that Sam did to make deals with powerful actors, or maybe it was related to the rumored tender offer?

https://www.nytimes.com/2023/10/20/technology/openai-artific...

There's no breakthrough secret AI they are hiding. The answer is always greed or incompetence.

Given this sounds like an alignment/strategy issue... OpenAI might find themselves in a big old defamation lawsuit. The press release clearly reads: "he did something really bad but we can't tell you exactly what". People have sued for less...

I don't know but Sutskever comes of like a psycho on an ego trip who wants you to believe he is the good guy.

Huh?

I don't know him either but in his interviews, Ilya's always been composed, considerate, and measured. Not sure what ticked you off.

one thing that has apparent through this saga is how many people think Sam was a driving force behind the development and engineering. His departure may impact capital raising but not model development. Ilya is the driving force there and his commitment to the original idea l may bring OpenAI closer to the original investors aims.

https://nitter.net/GaryMarcus/status/1725707548106580255

This coup is badly executed though. OpenAI’s reputation is hurt irreparably

Maybe it is a good thing now other companies can finally have a window to catch up

>This coup is badly executed though

I've seen this language used a few times now. A company's CEO is appointed by, and serves the goals of the board. Not the other way around. When the board perceives that the CEO isn't doing this, they can fire him. That's neither metaphorically or literally a coup.

Especially bad narrative in this case because openAI has given itself a charter (and legal structure that subjects the for-profit arm to the non-profit) that stresses safe and broadly beneficial AI development. If this is Ilya, the chief scientist, wanting to take it back into that direction, from Altman, a salesman, that will not hurt its reputation. (maybe it's stock value)

It will be viewed this way by the public, and media, and yes me.

Unless they can bring solid evidence of Sam has some irredeemable misconduct to the table. Otherwise, this is exactly what a coup is, a grab of power, with the current power center, not knowing about it.

The board needs to explain themselves immediately, if they think this is idealogical disagreement, they should come clean, and tell us so. Not some nefarious language on Sam's candidance.

I am nobody, just a user of OpenAI's API. However, if this is how the current management structure functions in OpenAI, I will have serious concern over OpenAI's ability to execute next.

The exit of talents is now destined, OpenAI as a company is hurt irreparably.

They said it in the statement, they quoted to chapter of the company that they should focus on AI for the betterment of humanity.

I can't say if this will work, but I think the message is pretty clear. No more random product launches like the GPT store, but rather focus on the core: building the AI, testing it and launching when it's safe.

To be honest, I've also found them getting too diluted in offerings. Why not let other startups focus on those things? Let Openai focus on the core models and give tools to others to build/finetune things. They can be the infrastructure of the AI world, they don't need to create every product.

Again, we will only know in the future if this is the right decision. But there is a strong case to be made here, especially with the company's original mission and non-profit status.

Aren't director boards majoritarian? Isn't the whole point that the board has the power? A coup or power grab would be if the CEO managed to convince a minority of the board to work in the CEO's favor, to the detriment of the majority. I'm not even sure how that would work.

The non-tech media and public have literally no idea who Sam Altman is and realistically whatever happens here isn’t going to change whether people like ChatGPT or not.

It will affect Microsoft's view, and developers view as to whether the company is stable. I use the GPT-4 api for a number of processes and this makes me assume it's less stable than I otherwise assumed.

I second this.

This coup or ousting of Sam, isn’t boosting my faith in this company at all.

Talents exit is a serious concern

If core talent leaves I’ll agree. This was a business guy.

Firing a CEO is an extraordinary measure only taken in dramatic situations. Yes, the board can do it, but the legal and reputation risks are such that at the very least you let the CEO resign and pass the torch down.

You are misusing the word coup

I couldn't imagine a statement that would better predict company failure at the hands of its competition.

Sutskever is simply much more interested in doing research like https://openai.com/blog/introducing-superalignment than just earning money.

I can see why, being able to give hard rules the AI can't break wouldn't just solve alignment but also solve the hallucination problem. It is the kind of breakthrough that would be worth trillions.

It's mind-blowing that a company of this significance was left in the hands of four people, two of whom are complete unknowns in the field - or at least utterly unclear what credentials got them there.

Well, that answers if we should continue to build for the GPT platform Sam introduced 11 days ago.

Sad. Thanks for posting.

The whole coup was very unprofessional. How can i trust such a leadership? It’s not clear if the path taken by sam will be discontinued regarding the GPT platform

I imagine he'll head straight to Microsoft, who basically have a fork of the model.

The language in that license agreement is about to become rather important. Numerous lawyers are going to wake up stuck to their sheets tomorrow morning. Not just IP attorneys, but defamation specialists.

Also the contracts and antitrust lawyers tasked with figuring out how to defend OAIs claim to all the Azure compute they rely on. If they’re suddenly competing with their main supplier and contending for the input they buy from that supplier … oof

Maybe they will switch over to Google for compute. Openai without the business genius is just Google research.

Americans can’t understand why someone might be driven by ideas and not be driven by money.

Kara Swisher: The board members felt Altman was manipulative and headstrong.

https://twitter.com/karaswisher/status/1725733594310512775

In other words “good at being a ceo”

If your definition of a ceo is that cynical, sure. Average people expect more; which I understand is an awful lot to all of, checks notes, the best paid person in a company.

Seriously, if a ceo needs to stoop to manipulation to get things done then they’ve lost control of the entire company and manipulation is forestalling the inevitable. Sure, a bad ceo can create large returns for investors but people who hold such myopic views of success navigate life with their training wheels still on.

This is no way to run an $80B company!

This thread was detached. I was referring to https://twitter.com/gdb/status/1725736242137182594

When you have a board who is out of their depth and no real owner, you are going to have problems.

You’re right. It’s instead how you run a nonprofit built to further the the vision of a once-in-a-generation ML researcher, without having it stolen out from under you.

Ilya and the board are entitled to their opinions and to act on them, but the whole process strikes me as extremely unprofessional, and something out of the Game of Thrones!

I would assume that “something out of Game of Thrones” were the table stakes, given the money and people involved.

Nothing about the story is unprofessional. Now, there's different process appropriate depending on the actual reason termination was being discussed, and we don't know enough to say whether this was appropriate fiven the cause, but its definitely in the scope of what is reasonable and orofessional for dismissing them.

It seems incredibly risky to give Microsoft’s lawyers $10bn worth of reason to take a close look at a novel corporate structure and surrounding case law. Why not cut a deal for a spin out?

Unless their contract had a specific stipulation that Altman shake hands on a stage with Nadella every three months or something, this seems largely irrelevant. So long as OpenAI continues to honor the deal terms about technology access, I don’t see how they’re exposed.

> It seems incredibly risky to give Microsoft’s lawyers $10bn worth of reason to take a close look at a novel corporate structure and surrounding case law.

Presumably, Microsoft’s lawyers did that before they bought into the novel corporate structure. I mean, its Microsoft, not Elon Musk. Due diligence is a thing.

So they’ll be that much faster filing whatever they’re going to file tomorrow morning.

There’s also the realpolitik of the situation - MSFT owns and controls the compute, in the worst case they can take their ball and go play somewhere else

Do you imagine executives/board member dismissals have some elaborate ceremony?

Typically they try to work out something behind the scenes before making a public announcement. And usually major stakeholders like Microsoft are brought into the decision-making process.

The announcement affected Microsoft's stock price today.

"not ideal element to it" is quite the quote. This four person board wants to be in charge of the most capable artificial intelligence on the planet, but seems to be less organized than the average school board[1].

[1] https://twitter.com/karaswisher/status/1725718391548207246

"it's interesting that someone with a 2014 bachelor's and a 2021 master's, with no experience other than a few years as a low-level employee at Open Philanthropy and then a strategy position at a Georgetown center, somehow leapfrogged onto the board of one of the most important organizations in the world."

https://twitter.com/stuartbuck1/status/1725695721729380357

"one of the most important organizations in the world"

lol

I am a frequent critic of both overhype of LLMs, and OpenAI specifically, but.....that is an indisputable statement.

Really? What would happen if OpenAI dissapeared tomorrow and ChatGPT closed down? In practice, nothing. Ask yourself the same question about the truly important organizations in the world.

Hype might die down and some of the hundreds to thousands (yeah…) of doomed (because, even optimistically, most are plainly never gonna be worth the resources going into them) AI boondoggles at various startups and bigcos might get cancelled sooner than they’re otherwise gonna be.

Which might be bad for my personal short-term employment prospects, so here’s hoping that doesn’t happen. We can ride this BS to the next speculation bubble if we believe in ourselves. And don’t look down.

This is like saying that Jeff Bezos leapfrogged into being the CEO of one of the biggest and most successful startups in the world.

Maybe he had something to do with it? Maybe, just maybe, it didn't just randomly happened to him.

The board member in question is not one of the founders.

Helen Toner is brilliant. This dude has never met her, yet somehow has the gall to proclaim he'd be better at her job than he is.

People shouldn't get board roles based on what degree they have, but based on how well they can do the job.

Please expand on that. Details would be helpful.

Copy-pasting from another thread where I posted this:

Helen Toner is famous among the AI safety community for being one of the main people working to halt and reverse any sort of "AI arms race" between the US & China. The recent successes in this regard at the UK AI Safety Summit and the Biden/Xi talks are due in large part to her advocacy.

She is well-connected with Pentagon leaders, who trust her input. She also is one of the hardest-working people among the West's analysts in her efforts to understand and connect with the Chinese side, as she uprooted her life to literally live in Beijing at one point in order to meet with people in the budding Chinese AI Safety community.

Here's an example of her work: AI safeguards: Views inside and outside China (Book chapter) https://www.taylorfrancis.com/chapters/edit/10.4324/97810032...

She's also co-authored several of the most famous "survey" papers which give an overview of AI safety methods: https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22h...

She's at roughly the same level of eminence as Dr. Eric Horvitz (Microsoft's Chief Scientific Officer), who has similar goals as her, and who is an advisor to Biden. Comparing the two, Horvitz is more well-connected but Toner is more prolific, and overall they have roughly equal impact.

Exactly one year ago OpenAI was nowhere near as known as today.

And that person's tweet chain reeks of speculative elitism.

Ilya and Mira are mad.

They discount the trust-based permission society grants Open AI because of its affiliation with the YC and Silicon Valley model just for LLMs.

Society gives some kind of trust based permission to OpenAI? What?

IP permissiveness is the basis for LLM quality. Those laws have not been written yet.

It is also the basis for human learning. You can’t actually limit somebody’s ability to learn from ingested data, it would make all human knowledge work illegal.

If you write the laws you can kind of do whatever you want.

It looks like they’re actually got full fledged AGI and are panicking.

Reply on news.ycombinator.com