Eh.
Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.
Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.
Secondly, all the competitors of OpenAI can plainly assess the truth or validity of Altman's statements. There are many companies working in tandem on things at the OpenAI scale of models, and they can independently assess the usefulness of continually growing models. They aren't going to take this statement at face value and change their strategy based on a single statement by OpenAI's CEO.
Thirdly, I think people aren't really reading what Altman actually said very closely. He doesn't say that larger models aren't useful at all, but that the next sea change in AI won't be models which are orders of magnitude bigger, but rather a different approach to existing problem sets. Which is an entirely reasonable prediction to make, even if it doesn't turn out to be true.
All in all, "his word is basically worthless" seems much to harsh an assessment here.
I personally can see why someone could arrive at that position. As you’ve pointed out, taking Sam Altman at face value can involve suppositions about how much he values his credibility, how much stock OpenAI competitors put in his public statements, and the mindsets people in general have when reading what he writes.
I don't find this implausible, but it folds slightly to Ockham's razor when you consider that this is the exact type of statement that would be employed to obfuscate a major breakthrough.
It just makes me crook my eyebrow and look to more credible sources.
The correct response is not to dismiss every statement from someone with a conflict of interest as "basically worthless", but to talk to lots of people and to be reasonably skeptical.
An AI, when presented with facts counter to what it thought it should say, agreed and basically went: “Won’t someone PLEASE think of the children!”
Love it.
I would imagine that any answer of ChatGPT on that topic is either (a) „hallucinated“ and not based on any verifiable fact or (b) scripted in by OpenAI.
The same question pops up for me whenever someone asks ChatGPT about the internals and workings of ChatGPT. Am I missing something?
I was curious about state persistence between prompt, or how to get my prompt better, or having a idea of the training data.
Only got crap and won’t spend time doing that again
For the most part it just regurgitated the corpo-speak with an odd sense of confidence. I know that’s the point of the model, but it can also be surprisingly honest when it incorporates what it knows about human motivation and business.
It’s pretty easy to have chatGPT contradict itself, point it out and have the LLM respond « well, I’m just generating text, nobody said it had to be correct »
It's easy to recognize and laugh at the AI replying with the preprogrammed narrative, I'm still waiting for the majority of people realizing they are given the same training materials, non-stop, with the same toxic narratives, and becoming programmed in the same way, and that is what results in their current worldview.
And no, it's not enough to be "skeptic" of mainstream media. It's not even enough to "validate" them. Or to go to other sources. You need to be reflective enough to realize that they a pushing a flawed reasoning methods, and then abusing them again and again, to get you used to their brand of reasoning.
Their brand of reasoning is just basically reasoning with brands. You're given negative sounding words for things they want you to think are bad, and positive sounding words for things they want you to think are good, and continuously reinforce these connections. They brand true democracy (literally rule of the people) as populism and tell you it's a bad thing. They brand freedom of speech as "misinformation". They brand freedom as "choice" so that you will not think of what you want to do, but which of the things they allow you to do will you do. Disagree with the scientific narrative? You're "science denier". Even as a professional scientist. Conspiracy theory isn't a defined word - it is a brand.
You're trained to judge goodness or badness instinctively by their frequency and peer pressure, and produce the explanation after your instinctive decision, instead of the other way around.
Excellent post, especially this part. Sums up the problem perfectly.
That was pretty much OpenAI's argument when they first published that GPT-3 paper. "Oh no so scary people might use it for wrong stuff, only we should have control of it."
In reality they just saw $$$ blink.
This is the second [1] OpenAI claim in the span of a few days that conveys a sense of "GPT-4 represents a plateau of accomplishment. Competitors, you've got time to catch up!".
And it's not just a financial incentive, it's a survival incentive as well. Given a sufficiently sized (unknowable ahead of time) lead, the first actor that achieves AGI and plays their cards right, can permanently suppress all other ongoing research efforts should they wish to.
Even if OpenAI's intentions are completely good, failure to be first could result in never being able to reach the finish line. It's absolutely in OpenAI's interest to conceal critical information, and mislead competing actors into thinking they don't have to move as quickly as they can.
He said that we are at the end of the era where capability improvements come primarily from making models bigger. Which stands to reason... I don't think anyone expect us to hit 100T parameters or anything.
Turns out there are diminishing returns and advances come from other dimensions. I've got no opinion on whether he's right or not, but he's certainly in a better position to opine that current scale has hit diminishing returns.
In any event, there's nothing special about 1T parameters. It's just a round base-10 number. It is no more magic than 900B or 1.3T.
- Abraham Lincoln
That's my theory and if I was a tech CEO in any of the companies competing in this space, that is what I would plan for.
Training an LLM will be the easy part going forward. It's building an ecosystem around it and hooking it up to everything that will matter. OpenAI will focus on this, while not-so-secretly training their next iterations.
Yeah, I also had a hunch he wasn't an AI. (I assume you meant "AI researcher" there :))
All joking aside, I wonder how that's affecting company morale or their ability to attract top researchers. I know if I was a top AI researcher, I'd probably rather work at a company where the CEO was an expert in the field (all else being equal).
> OpenAI’s ChatGPT unleashed an arms race among Silicon Valley companies and investors, sparking an A.I. investment craze that proved to be a boon for OpenAI’s investors and shareholding employees.
> But CEO and co-founder Sam Altman may not notch the kind of outsize payday that Silicon Valley founders have enjoyed in years past. Altman didn’t take an equity stake in the company when it added the for-profit OpenAI LP entity in 2019, Semafor reported Friday.
It's a massive bet for a company to push compute into the billion dollar range - if saying something like this has the potential to help ward off those decisions, I don't see what's stopping them from saying it.
I basically see Microsoft talking when Sam talks.