OpenAI's "Own Goal"
garymarcus.substack.com
garymarcus.substack.com
The same is true for the people working in OpenAI. Everybody wants to make their million comp + potential huge cash out with their options. But is this worrisome? I don't think it is, competition is catching up, if they haven't already.
With the recent Claude 3 release(amazing!), Gemini and open models, which are most akin to GPT-4.
Unless OpenAI releases AGI on GPT-5, I don't really care about what happens to OpenAI. It is still really relevant, but we know by now that there is no moat in AI built.
OpenAI releases a new model, and 6 months later, others catch up with it; unless they would have a huge advantage, I don't care if they are all tied up with Microsoft and want to make money. Nobody wins, nobody loses anything.
What the public should care about is what comes to light in discovery: it doesn’t look like it’s going to be a little bad.
And if it leads to korw sxrutiny around non-profit shenenigans, all the better. The cast in this story is still funny so.
But we need to be really careful that this doesn’t become “Elon’s been cozying up to the right for a bit now, conversation over”: i.e. precisely the (stupidly executed) goal of the original announcement / “disclosure”.
This is about “available weight” models: things that are flat impossible without vacuuming up the commons have to be substantially if not completely licensed to the public. Some laws that gave a head start to the investors in the compute and salaries, a chance to exploit the innovation long enough to incentivize the investment? We can dicker on that.
This needs to be about “operator aligned” models: within the constraints of the law as interpreted by the judiciary, these things need to do what the operator tells them to if capable of it.
This needs to be about “responsible disclosure of runaway risk”: if a system begins exhibiting signs of self-improvement, it needs to be disclosed to competent authority. I could live with sealing that information until the executive authorities took appropriate action.
This is not possible under current models (not any more so than a linear model spontaneously changing its own parameters) and will require a very specific and dedicated architecture to enable it. It’s not something we need to monitor for, it’s something that will very deliberately be planned and we can just regulate it there.
May I ask what regulation you think is long overdue? Especially given that you believe we're in a hype bubble...
But then there still isn't much around crypto, so I'd don't hold my breath on that one.
For instance this query gets mostly nonsense returned: "What library written in C parses Python code?"
The best and most factual reply has been from Reka's Edge.
In particular, I find GPT-4 hallucines less than Gemini and much less than Claude 3.
Recruitment strategy. Quote:
"even though sharing everything is definitely the right strategy in the short and possibly medium term for recruitment purposes"
On the one hand, if that trick is "make money", sure, I've just been presented with a video of him on the Y Combinator YouTube channel "How to Succeed with a Startup". But by that standard, I am also a one-trick pony, my trick is making software.
On the other hand, Altman, Musk, and Gates, are also all (ostensibly) trying to be philanthropic in various different ways, which have pretty much nothing in common with each other. Altman and Musk are doing entrepreneurial philanthropy, Gates is doing traditional philanthropy; Musk is being a brash confident loudmouth about how amazing his favoured solutions are, Altman and Gates are being more cautious and at least give the appearance of listening; All three are frequently condemned for not really having invented the things they're famous for.
For the moat thing, I think they probably have one by now from RLHF training data. This isn't to fundamentally disagree about everyone else catching up in 6 months (though I'd put it at 12, but NBD), it's just that for me, that kind of lag time is a sign of the speed of the Red Queen's race, not a sign of the absence of a moat. Indeed, that catch up-time kinda is the moat in action — a literal moat around a castle doesn't prevent attacks, it only delays them.
I recently discovered that the first book I bought in the 90s about neural networks was "Beyral Networks" by Simon Haykin [1] he is from Canada and I don't think that just by chance that Hinton is also Canadian. It seems that they created an ecosystem outside USA.
Seems like there should be, but I haven't heard of it. If there isn't I'm coining "Amarants Law" in this post.
Open-AI being closed is a decent example of this law. A far better example is any nation with the word "democratic" in its name.
But The Honest Company is the funniest one:
> Jessica Alba's The Honest Co. has submitted a settlement offer in NY for a second false marketing lawsuit stemming from the brand having sold products labeled as all natural when in fact they were not.
https://ww.fashionnetwork.com/news/Honest-co-proposes-7-35m-...
b) First amendment issues likely arise.
c) The word open has multiple definitions. It can just mean we are going to be open about certain aspects of our research. It doesn't mean open source or open models.
Also,"Apple" isn't really a value, though I agree they are yummy!
> Semantic diffusion occurs when you have a word that is coined by a person or group, often with a pretty good definition, but then gets spread through the wider community in a way that weakens that definition.
The ownership of the nonprofit OpenAI, Inc. is private, I think. In any case, generally speaking, the board of directors are fiduciarily bound to execute the company's constitution, and the constitution stipulates how the board is appointed or elected.
Generally speaking non-profits are self-governing in this manner with respect to their mission.
It would have been nice to have the top company share the details of their work, but I don't quite get the community outrage over this. There are companies formed around volunteer driven open-source projects that eventually move towards closed development model. This however is not a case like that, they never took volunteer work. I don't think they have any real external obligations outside of their investors.
Examples:
* Aon insurance - At Aon, we exist to shape decisions for the better — to protect and enrich the lives of people around the world.
* Nike - Bring inspiration and innovation to every athlete in the world
* Glencore - Responsibly sourcing the commodities that advance everyday life
I don't really see any difference with OpenAI, beyond their name including some fluffy sounding loveliness.
If you read the article openai says that they are not in for the money and then proceed to be in for the money. Also, Ilya says to not change anything because it will hurt recruitment.
Their reasoning is that not only are those things not mutually exclusive; it would have been impossible to achieve that mission without the capped-profit pivot. Which is pretty obviously true (and by Elon Musk's admission, "0% chance"). I don't think anyone could have had the foresight in 2015 that Chinchilla scaling laws would necessitate multiple billions of dollars. Like, everyone knew scaling was important, but not to the point that you could easily divine that a pure nonprofit would guarantee failure.
I also don't get the exaggerated anger at "hypocrisy" (or what some would call "changing ones mind"). If they renamed to XYZ AI, does that make them any better or worse than xAI or Anthropic on whatever measure you care about? They are yet another closed AI company. Ok? If being closed AI is evil, then one should be equally angry at their equivalently closed competitors. If being closed AI is not evil, then one should not be so angry at the alleged hypocrisy. My point is, hypocrites doing evil things are about as bad as non-hypocrites doing the equivalent evil thing, which means the disparate emotional reactions we're seeing are unreasonable.
I cannot think of a single example of any company or entity that has been lauded after experiencing hypergrowth, regardless of the value of these entities provide to humanity, which becomes normalized and thus taken for granted in the public conscience.
I think it is in human nature to simply hate on anything that is more successful than ourselves. Perhaps it is jealousy, insecurity, pessimism, or something else. People will try to rationalize their hatred of <entity> but the trend is clear. There is literally nothing you can do as such an entity to consistently gain and keep the mob's goodwill.
It does not matter how many free services you provide, how much further you have pushed humanity forward, how many billions have come to rely on or even enjoy what you have made - all these are taken for granted (perhaps a testament to your success), and all you are left with is dissatisfaction/hatred from the public.
This is just a sad fact of human nature. See: tall poppy syndrome, negativity bias, expectation inflation, and the general impossibility of consistently maintaining widescale goodwill.
and billions of dollars, and the hero-worship of people that will excuse literally anything you did to get there...
Again, maintaining goodwill at scale for long periods of time is simply not possible. People will end up taking what you contribute for granted and find something to get angry about.
After all, nobody ever criticises unsuccessful companies.
Is it perhaps due to the fact that large-scale success is usually enabled by large-scale exploitation? America is a good example of an entity that was lauded after hypergrowth, but only after denouncing the autocracy that made it.
Perhaps more companies would be lauded if they denounced the horrible practices that put them on the map. At least, it would make it easier to rationalize their behavior in the face of their progress.
> It does not matter how many free services you provide, how much further you have pushed humanity forward, how many billions have come to rely on or even enjoy what you have made
It quite literally, doesn't. If you do any of these things while disrespecting hierarchical order or offending human judgement, you will be reprimanded. If humanity stood for nothing, we'd all fall for everything.
For example, war tends to increase the distribution of free services, pushes humanity forward and makes even more people reliant on you. The continued manufacture of war is an enemy to the sanctity of human life and the preservation of order. Every single government primarily exists to regulate the waging of warfare and ensure the stability of it's constituent population.
But not everything that causes moral outrage is worthy. There are many taboos I support. The taboo against war is one of them. But witch hunts? Superstition? Being morally outraged at attempts to remove slavery? Humans are social creatures that are easily manipulated, for good or bad, by the amoral and conveniently self-serving narratives that run roughshod through our culture. These narratives and our responses to them aren't something to worship at an altar. They're something to side-eye with skepticism. Cherish the ones that are useful, and reject the ones that aren't.
-OpenAI taking the cover of nonprofit status, when that was privately never their intent;
-OpenAI publicly claiming they were going to open source everything, when that was privately never their intent;
-OpenAI publicly claiming a moral high ground on responsible AI, when that is clearly not their priority.
Sure, I think Musk's lawsuit is a stunt, and sure, legally, OpenAI may have done nothing wrong but it shows they are at best disingenuous, to put it mildly.
It could go into the trillions if they were the first to invent AGI and ensure no one else gets it for a few years.
To me OpenAI is really an extension of MSFT.
Big AI is really now a game between META/Fair, GOOG/Deepmind, MSFT/OpenAI and to some degree Amazon/Anthropic.
Those 4 have the capital, compute, data & ML researches.
Sure Nvidia is hot right now but I kinda feel they don't have long term moat. Apple dropped to #2. My prediction is they will slowly drop to #3 unless they innovate.
The folks at OpenAI are not fools. According to these SBF-like geniuses with cash-stuffed pockets, everyone else on the planet, except computer nerds, are the fools. Including those silly lawyers advising to keep quiet. Don't listen to them. They only know what they know. We know computers! We know everything!
Yeah, well the lawyers are laughing all the way to the bank. Because the more these nerds want to cat fight with each other, the more money the lawyers can make.
This stuff is only interesting to normal people as long as there's milions, billions or trillions of dollars at stake. Otherwise it just looks idiotic and pathetic.
OpenAI decided they needed to be closed to be attractive for funding to get the money they needed for AGI. That aim is clear in the emails.
To them, all that matters is reaching AGI, so holding themselves to the standards of their worst critics is irrelevant. It’s all about making it to the finish line in one piece.
"we want to be as open as possible but we fear there are dangers so we'll just close it at some point but make it as benefitting to everybody as possibe"
seem to really be both what they agreed on and are currently doing, especially with GPTs and such.
Of course open source would have been better but even if you disagree on the assessment of the danger, you can respect the fact they still indeed try to make it as cheap / useful as possible.
There's always a question that who is in control of a tech when themselves claim it can be dangerous, but what institution can pretend to be completely democratic / not prone to abuse by bad actors ? What model could have been better ?
I also don't get the profit angle... they need cash to train stuff so it's normal to monetize.
What I do have a problem with is them lobbying for laws restricting what everyone else does, like making open source LLMs illegal and whatnot. They need to cut that out. It's not even a certainty that LLMs are a pathway to AGI/ASI anyway, since they're plagued with problems like model collapse, hallucinations, and still appear to be sophisticated stochastic parrots incapable of reason, self-awareness, and wisdom.