Hacker News Reader

Comment by PostOnce | Hacker News Reader

It's several reasons, first it's the lies and the abuse of a charity, that wouldn't be an issue if they had began as a private company instead of robbing a charity.

But secondly, even if they were a private company, it's dishonest and reprehensible to claim to congress that you want to "protect the public" when you really only give a shit about protecting your moat, I'm not happy about that either.

I'm also tired of tech oligarchs general tomfuckery in all our daily lives, as I suspect many more people here are. OpenAI is just particularly egregious about it.

I also think it's my civic duty to let other developers know that OpenAI does not have, by any stretch of the imagination, a stranglehold on this technology or any secret sauce. That's why they're lying and sweating in front of congress.

View on HN

Yeah, they went from v1 to regulatory capture in the span of months

> regulatory capture

If this leak is correct, regulatory capture is likely the only moat OpenAI could have hoped for. It would explain why Sam was so absolutely adamant that this tech needed to receive oversight.

If correct, every big tech company now has a recipe to build their own GPT-4. I'd expect for the open source efforts to try to duplicate the results as well. LLMs will increase in quality across the board and become fairly interchangeable, leading to thin margins and a race to the bottom.

We could be watching $13 billion going up in flames, all of it predicated on a secret as flimsy as the Coca-Cola recipe.

Just wait until the RLHF class action hits them.

$13 billion isn't $13 billion if $N billion of it has to mandatorily be spent on high margin (70%?) Azure services.

Tbf the coca-cola recipe is a more sustainable trade secret, since the product is “perfected”.

GPT4 is a work in progress so a competitor can be objectively better.

I’m hoping for open source models to start incorporating some of these ideas.

Did you actually read my comment or did you have a preplanned diatribe?

They didn't start the private company until the person who promised them $1B reneged. That person reneged because they tried to forcefully takeover and were rebuked.

They were running out of money and forced to raise funds in a very for-profit way or fold.

Edit: Rate limited because the hivemind has decided that it's unacceptable to insult their leader

People keep acting like if it wasn't for OpenAI we'd be in some LLM utopia. The reality is some other big tech giant would reach the current SOTA first and we'd be in the same predicament except with a company with 1000x more machinery to do the things you're complaining about.

It's ridiculous the sense of entitlement some people must have to keep insisting that OpenAI should have crawled into a cave and died because some megalomaniac threw a tantrum.

So fold.

Running out of money to run a charity means they now suddenly need to fuck over the people the charity was supposed to help (that is, everyone on earth) ?

Imagine if a feed the homeless charity accepted private investment and went around installing anti-homeless spikes everywhere after stopping feeding anyone. It's a similar style of behavior.

It's not even remotely the same as anti-homeless spikes.

We've just had the massive fine against 3M for knowing and hiding the risks of PFAS, and the top comments here were "increase the fines! Lock up the bosses!"

Now we have a company going "we had to put a lot of effort into preventing this model from cheerfully outputting Al Quaida propaganda, explicit rape threats, and detailed instructions for an amateur to make deadly chemicals using only home supplies", and despite that effort they still had legal trouble in Italy because the output was unsuitable for minors (as well as the GDPR issues that were the primary headline at the time).

And a lot of researchers with no financial incentives saying "yup, there's danger in these AI".

The reaction here?

Disbelief OpenAI might be saying what they mean, and meaning what they say.

And people parroting "moat!" as if none of the other FAANGs could trivially cross any regulatory barriers that emerge.

It's like the entire topic of AI has been politicised harder than "is nuclear power green?"

> "outputs al qaeda propaganda"

So does Microsoft Word. Both require a human to tell the software what to output.

> as if none of the other FAANGs could trivially cross any regulatory barriers that emerge.

That's the point.

This miracle technology does not belong to a few rich men, it belongs to us all.

Tech oligarchs are provably not more responsible than the rest of us, and do not deserve to lock us out of the garden whose fruit they seek to pick.

A rich man can meet the regulations that allow him to build a robot to take your job, but YOU are not allowed to build the same robot to help boost your own income? Because that might be irresponsible?

> So does Microsoft Word. Both require a human to tell the software what to output.

Clippy does what now?

Or do you mean "I can type", because if so you're minimising the very same capabilities that you're later describing as a miracle and saying belongs to us all.

> Tech oligarchs are provably not more responsible than the rest of us, and do not deserve to lock us out of the garden whose fruit they seek to pick.

"No more responsible than the rest of us" is a dangerously low standard.

The rest of us, collectively rather than each and every one of us, play lotteries, drive dangerously, addict ourselves to drugs, pickle our livers, and win Darwin awards.

For all our sophistication and sophistry, we're all just fancy balding primates with fairly similar tribal attitudes and motivations.

> YOU are not allowed to build the same robot to help boost your own income

Yes, obviously, with literal robots there are countless examples of public liability insurance and health & safety legislation. With computers, likewise, because they're connected to stuff.

Lol, I’ve never heard a single person suggest that we’d be living in some LLM utopia if OpenAI was gone.

Who have you been talking to?

> the abuse of a charity,

A non-profit and charity are two different things.

While a charity is a form of non-profit, it has to follow certain rules to qualify as one. Their profits must go towards the charity.

A non-profit is a company that is set up not to make a profit. It is allowed to make a profit if it does. This is what OpenAI was.

They switched to a "capped" for-profit model so that they could get more funding. It also allowed their employees to invest in the company, and openAI gave equity to their employees.

There was no lies or abuse. Where did you get that information from?

> It is allowed to make a profit if it does.

A nonprofit is subject to the non-distribution constraint: any revenues that exceed expenses must be committed to the organization's purpose, not taken by private parties.

https://en.wikipedia.org/wiki/Nonprofit_organization

The "regulatory capture" conspiracy theory makes no sense to me. It takes 9 figures in cash to create and run one of these super big models. Only big tech was ever going to create them, and big tech is already very experienced at navigating regulation, regulation wasn't ever going to stop them from competing. And in general, our democracy works better than the nihilist libertarians give it credit for.

I bet it'll be 6 figures within 18 months.

Even if it was free to train, FAANGs can beat OpenAI on spending to follow regulations.

The thing about these models is compute scales quadratically with model dimensionality and memory scales quadratically with sequence length.

We are nowhere near diminishing returns for either variable, so sure current models maybe scale quickly but the cutting edge will want as much compute as possible for a long time.

That’s kind of the humor of everyone saying this leak somehow leaves OpenAI vulnerable.

The work isn’t deciding if an MoE is the right architecture or not, it’s how to run 25k GPUs concurrently in a fault tolerant way (likely the true reason for the deep Azure links).

Memory does not scale quadratically with sequence length.

During training, you have to store a dot product of Q and V that has dimension Ncrt^2.

That's quadratic scaling, no?

presumably you mean a dot product of Q and K, and no you do not have to store this: https://arxiv.org/abs/2205.14135

I mean, sure you can work around it, but from your own link:

>since the time and memory complexity of self-attention are quadratic in sequence length

Except in practice this is not true, and hasn't been for more than a year. It's not just a workaround either -- FlashAttention is both faster at runtime and uses less memory.

I interpreted openAIs regulatory capture bid as more of an attempt to create competition-hostile regulation than an attempt to reduce regulation to cut costs.

My point is that openai's competitors have no problem handling regulations.

Reply on news.ycombinator.com