Hacker News Reader

> A hacker news rumor was that they are paying $5M-$20M per year to the top neural net experts probably to make some exotic architectures to surpass transformer

This reminds me a TV interview of the author Patrick Modiano, just after he won the literature Nobel price. The presenter asked him if the money would help. The author answered essentially that the next time he would be in front of a white page, the money surely wouldn't help.

In the case of surpassing transformers, money could help to give access to more compute power. It could also help to prevent the research from being public.

Modiano is a rich man, born into a rich family. Wealth doesn't help in front of a white page, but it sure helps being able to stay in front of that white page instead of having to go take up a job because you're not sure what you're eating tonight.

As always, wealthy people and their "money doesn't make happiness" bullshit.

Since he already didn't need to work another job to pay the bills, the extra money from the Nobel prize does not make a difference in this case as he can already put all his time into writing.

Money doesn't buy happiness, but it staves off abject sadness.

If someone is already working on a problem full-time, money only helps to the extent that resources they can be buy with money are the limiting constraint. However, beyond deep work needed for a single individual, when you need to explore potential opportunities in a broad space of possibilities, money can hugely effect the search of that space because work needed for major breakthroughs remains parallelizable. You can delegate subtasks to people if you can afford those people. You can hire more of the few specialized people who know about a niche to work on your problem instead of other problems. You can exploit synergies from crosspollination of ideas from bringing together brilliant minds into the same conversations. The influx of money is very very likely to increase the pace of innovation in AI. The breadth of possible avenues for breakthroughs is largely yet-to-be-explored.

I'm not an expert but isn't size the distinguishing feature of an LLM? It's the first L.

They needed an architecture that could take advantage of the scale, first. That's what BERT did.

> They overturned previous scaling laws

Can you link to a comparison or graph of obsolete and new scaling laws?

The ones I had in mind were the newer 'Chinchilla' scaling laws (https://arxiv.org/pdf/2203.15556.pdf) vs. the older 'Kaplan' scaling laws (https://arxiv.org/pdf/2001.08361.pdf)

Curious if anyone can confirm $5-20M figure. Seems absurdly high but what do I know

Can't confirm OpenAI's position in particular, but $500k/yr/person is table stakes for a decent engineer directly connected to the company's bottom line. Double that for an actual expert, double it again if they're consulting, and put together a team of 3-10 of them. Those numbers aren't too far off.

I can see $5M per person being possible. $20M is the absurd part. 5 years with such a comp leads one to a net worth that is borderline filthy. Like elite, world renown athletes and actors level of wealth. Again, could all be true but just unexpected from my experience.

I mean if you are the top expert that will change the face of humanity, that is vastly more valuable then an athlet that entertains people, no !?

They certainly have that value. I wonder if they have that monetary value in modern capitalist society

> I can see $5M per person being possible. $20M is the absurd part. 5 years with such a comp leads one to a net worth that is borderline filthy. Like elite, world renown athletes and actors level of wealth.

5 years at $20 million equals the highest paid FIFA player’s single year pay (or Tom Cruise’s pay for one movie, Top Gun: Maverick), or about two years of the highest paid NFL players, so it won't catch up the their wealth over that time (assuming similar lifestyle) as its losing ground to them every year.

(And the claim was total, not per person, salary, anyway.)

Highest paid is a different metric though. Highest paid AI person is probably Ilya Sutskever who owns a good chunk of OpenAI and is worth billions

Those are highest salaries, they don’t include capital returns. Unsurprisingly, people make 7 figure salaries in sports, and who have been doing so for more than a few years, often have significant capital returns, as well.

There are CEOs that get paid a lot more than $5M/year in stock. Arguably that's also ridiculous, but it's certainly possible that paying a team of 20 highly skilled engineers $5M/year each brings more value to the company than paying a CEO $100M a year.

$20M in startup stock is not the same as $20M in Apple stock. I can totally believe OpenAI paying that figure on paper. You’d need to adjust the number for risk though

$5-20M in total yearly salaries, not per person. I think...

I wouldn't put any stock into a random twitter rumor by someone likely looking for clout. The source, some guy with likely a purchased checkmark and 12k followers (who knows how few before he claimed to have this insider knowledge), claims four(!) different "extremely reputable" sources that have independently confirmed it. How many people exactly are they making these offers to? Do they all happen to know this guy, someone with no discretion apparently, and everyone decided to tell him this information for what reason exactly?

99% chance it's made up.

That said, if they thought a specific individual had even a reasonable chance of coming up with an improvement on the current state-of-the-art AI architecture that they'd be able to keep entirely to themselves, $20M would be a massive bargain.

The rumor is still almost certainly fake, but for someone very specific at this critical time in the field, I don't know if the number would be that absurd.

Twitter rumors also claimed a parameter count of 100 trillion parameters and they visualized it with two circles with a huge size difference to make it look intimidating.

I guess the reason why AI is so interesting is that human stupidity is so widespread.

Good point. Super unlikely that the top people all know this guy. There are probably a hundred or so of them in the whole world.

I’d pay creator of GPT that money easily. Probably not anyone else

Here was my only source of the rumor:

https://news.ycombinator.com/item?id=35565025

maybe it's not per-person but the total for a small group. 7 6-figure salaries doesn't seem absurd.

You mean 6 7-figure salaries

That money won't help unless they get permission to start their own research department.