Hacker News Reader

Comment by brap | Hacker News Reader

As someone who is out of the loop, what’s the verdict on R1? Was anyone able to reproduce the results yet? Is the claim that it only took $5M to train generally accepted?

It’s a very bold claim which is really shaking up the markets, so I can’t help but wonder if it was even verified at this point.

View on HN

> Is the claim that it only took $5M to train generally accepted?

Based on Nvidia being down 18% yesterday I would say the claim is generally accepted.

> Nvidia being down 18%

The only part of DeepSeek-R1 I do not like. I hope it's over, but I am not holding my breath.

Nvidia is now up only 1906% over 5 years. What a disgrace

It crashed all the way back to June 2024 levels, eons of progress wiped out

It is still unconfirmed since no one outside of deepseek reproduced it.

If confirmed, Nvidia could go down even more

based on information and background they thoroughly gave when releasing their research its pretty easy to put together that it did take them significantly less resources to train this model. only having specific parameters available at a time instead of activating everything all at once is pretty ingenious.

that and they just happened to be undergoing a large scale "cyber attack"

I'm not sure I see the bear argument for NVidia here. Huge AI models certainly drive NVidia sales, but huge AI models are also widely thought to be untrainable and nearly un-runnable save for large datacenters.

To me, this is ripe for an application of the Jevons paradox. If architectural improvements make similar models cheaper, I would expect to see more of them trained and deployed, not fewer, ultimately increasing the market for GPU-like hardware.

Because the markets are rational, all-knowing, and have never been wrong?

That was not the question.

It is if you're using market movements as evidence of anything factual. If markets aren't rational, you can't use them that way.

do you only take advice/learn from all-knowing people?

Do you know any?

But here's my advice: drop the fallacious arguments and try something more honest.

my argument isn’t fallacious - it is logical: we can learn/use evidence from something without presuming it is all knowing. you are putting words in others mouths that they did not say

I'm sorry, I thought you introduced the "all-knowing" out of nowhere, but this was indeed mentioned by willsmith72. I'd missed that.

Still, his implied assertion that markets that markets can often behave irrationally, and can't be used as evidence of technical matters, seems pretty valid to me.

But I suppose you could see it as a sign that something is at least temporarily "generally accepted" among investors. That doesn't mean it's generally accepted among AI researchers, though.

Although I thought it was $6M rather than $5M, and that that was only the last step, and not the total investment. What does seem to be generally accepted among investors that this isn't good news for NVidia's profits, but that still doesn't mean that all the specific facts are generally accepted.

Don't worry, NVDA will bounce back and you will get a chance to get out.

Efficient market hypothesis is for nerds!

No, because the market is an aggregate of opinions, so it’s entirely fair to say it’s “generally accepted.” That has nothing to do with whether something happens to be true or not.

It may provide a financial opportunity for someone who disagrees with that aggregated opinion though.

as opposed to HN comments??

While Deepseek was an instigator in the price movements I would not say its accepted.

I don’t see them as related. The market moves when there is money to be made. It’s only tangentially related to any kind of general sentiment.

“I don’t believe this, but I know others will, so I’m selling”

Huggingface is working on reproducing it: https://github.com/huggingface/open-r1

They claimed that it only took $5 million of GPUs to train Deepseek v3, which was the base model. They did not claim that the total costs were $5 million. They omitted the costs of additional hardware, electricity, personnel, training dataset acquisition, etcetera. They likely spent tens of times more on this at a minimum.

That said, what they did with $5 million of GPUs is impressive. Reportedly, they resorted to using PTX assembly to make it possible:

https://www.tomshardware.com/tech-industry/artificial-intell...

I think the jury is out. With folks trying to replicate the process we will see if the low budget is true or not. I am still on the fence, there was comments from Scale CEO that they have a huge number of H100s they used. On the market side I think regardless if this was true or not, this gave people the opportunity to sell what is perhaps overinflated valuations.

That's likely only the marginal cost of training this model, and doesn't include a lot of other costs, like the datacenters and GPUs themselves which they already had and also the staff.

If they aren't lying because they have hardware they're not supposed to have, which is also a possibility.

these claims are getting more wrong every time i see them, weird game of telephone going around tech circles.

the cost absolutely includes the cost of GPUs and data centers, they quoted a standard price for renting h800 which has all of this built in. but yes, as very explicitly noted in the paper, it does not include cost of test iterations

r1 probably cost way less to train, $5m is the alleged price tag for dsv3

Reply on news.ycombinator.com