fspeech

It appears that they have really good toolings so they can focus on hiring really smart new grads and giving them a lot of leeway on choosing what to work on without worrying about how productive they will be. One particular memorable response when the CEO Liang was asked about not using KPI goes roughly like this: since we all reviewd and worked with the candidate, if he is not productive it must be our fault to not have given him the right support to make him productive.

Efficiency could lead to cheaper hardware for everyone, themselves included.

You need that to optimize load balancing. Unfortunately that gain is not available to small or individual deployment.

A better approach is to split the model with MOEs running on CPUs and MLAs running on GPU. See the ktransformers project: https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...

This takes advantage of the sparsity of MOE and the efficient KV-cache of MLA.

China proves that you don't need 100 times return on capital to get potential innovators to take risk. 100% may be enough. People say China couldn't innovate due to its weak IPR protection, but it is turning out that you don't need to be an absolutist. Sure outright theft is bad, but imitation can be good for competition. You want to give the innovators a head start so they can profit from their efforts, but you don't want to make the protection so tight that they can sit on their initial efforts and profit for years to come without further innovation.

You need a Chinese number to sign up and facial verification to make payments, as far as I can tell.

The smaller cluster is not necessarily a good thing. It may not be able to take advantage of unbalanced load on experts.

Deepseek doesn't have the infrastructure to support Apple. I suspect that they are also not interested in tailoring to Apple's needs given their mission and small size.

In a polytheistic context the invocation is much less severe, roughly equivalent to Apollo's Eyes or Zeus's Eyes.

That's due to the translation. The original term 天神 comes from the polytheistic Chinese folk religion so it doesn't have the same connotation.

Deepseek's open source inference code, while correct, may not be fully efficient. For example the MLA needs the right associative matrix multiplication order to be efficient.

Do you have any benchmark run yet? I am interested in knowing how many tokens/sec you can get to. Though in the end it should be more efficient to run the model on distributed server clusters.

Thanks for the thought provoking take.

Right. And the number is based on rental rates of GPUs so how many GPUs they own is irrelevant to the claim.

Recent developments like V3, R1 and S1 are actually clarifying and pointing towards more understandable, efficient and therefore more accessible models.

Unless the world demand for these raw material goes down or production elsewhere goes up someone will still buy these raw materials. If tarriff is high enough to force US to source from and Canada to sell to more distant places the world will be less efficient.

The key question is: has demand elasticity increased for Nvidia cards? An increase in elasticity means people are more willing to wait for hardware price to drop because they can do more with existing hardware. Elasticity could increase even if demand is still growing. Not all growths are equally profitable. Current high prices are extremely profitable for Nvidia. If elasticity is increasing future growth may not be as profitable as the projection from when Deepseek was relatively unknown.

That's not how commodities are priced. Commodities are priced at the marginal cost of production.

Inference can be done on much more diverse hardware.

You need to also look at valuation and how much growth justifies that valuation.

You are not taking into account why people are willing to pay exceedingly high prices for GPUs now and that the underlying reason may have been taken away.

It doesn't make sense to compare individual models. A better way is to look at total compute consumed, normalized by the output. In the end what counts is the cost of providing tokens.

But why would the customers accept the high prices and high gross margin of Nvidia if they no longer fear missing out with insufficient hardware?

If you normalize Nvidia's gross margin and take into account of competitors sure. But its current high margin is driven by Big Tech FOMO. Do keep in mind that 90% margin or 10x cost to 50% margin or 2x cost is a 5x price reduction.

Since intermediate steps of reasoning are hard to verify they only award final results. Yet that produces enough signal to produce more productive reasoning over time. In a way when pigeons are virtual one can afford to have a lot more of them.

It does if the spend drives GPU prices so high that more researchers can't afford to use them. And DS demonstrated what a small team of researchers can do with a moderate amount of GPUs.

Not necessarily if you are pushing against a data wall. One could ask: after adjusting for DS efficiency gains how much more compute has OpenAI spent? Is their model correspondingly better? Or even DS could easily afford more than $6 million in compute but why didn't they just push the scaling？

How could more competition be bad? Sure some startups' business plans no longer work and have to pivot but many more startups now have access to better products at lower cost.

Not sure why it's called grandiose. They are worried that supplies of jet engines could be cut off. Their own version doesn't have to be better if the alternative is no jet engine.