am17an

One often overlooked after that is ggml, the tensor library that runs llama.cpp is not based on pytorch, rather just plain cpp. In a world where pytorch dominates, it shows that alternatives are possible and are worthy to be pursued.

Holy smokes we're cooked.

Maintainers time is a more scarce resource than free tokens. I would much rather get my time back after reading those PRs

1) Python is unreadable." Would you prefer C or C++?

> Unironically, yes. Unless I never plan to look at that code again

Use llama.cpp? I get 250 toks/sec on gpt-oss using a 4090, not sure about the mac speeds

Well a 1000 line PR is still not welcome. It puts too much of a burden on the maintainers. Small PRs are the way to go, tests are great too. If you have to submit a big PR, get buy in from a maintainer first that they will review your code.

I agree, this would be in the same vein as "STL returns a verbose type, it's okay to use auto here because no-one cares"

This has got to be a hell of a story if it's true.

Usually codebases disallow auto because without an IDE it's difficult to see the type. I think this reduces the cognitive load of C++ a bit. The only time it is allowed is getting iterators types from STL containers.

I remember fretting about these rules when reading Scott Meyer's Effective C++11, and then later to realize it's better not to use auto at all. Explicit types are good types

All a LLM does is hallucinate, some hallucinations are useful. -someone on the internet

I hope this is not a serious argument. This deal is OOM larger than other deals

It's bad for consumers period. A deal that hampers 40% of global supply shouldn't be a thing, it's predatory. I know DRAM is not a necessity, but considering that PCs are going to be affected means this affects real things like schools and hospitals. There's being smart while making a deal and there's knee-capping the market with your leveraged to the tits business

This is one of my favourite problems, I still remember that it has a very real edge case even though I solved it more than 10 years ago. Thank you for the problem!

The non-thinking version is the best writer by far. Excited for this one! They really cooked some different from other frontier labs.

What is parallel decoding?

Humbert is a bit on the nose, for those who get the Lolita reference.

The answer is 0

I don't feel weird about it because it's a trillion dollar business, and it's in their best interests to muddy the waters regarding any cohesive argument. But when you just look at these things from first principles, boasting about 1 billion hours of shorts watched in a month or something ridiculous like that, AI generated content on loop, teens/adults getting into status/envy games on these platforms, the "connection" aspect looks like a very small positive.

I don't think your example of off-label Ozempic is isomorphic - the more apt comparison is one I already gave: Why did it take us 30 years to realize cigarettes cause 400k deaths a year? The answer is because there was simply huge amounts of money involved.

The kind of person who doesn’t believe this is bad for children, is the same kind of person who would believe Big Tobacco sponsored studies about how cigarettes don’t cause cancer in the 80s. For the data driven like yourself, I remember a Jeff Bezos anecdote, “When the data and the anecdotes disagree, the anecdotes are usually right"

Try spending time with a phone addicted teen and you might have a more sympathetic take.

Probably for the better, its incredibly damaging to young brains.

This model is literally amazing. Everyone should try to get their hands on a H100 and just call it a day.

They must really be having a bad time if Anthropic of all labs is willing to share their infra details. On the actual precision bug, it is quite unfortunate on FMA side, numerical issues are often deeply bewildering and no AI can solve them (yet.) Also goes to show, if you are in a super crunch situation like this one (competitor literally eating your lunch every day), you need humans to understand what went wrong and even then it can take weeks to rectify.

Somehow that's still an understatement

Isn't this factually wrong? Grok-4 used as much compute on RL as they did on pre-training. I'm sure GPT-5 was the same (or even more)

The point is those kernels exist already, you can just use them off the shelf. In the case where you're trying to write a production grade kernel without operating at that part of the stack... well good luck with that.

There’s a GitHub link which is open from last year, about the missing license in ollama. They have not bothered to reply, which goes to show how much they care. Also it’s a YC company, I see more and more morally bankrupt companies making the cut recently, why is that?

This is nice and useful because the new GPT-OSS model uses this technique. Kudos to the original authors!

Seriously, people astroturfing this thread by saying ollama has a new engine. It literally is the same engine that llama.cpp uses and georgi and slaren maintain! VC funding will make people so dishonest and just plain grifters

I’ve never seen a PR on ggml from Ollama folks though. Could you mention one contribution you did?

Asymptotically, prompting is a programming language.