Hacker News Reader

Comment by kpw94 | Hacker News Reader

Remember just 6 months ago that "Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI" (https://news.ycombinator.com/item?id=47088037)

(Ggml.ai is llama.cpp.)

Curious if the “I consider HuggingFace more "Open AI" than OpenAI” sentiment in that top comment will still apply with NVIDIA as the boss now...

View on HN

Nvidia is promoting "unmetered intelligence". Jensen wants everyone to have a spark dgx with an nvidia chip in their home.

Maybe but they make their real money in data centers. Certainly, they want to have fingers on every trend. Owning something the Government wants to control gives you more access to trade repression of open models for more Government favors and money.

Wasn’t Huang the guy who said your $500K employee should burn at least half his salary on tokens? I don’t think Nvidia is focused on cost-effective general computing for anybody.

It is possible to serve different markets with different agendas and have both of them serve nVidia’s business. Think of all the freemium software out there; widespread so everybody can use it, but the people who REALLY use it REALLY pay up.

NVidia has reasonable incentives to keep things open indefinitely, it wants people to use it's GPUs

Issue is that llama.cpp is the best way to run models on hardware that isn't nvidias.

Except when they have less than 16 gb of ram?

A lot of llama.cpp contributions come from the community and ecosystem, like Unsloth. If something goes awry, I fully expect lots of forks.

There already are a lot of forks for things they decline to implement. TurboQuant, ROCmFPX, and more. I need to set up an agent that will loop on merging them.

It'd be really nice if I could use my egpu 4090 on my macbook pro..

For LLM workloads this might help? https://docs.tinygrad.org/tinygpu/

Interesting, I've never seen this, thanks

Nvidia is halving the production of its higher end GPUs, doubling prices and heavily segmenting the market. They don't give a single shit about individuals. One wafer that makes 10 5090s that might sell at 3000 each, or one wafer that it already sold 6 months ago for 200k to one of the incestuous AI companies it works with?

Reply on news.ycombinator.com