Hacker News Reader

Comment by doctorpangloss | Hacker News Reader

Why is there so much investment in ecosystems outside of Hugging Face? And why do they keep reinventing PyPi?

I don't think diversity in a software ecosystem is a bad idea. Nothing is stopping Hugging Face from changing their terms and screwing us over

Google is great. Their slogan is “Don’t Be Evil.” We don’t need to invest in anything else.

(Time passes.)

“This ad-heavy, search engine is horrible. We need more solid competition!”

Supplier diversity prevents or reduces situations like the above.

Huggingface isn't local, while Ollama is.

This is more important than people seem to realize. Local means almost no data security issues. Not local means, well, all the data security issues.

What? That's not true at all. Citation needed!

"Huggingface isn't local, while Ollama is." - ms-menardi

It's just wrong and a lie. Huggingface is local.

I think you are arguing at cross-purposes. If you use HuggingFace to simply download models to run locally with ollama or llama.cpp, then you can say it is "local". But you can also use it as a service to run models (which is how they make money). Then they obviously aren't local.

Huggingface let's you run locally with huggingface exclusive code! They started this way and the managed/hosted solution came later.

You downvoters are the folks who jumped into this craze post ChatGPT.

I have no idea where people got the idea that Hugging Face isn’t local. I mean they show you how to run everything locally, with all the quantization strategies you could want, with far fewer bugs.

Who is reinventing PyPi? Ollama isn't even written in Python.

I think that's part of the point, PyPi and Python and Conda is an absolute shitshow of an ecosystem right now. 3 packages depend on 3 different versions of Pytorch. System installs CUDA 12.3 with the NVIDIA driver but but Xformers wants CUDA 12.1, and the NVIDIA driver that would provide 12.1 doesn't compile on Ubuntu 24.04. Pytorch provides its own CUDA that's different from the system version, and that makes another package unhappy. One package installs opencv, another package overwrites it with opencv-headless, pip install apex doesn't work with gcc-12, ...

Main reason I started using ollama is because it actually worked. I spent half an hour trying to get huggingface models to run on my GPU with no success. Meanwhile `sudo pacman -S ollama-rocm` seemed to just work out of the box.

Exactly, and as a bonus it runs an JSON API on localhost so you can call it from anything on your LAN, it's really, really amazing

Personally I love that ollama gives me a really simple API that I can code against in any language and on any platform that I care to. I can `docker run ollama` on my gaming rig and then hit it up from a Raspberry Pi running anything, Python or not.

Reply on news.ycombinator.com