Hacker News Reader

Hacker News

Comment by onlyrealcuzzo | Hacker News Reader

Parent Full thread

onlyrealcuzzo

Don't we already have small models highly distributed?

View on HN

We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...

I'm not sure we can look forward to self-hosted models ever being mainstream.

Like 50% of internet users are already interacting with one of these daily.

You usually only change your habit when something is substantially better.

I don't know how free versions are going to be smaller, run on commodity hardware, take up trivial space and ram etc, AND be substantially better

> I'm not sure we can look forward to self-hosted models ever being mainstream.

If you are using an Apple product chances are you are already using self-hosted models for things like writing tools and don't even know it.

The "enshittification" hasn't happened yet. They'll add ads and other gross stuff to the free or cheap tiers. Some will continue to use it, but there will be an opportunity for self-hosted models to emerge.

> Like 50% of internet users are already interacting with one of these daily. You usually only change your habit when something is substantially better.

No, you usually only change your habit when the tools you are already using are changed without consulting you, and the statistics are then used to lie.

You make a fair point, I'm just hoping this will happen, but not confident either to be frank.

Because small models are just not that good.

The vast majority won't switch until there's a 10x use case. We know they are coming. Why bother hopping?

I use gpt4all and have downloaded some models. It's not that hard, and doesn't take a huge system. It's pretty easy if you use models on their list, it's a bit more work to find the right chat interactive script (forgot what they are called) for your class of model if you downloaded one that's not on the list, but not that hard. I have 8GB VRAM, and it's slow but they work. I like that it's local and private, but then I don't use them for much other than as an oddity that is fun to interact with.

Reply on news.ycombinator.com