Hacker News Reader

Comment by _def | Hacker News Reader

> You'll start depending on it and then it'll get taken away from you.

You can not get around this, conceptually. Local inference will keep depend on updated models for quite a while. Partly because they will contain outdated training data, partly because of demands for the improved models. And it's still not clear where this will lead us. We're still in the rosey phase where people get lured in.

View on HN

But, we know 100% that cloud-provided anything can and will get nerfed, broken, removed, or in some other way rug-pulled. It already happens all the time, and companies are getting more and more aggressive/stingy about what counts as "yours" and what counts as "bought" and what counts as "acceptable use".

Yes, bringing everything local still means you need to download things, maybe over and over. But once it is on your machine, nobody can yoink it from you just because they want more money or they don't like what you're doing with it.

Is there any hope that something like unsloth studio will let people retrain models continually so that, when the day comes that there are no good open models being released, something like the final generation of qwen whatever can continue being relevant into the future?

It would likely fall on the harness (and possibly local datasets) to encourage that the model reach out for up to date information instead of relying on it's own internal "knowledge".

We already see a lot of that with `web_search` tooling, so I imagine it would just become more essential to have tools like that.

Maybe this is just my lack of experience thinking, but that seems really unlikely to scale without retraining. The context will balloon with updated syntax for languages and APIs for libraries and so forth. Imagine if, for example, something on the scale of custom elements / web components were introduced in this post-open world. You couldn't fit all the information needed in a local model's context to tell it how to write a new custom element of any real complexity, especially if it needed to interact with the new post-release web transport specification to interact with a new language's client for a new database.

You can just make archives of the web and language specs, best practices, etc and RAG it via a pipeline every month or so.

Updated models isn’t the same thing as hosting them somewhere else

Hermes injects the date/time into the stream, and you tell it to web search for anything newer than the training cutoff date. Works like a charm with deepseek flash.

We will try anyway. No need to try to FUD people into passive acceptance of the cloud.

Reply on news.ycombinator.com