Hacker News Reader

Comment by michaelbrave | Hacker News Reader

I've seen quite a bit of this too, the other thing I'm seeing on reddit is I guess a lot of people really liked 4.5 for things like worldbuilding or other creative tasks, so a lot of them are upset as well.

View on HN

There is certainly a market/hobby opportunity for "discount AI" for no-revenue creative tasks. A lot of r/LocalLLaMA/ is focused on that area and in squeezing the best results out of limited hardware. Local is great if you already have a 24 GB gaming GPU. But, maybe there's an opportunity for renting out low power GPUs for casual creative work. Or, an opportunity for a RenderToken-like community of GPU sharing.

The great thing about many (not all) "worldbuilding or other creative tasks" is that you could get quite far already using some dice and random tables (or digital equivalents). Even very small local models you can run on a CPU can improve the process enough to be worthwhile and since it is local you know it will remain stable and predictable from day to day.

If you're working on a rented GPU are you still doing local work? Or do you mean literally lending out the hardware?

Working on a rented GPU would not be local. But, renting a low-end GPU might be cheap enough to use for hobbyist creative work. I'm just musing on lots of different routes to make hobby AI use economically feasible.

The gpt-oss-20b model has demonstrated that a machine with ~13GB of available RAM can run a very decent local model - if that RAM is GPU-accessible (as seen on Apple silicon Macs for example) you can get very usable performance out of it too.

I'm hoping that within a year or two machines like that will have dropped further in price.

You are absolutely right that a rented GPU is not local, but even so it brings you many of the benefits of a local model. Rented hardware is a commodity, if one provider goes down there will be another. Or in the worst case you can decide to buy your own hardware. This ensures you will have continuity and control. You know exactly what model you are using and will be able to keep using it tomorrow too. You can ask it whatever you want.

I mean - I 'm quite sure it's going to be available via API, and you can still do your worldbuilding if you're willing to go to places like OpenRouter.

Reply on news.ycombinator.com