Hacker News Reader

Comment by anthropodie | Hacker News Reader

I want to know if it will ever be possible to run this kind of AI at home once its training is complete. I dont need all the knowledge just subset that I'm interested in.

Actually I'm more interested in its ability to transform things. For example I can ask it to convert docker-compose to docker run command, it can manipulate JSON, it can sort numbers in table when prompted. I'm more interested in these abilities rather than just getting answers for which I already have Google

View on HN

It uses GPT-3 under the hood which requires about 350 gigabytes of GPU VRAM (back of the envelope calc, likely more) to perform these inferences.

7x NVIDIA RTX A6000's so ~$32,550

This is honestly affordable for a lot of upper-middle class people and might well it worth it. It's like the cost of a car. I can seriously see this writing a book for me if I can get it tuned to study only my writing style and remember all of my texts. But it could also only cost $14000 14x RTX 3090s.

I’ll wait a year and buy $2k worth of hardware that runs it.

Or wait a further few years and spend $20.

Or a couple years later for it to be 20 cents.

Or a couple years later for them to give it to you for buying a bottle of Coke.

In the interim, they will find ways to make money from us.

Difference is in first mover advantage. If you can be the first to use it to bring value to yourself and your clients, you can easily make up the cost of that hardware.

5x limit per manufacturer, maybe 3rd party for the additional?

So maybe in 5 years we would be able to run it in our $800 smart glasses.

If this was open sourced it may be quickly optimized, the amount of VRAM required for image generation went down very quickly, I'm sure Dalle-2 is still using enormous vrams but other solutions are not.

Yeah buddy!

Look up fine tuning GPT-J in 8 bit mode.

People have made domain-specific models that perform well (IIRC, better than GPT-3 in their domain).

The team behind Stable Diffusion is also working on one that's supposed to be pretty good.

you can do that today in the free release?

I think he wants to self host. It sucks to have no ownership of such a powerful tool I would pay upwards of $3000 to be able to self host something like this.

Rest assured someone is working on a self-hosted (distilled) model. Stable Diffusion has shown there is a viable market for open, consumer-hardware inferencable models.

Reply on news.ycombinator.com