Hacker News Reader

Comment by Der_Einzige | Hacker News Reader

It all comes down to the VRAM. The average person will never have the money to buy a single H100 96gb let alone a DGX server or it's future equivalents

You get fundamentally more powers when you add more VRAM in ways that are just hard to explain to folks outside of this ecosystem. Everything around the VRAM are basically small details in comparison

View on HN

I know it’s not average person money, but the average person is not looking for hardware to run inference on LLMs.

A100 seem in the €15k ballpark and H100 double that.

Lot of money but I am actually surprised. A dedicated regular guy could buy this. I mean people buy cars and don’t really need them either. Again not saying it is a bargain, but it’s not billionaires only territory and that is good news (it’s early days!).

"Everything around the VRAM are basically small details in comparison"

...with current algorithms and our lack of understanding and insight in to how/why they work on a deep level or what intelligence and consciousness is.

With time hopefully all of these will improve and perhaps future AI's of good quality will be affordable to mere mortals.

Plus, I assume people want to have their assistant on their phone, not their desktop computer. So until everything can run locally on your phone, I think people will prefer the cloud versions.

I assume some people want their assistant to work on data they don’t want to share with megacorps and governments, and some of those people can figure out how to make their phone talk securely to a home server over the internet.

Reply on news.ycombinator.com