Hacker News Reader

Comment by gpugreg | Hacker News Reader

> Some of us might be rich

I sure wish I had a few 100M of disposable income to train a frontier model.

> or in the future it could be useful when training is cheaper.

I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

View on HN

You say that but for like 2 years there was a guy on huggingface releasing quants as TheBloke. No identity no nothing except he likely had a grant or a university job that let him do it. Quants aren't the same as training but it was beyond 99% of people at the time (as you say this is)

Cant stand when you guys try and force something as impossible on the rest of us simply because you could never accomplish it.

I did not say that it is impossible. I just think that we need architectural improvements, or maybe even a fundamentally different approach to get something like Kimi K3 for cheap. The point I was trying to make was that we shouldn't just laze about and hope that hardware improvements will get us there.

(Also, I know Tom Jobbins (TheBloke), and have personally contributed to increase the adoption of GGUF, e.g. in the transformers and ktransformers libraries, so I find the personal dig quite amusing.)

If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful.

Looks like global energy consumption has risen by an order of magnitude from 1900 to 2000: https://www.encyclopedie-energie.org/en/world-energy-consump...

Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.

Prices have dropped by a factor of around 6 in real terms since 1925 for US household electricity.

https://chatgpt.com/share/6a6f3f3b-7624-83ed-a11a-248cb39728...

Only an order of magnitude? I'm both surprised by both the fact it's so low and the fact 1900 is 3x of 1800.

Reply on news.ycombinator.com