Hacker News Reader

Comment by SheinhardtWigCo | Hacker News Reader

GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is 72 GHz.

You might be surprised to learn that dell "hpc engineers" (or maybe HPE?) have attempted to sell me hardware using that exact logic to compare servers.

VMware still uses ghz for displaying load and compute capacity on your servers.

Well, they're not wrong...

Given a highly-parallelizable numerics job, and all other things being equal, a 20 core 2.0Ghz machine will run the job in the same time as a 40 core 1.0Ghz machine.

...though usually we'd just give it in FLOPS - though FLOPS doesn't describe non-float operation performance.

Probably just a sales/marketing person who learned a new technical word and wants to sound knowledgable to the customer but it back-fires.

At the risk of torturing this analogy beyond recognition, it’s more like having an 8 core CPU but with a weak 150W PSU such that you can only run one core at a time.

Analogy sufficiently tortured

Is each core specialized to some particular type of task or something like that? Which would make this less silly of an idea.

in my understanding with an 8 way MoE youre still running all 8 for every request. only if you’re making a router model then itd be the “weak PSU” analogy. and even then a sufficiently perfect router would essentially have to know enough about its routes to almost simulate them which idk if i can believe works

Amdahl's law though ...

...doesn't really bite you for embarrassingly parallel numerics work.

Add the hurts please!

No, GPT-4 is 1.7T params in the same way that an AMD Ryzen 9 7950X is (24 times X million transistors). It's not wrong to say GPT-4 is 1.7T parameters, your statement is wrong. Parameters can be configured in different ways, just as transistors on a chip can. In the same way that having 24x transistors does not imply 24x performance, having 24x parameters does not imply 24x performance.

I think frequency is not additive like the parameters. If we are looking for analogy in compute power, then FLOPs is a better analogy to parameters.

I'm fairly certain the point of the gp was that the number of parameters are also not additive.

if you have a basket with 4 apples and a basket with 3 pears, are you not having 7 fruits ?

just because the first selection layer is very thin doesn't mean that the network cannot be considered composable

You can’t claim it’s a 7-fruit Basket, unless you find a way to fit them in the same basket

but you can claim its a 7 fruit grouping of baskets, we’re just in a territory where the fruit count matters less and less now

which happens to be consistent with what openai is telling us

Parameters are the number of weights needed to specify the model given the architecture; they are exactly additive.

A tree is not a clique.

If you can add FLOPs, and you have a number of cores with the same fixed number of FLOPs/cycle, it follows that you can add their frequency.

For the vast majority of real-life workloads there are vast differences between one core having X GHz or Y FLOPs vs lots of cores that sum up to that number. Which is the point GP tried to make.

Reply on news.ycombinator.com