Hacker News Reader

I think frequency is not additive like the parameters. If we are looking for analogy in compute power, then FLOPs is a better analogy to parameters.

View on HN

I'm fairly certain the point of the gp was that the number of parameters are also not additive.

Parameters are the number of weights needed to specify the model given the architecture; they are exactly additive.

A tree is not a clique.

if you have a basket with 4 apples and a basket with 3 pears, are you not having 7 fruits ?

just because the first selection layer is very thin doesn't mean that the network cannot be considered composable

You can’t claim it’s a 7-fruit Basket, unless you find a way to fit them in the same basket

but you can claim its a 7 fruit grouping of baskets, we’re just in a territory where the fruit count matters less and less now

which happens to be consistent with what openai is telling us

If you can add FLOPs, and you have a number of cores with the same fixed number of FLOPs/cycle, it follows that you can add their frequency.

For the vast majority of real-life workloads there are vast differences between one core having X GHz or Y FLOPs vs lots of cores that sum up to that number. Which is the point GP tried to make.

Reply on news.ycombinator.com