Hacker News Reader

Comment by iLoveOncall | Hacker News Reader

> We propose intelligence per watt (IPW), task accuracy per unit of power

Stupid metric. It's not because a model is better performing that it necessarily requires more energy or compute.

I don't think that's what they are saying. In fact if they did the metric would be pointless. Rather they are saying by estimating that value on different architectures, one can find more efficient ones. They use open model to be able to remove unknowns. They aren't advocating for one model or another, only more efficient architectures.

Intelligence per Joule would be more appropriate in many cases. If a model can do the same work but takes 10 times as long as a bigger one that can still be useful (e.g. due to memory constraints), but at the same wattage it burns 10 times the energy. Even more so on mobile devices.

They also define and measure an "IPJ" as well as "IPW"

> the NVIDIA B200 achieves 1.6× to 2.3× higher intelligence per joule than the APPLE M4 MAX across QWEN 3 and GPT-OSS model variants

The B200 = "cloud", M4 = "local".

So "cloud" does even better in energy than it does in power compared to "local". Or, to flip it, "local" is both slower and more expensive than "cloud".

Then the metric should be Watts per Intelligence, not the contrary.

Is there any practical difference? One is just an inverse of the other and can be trivially derived.

Yes, there is a massive difference, and you cannot invert them to get the other.

Watt per Intelligence means that you have a fixed, deterministic, measure of intelligence, and you calculate how many watts it takes to get there.

If your goal is to measure which model can reach a specific outcome with the least energy possible (which is what GP says the goal is for this metric), then you cannot have a variable outcome, which is what intelligence per watt describes. As opposed to watt per intelligence, where the outcome is fixed and the numerator defines how much energy expenditure is needed to reach this fixed outcome.

Here is a more everyday example: distance traveled by a vehicle and the amount of fuel consumed.

> Fuel efficiency can be expressed in terms of the volume of fuel to travel a given distance, such as in litres per 100 kilometres, or through its inverse, the distance traveled per unit volume of fuel consumed, as in kilometres per litre.

If you have a car that consumes 1 liter of fuel per 100 kilometers, it's the same as saying it travels 1 kilometer per 0.01 liter of fuel. They are equivalent, describing the same relationship and proportion. It makes no difference whether the quantity is a "deterministic" or "variable" measurement. You can use either unit, depending on the aim of your calculation.

This has absolutely nothing to do with the problem at end. I'm so tired of people bringing up random metaphors that don't apply to justify obvious bullshit when it comes to AI.

> We propose miles per hour (MPH), distance travelled per unit of time

Stupid metric. It‘s not because you spend more time that you travel farther.

Not a well-thought-out take.

Reply on news.ycombinator.com