Hacker News Reader

How, summing (not averaging) to 58 of 1000 possible points (0-100 in each of ten domains), are we calling this score 58% rather than 5.8%?

View on HN

It's confusing. The 10 tracks each get 10%. So they add up all the percentages from every track. When you see the first table, 10% on math means "perfect" math basically. Not 10% of math track.

0-10 in each domain. It’s a weird table.

The simple additive scoring here is sus here. It means a model that's perfect on 9/10 axes but scores 0% on Speed (i.e., takes effectively infinite time to produce a result) would be considered "90% AGI".

By this logic, a vast parallel search running on Commodore 64s that produces an answer after BeaverNumber(100) years would be almost AGI, which doesn't pass the sniff test.

A more meaningful metric would be more multiplicative in nature.

Reply on news.ycombinator.com