Hacker News Reader

Hacker News

Comment by warpech | Hacker News Reader

Full thread

warpech

The chart is messed up... Bonus mess points for choosing 1 instead of 0 as the starting poing on the vertical axis.

My best guess to understand it is:

- On the horizontal axis, we have consecutive Chrome releases.

- On the vertical axis, we have total time of benchmark execution, relatively to Chrome 75

The source of the data is the "js-framework-benchmark"[0] run in a controlled environment in Chrome[1]

[0]: https://github.com/krausest/js-framework-benchmark

[1]: https://krausest.github.io/js-framework-benchmark/2020/table...

View on HN

Starting at 1 is standard when you're showing measures relative to a baseline. It's a multiplier. 1.4 = 1.4x measure over the baseline.

Is it? I've never seen a graph do this. Even e.g. fund performance comparing to index, normalized to starting value, show below-the-line in all graphs I've seen.

E.g. this https://www.blackrock.com/uk/intermediaries/products/230286/...

Doesn't start from zero, but is very clear that it doesn't start from unity either.

It’s pretty easy to understand this chart. In finance absolute numbers make more sense as they are “static”, but in benchmarks it’s typically not very useful to show eg the time spent, as this depends entirely on your hardware and is not very interesting. Rather, you solely want to compare with other numbers, and as such you normalize everything to 1.00 as the starting point.

This type of chart is extremely common in benchmarks.

I'm not saying it's hard to understand.

And how are finance numbers, in what I linked to, static?

I have no problem with normalizing the number. That seems like the obvious best way. That's why they also do it in the finance graph I linked to.

Sorry, I think static was not the correct word; I think I meant absolute, ie Tesla stock being worth $1k is the same for everyone. This is not the case with benchmarks, where the numbers are fairly meaningless and different for everybody, depending on their hardware.

How is it not the same with stock and funds, e.g. what I linked to?

What I linked to doesn't show the fund price, it shows the value normalized. Just like benchmark values usually are. It's not the normalization I (and others) am saying is bad, it's the graph.

But yes, stocks are usually shown graphed with their ticker value. Funds are usually not.

The complaint is that the axis is zero suppressed, not that the data starts at 1. The latter is fine, the former a typical dark pattern.

For multiplicative values, 1 is effectively the zero axis.

I understand what you mean, but no. If you use a linear y-asis, you want to see for example that it goes to twice the amount, or halve the amount. The data-to-x-axis distance does this when the zero is not suppressed.

Alternatively, you can go to a log plot, where constant factors produce constant shifts. But even then, one would typically not put the x-axis origin at 1 on the y-axis.

What happens if you have a multiplier value of 0.4x? Can't do that on an axis that starts at 1.

If the y axis is relative benchmark time, it has to start at 1.

Why? What if Chrome 76, 77, 78, 79, 80 were faster than 75? The line needs space to go down.

It would go down if the data point lay there.

Fair enough :)

While the data has to start at 1 (or 100%), the axis absolutely does not.

What happens during the first second of time before reaching 1?

It seems clear enough to me. And you too since your guesses are all right.

> Bonus mess points for choosing 1 instead of 0 as the starting point on the vertical axis...

What purpose would a block of whitespace at the bottom serve?

The length of the line between 0 and 1 would put the measurements in right perspective.

Not really... would the performance ever be “0”? No. So it doesn’t make sense to be on the graph. Really, it should be logarithmic: 100 ops/s, 10 ops/s, 1 op/s, .1 ops/s.

I think you are right. It is just a new perspective for me, though I can read in other comments that it's quite common in benchmarks.

Charts that I'm used to have units like FPS or MS on an axis that starts at 0.

Reply on news.ycombinator.com