Comment by didibus | Hacker News Reader

Comment by didibus | Hacker News Reader

Parent Full thread

I'm not sure it is flawed. It just seems that you found the root cause of why NumPy/CuPy are slower, but it seems there is no way as a user of NumPy/CuPy to perform the float32 computation, even though you were explicit and said `:dtype "float32"`.

All these libraries under the hood delegate to the same things for the actual computation, so the only advantage of one over the other would be this type of stuff.

That NumPy/CuPy coerce to a float64 under the hood, and gives us no way to force it not to do so, is a disadvantage that they have in my opinion.

In any case, like the article says, the performance of NumPy/CuPy is still good enough, and maybe NumPy thinks it knows better then it's users that they should be using higher precision always.

Edit: I know I'll face the brigade of NumPy users. But I invite people to engage in the discussion unbiased. With a switch from CPU to GPU you'd expect more than 29% performance boost. If CuPy went with a float32 proc, it would give you a 96% performance increase! So ya, I think we can start to argue CuPy isn't doing the right thing here.

> user of NumPy/CuPy to perform the float32 computation

This is just getting tiring.

Numpy and Cupy are perfectly capable of doing float32 computation - their only "fault" is that they coerce data to float64 in this one fairly unimportant function (which you can implement to your liking in 3-4 LOC). Hell, PFNs entire deep learning system, Chainer (which mind you, both predated and inspired PyTorch, and is still quite competitive with it), is built entirely on top of Cupy!

Benchmarks make sense only when the outputs are the same - in this case, they certainly are not. It's the responsibility of the author to make sure that the outputs are the same (a real benchmark), or to argue that Numpy/Cupy are wasting time by using float64 in xp.cov (an issue of implementation). He does neither.

> Numpy and Cupy are perfectly capable of doing float32 computation

You're arguing that they would perfectly be capable of an implementation which would use float32, ya sure, if you change their source code you can have them be faster, that's an obvious to me? The question is, what kind of performance can you expect as a user using them as a library, and this benchmark is revealing of that in this case. What is wrong with it?

It's like saying that you can change the implementation of Python and it will make Python code run faster, like sure you can, but you don't benchmark hypothetical future versions, the current version of CuPy uses float64 for this function, and that makes it that it barely runs faster then the CPU version. Point in case, I don't know what else you're trying to disagree about here.

> their only fault is that they coerce data to float64 in this one fairly unimportant function

I don't know that, all I know is when picking one function and benchmarking it, we found a flaw which result in performance gains from CPU to GPU to be extremely underwhelming. How many more if we started to benchmark the rest?

> if you change their source code you can have them be faster, that's an obvious to me? The question is, what kind of performance can you expect as a user using them as a library, and this benchmark is revealing of that in this case. What is wrong with it?

You don't need to change the numpy or cupy source code, just write your own version of the function in a few lines of code using other primitives that numpy/cupy provides.

Note that is exactly what is done for the neanderthal version.

How am I supposed to know that corrcoef is slow and I should rewrite it? I didn't know before, now thanks to this article I do.

Reply on news.ycombinator.com