Hacker News Reader

Comment by ktta | Hacker News Reader

It's crazy for me to see that C is much slower than rust. I'm almost sure that there's something wrong there.

In table 1

   Rust    Fortran     C         Go

0m13.660s 0m14.640s 2m32.910s 4m26.240s

They did have a note that C could be faster if language specific features could be used.

Is rust really that performant compared to C? Or did they neglect to even bother checking C's performance?

View on HN

For me, clang -O3 optimizes the main function into this single loop:

    LBB3_1:
        addsd	%xmm1, %xmm0
        addsd	%xmm1, %xmm0
        ucomisd	%xmm0, %xmm2
        jae	LBB3_1

%xmm0 begins at 0, %xmm1 is 0.04, and %xmm2 is 3.6525E+8.

In other words, it's measuring how long it takes to count to 3.6525E+8 by adding 0.04 repeatedly. LLVM (which both Rust and clang use as a backend) optimizes away the actual work of the program.

GCC doesn't seem to optimize things out as aggressively, so that probably explains the difference: https://godbolt.org/g/lBRIIW

I.e the program has no side effects so it can be completely optimized away.

Never trust a benchmark you didn't falsify yourself.

gcc seems to do something similar for -Ofast. Adding something like

  printf("x[0][0]=%f\n", x[0][0]);

at the end of main fixes this.

Benchmarking is hard.

We do not claim that Rust is so much performant than C, we just showed that Rust can be as fast as Fortran or C. And indeed, there must be something wrong with the C implementation. Pull requests with improvements are welcome:

https://github.com/marblestation/benchmark-leapfrog

GCC produces almost 30% more instructions for the C version at -O3 level as opposed to -O2.

Clang produces practically the same between the two levels.

On OS X, so can't test if this actually makes a performance difference.

On my laptop (i7-5500U), I get the following timings (always with -march=native or -xHOST):

    GCC 6.3, -Ofast:      5.93s
    GCC 6.3, -O3:        93.06s
    GCC 6.3, -O2:       134.43s
    Clang 3.9, -O3:     346.92s
    Clang 3.9, -Ofast:    5.87s
    Intel 15.0, -O3:    110.84s
    Intel 15.0, -Ofast: 106.88s

The Intel compiler is relatively old, but mostly it seems to be a case of IEEE 754 conformance -- if enabled, things take longer, if disabled, they’re (probably) as fast in C as in Rust.

>On OS X, so can't test if this actually makes a performance difference.

The `time` command is part of base OS X.

But gcc is not.

In my reading of the comment it seemed to imply that the commenter already had both clang and gcc installed on their computer since they were talking about the amount of instructions produced by each. Perhaps I misunderstood?

I'm not sure, to be honest. I just assumed they were running GCC in a VM, but actually there was no real reason for me to assume that.

Interesting, the code looks more similar than I expected, yet the results are very different! Could it be that the difference is due to aliasing rules? It would be interesting to see what happens after adding a few restrict keywords here and there...

You would think that the presence of any unsafe block in the program's text, or even linkage with libc could result in (legitimate, non UB) aliasing in lots of code. Though I suppose that the private-linkage-by-default could help drastically reduce the scope it has to consider, maybe in some cases it can safely say that data is not aliased.

No, there is something wrong with the Rust version. I added println!("{:?}",x); to the end, and now I get 134 second runtime.

Also the output is [[NaN, NaN, NaN], [NaN, NaN, NaN]], which is a bit worrying...

All the particles in the simulation start at (0, 0, 0) so gravity is infinite, and the whole computation is operating on `NaN`s.

Not sure how much this affects the benchmark, but it'd probably be smart to randomise the starting positions.

Since the masses are also zero, the gravity is actually NaN.

Operating on NaN is hugely slower than normal floating point numbers. You're hitting a slowpath in the processor's floating point logic.

That is fixed now, conclusions are still valid :-) Thanks!

It depends heavily on the (micro-)benchmark used.

Rust is able to avoid allocations and de-allocations in many areas where C programmers would manually use alloc and free. I must admit though, that's less relevant to scientific programming, where the heaviest work is mainly matrix matrix multiplication and the sizes of everything is known in advance.

Without reading through the paper, I expect work on the Rust/Fortran side is being vectorized and/or being kept in L1/L2 caches, whereas the C and Go compilers don't use SIMD instructions and/or are generating code that triggers cache misses.

>It's crazy for me to see that C is much slower than rust.

It's not in the general case, but can be for particular code.

Unless the C program was written badly, the fact that Fortran is also faster seems to point to compiler optimizations not easy/possible in C (e.g. involving aliasing assurances).

There's something wrong there. Either a corner case, or they are multithreading and vectorizing the Rust/Fortran but not the C.

Which would be the equivalent of writing a comparison between a sledgehammer and jackhammer, without turning the latter on.

The code is available here:

https://github.com/marblestation/benchmark-leapfrog

It would be great if somebody have a fresh look at the C version, since I completely agree that it is not normal that its execution time is so far away from Fortran or Rust.

On my system:

  gcc -O3
  1m40.302s

  gcc -O3 -ffast-math
  0m5.110s

  rustc -C opt-level=3
  0m9.278s

gcc (GCC) 6.3.1 20170109

rustc 1.15.1 (021bd294c 2017-02-08)

Edit: Looking into this a bit more, I think 'rustc -C opt-level=3' is optimizing out the actual integration. If I put a println!("{}", x[0][0]) at the end, I end up with 1m43s. Not sure what 'gcc -O3 -ffast-math' is doing; I haven't looked at the disassembly.

Fantastic, thanks!!!

-ffast-math might be a bit cheating here, for scientific work it's not always applicable. Unless rustc uses fast math by default? That would be weird.

It's surprising that gcc does such a poor job at -O3 though.

Rust doesn't even have a way to turn on fastmath, AFAIK.

It does, for individual operations: https://doc.rust-lang.org/std/?search=fast

> If I put a println!("{}", x[0][0]) at the end, I end up with 1m43s.

Yes, without that it's basically cheating. There is now a GitHub issue: https://github.com/marblestation/benchmark-leapfrog/issues/6

The submitter writes: "final output [for Rust] is 134 seconds, and also the output is [[NaN, NaN, NaN], [NaN, NaN, NaN]] (batman)"

I don't belive in this result! C slower than Rust in 11 times!!! Are they kidding?!

http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan... -- in this benchmarks C in most cases faster than Rust, but only in few they have the same performance.

Compiler optimisations can be very nuanced, especially in C. You might find that changing the code very slightly, or even just using a different combination of compiler options, can speed up the code a surprising amount.

Edit: Looks like using -Ofast instead of -O3 makes a big difference: https://news.ycombinator.com/item?id=13634119

In most of those benchmarks rust is slower because it lacks SIMD support, not because of compiler optimizations.

Or rather, Rust is slower on those because SIMD is currently only supported on nightly, and the benchmarks game chooses to use stable Rust exclusively.

I also don't work with VS Release Candidates for production code, as such I think it is faire to use only stable Rust.

From what I've seen, most of the Rust community agrees that using stable a perfectly fine choice; kibwen is offering context, not complaining.

I agree with him, I express myself badly.

https://news.ycombinator.com/item?id=13636724

Offering context? ;-)

That is a slightly different thing: stable is a perfectly good choice, but one can still debate the fairness or not of being compared to C-with-language-extensions i.e. not standard C (a joke which I think you missed, in one of kibwen's replies).

There are places on the website that abbreviate to "C" or shorten to "programming language", but whenever practical it's more specific - "C gcc" & "programming language implementation".

Indeed, though I (perhaps optimistically) expect SIMD to be stabilized on Rust this year. :)

1. Have those Rust programs already been rewritten to use SIMD and measured ?

2. How many years already has SIMD not supported on stable been posted as a response in these discussions ?

1. Yes, but nobody's bothered to submit them because there's no point.

2. 1.75, which is coincidentally how long a stable version of Rust has existed. :P

I hope that one-day real-soon-now the much talked of SIMD support makes it into Rust stable.

There's a pre-RFC being passed around privately right now, I'm happy to say that your cherished dream will soon become a reality. :)

My cherished dream is no more "because SIMD is currently only supported on nightly" excuses :-)

Maybe that C code was the safe version which is inline with the rest?

Reply on news.ycombinator.com