Can anyone tell how performant this code is vs original C code? How much do we lose (if anything) by using Rust in comparison to C?
An entire game, written mostly in C for a (comparatively) extremely primitive CPU with a similarly primitive compiler. Performs all rendering in software, with tightly optimized x86 assembly targeted towards said extremely out of date architecture, and was the absolute bleeding edge of what was possible when it was released. On top of that, DOS was the target, which meant that a fair part of the codebase is dedicated to what programmers today would think of as an operating system, from device drivers to interrupt handlers to memory managers.
This:
A level renderer with none of the game logic, written in a higher level language that, nonetheless, has access to a much, much more sophisticated compiler and target hardware than the original game had, that offloads most of the rendering to the GPU anyway, whose author was never once required to think about optimizing anything because any computer made in the last decade could render such scenes in its sleep a thousand times over without so much as a snore.
I'm not at all trying to poo-poo the author here, but there is no meaningful comparison you can draw from this.
No perceptible performance hit on a 486? Somehow, I don't think that the 486 is an especially important target for the LLVM or Rust developers (though such chips are used in embedded applications to this day).
But the real reason that the comparison makes no sense is that the bulk of the processing time (but not the bulk of the code, mind you) is spent in those tightly optimized assembly loops. Either you keep it as such and it's not really much of a battle between C and Rust any more, or you rewrite the critical sections in Rust and you'd lose as hard or more as if you wrote them in C.
https://users.rust-lang.org/t/i586-support-illegal-hardware-...
Edit: the original renderer is actually very readable, https://github.com/id-Software/DOOM/blob/77735c3ff0772609e9c...
I don't see a great scope for auto-vectorisation etc there. You could write a new 2.5D renderer which assumed its availability, but that would be not quite directly comparable.
http://benchmarksgame.alioth.debian.org/u64q/rust.html
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
http://benchmarksgame.alioth.debian.org/u64q/which-programs-...
Edit: Looks like an C++ optimization issue in that specific task. Broken down by task, C is at the top followed by Rust.[1] Rust is at or near the top in every category. Very impressive.
[1] http://benchmarksgame.alioth.debian.org/u64q/performance.php...
Remember, it depends how you write the programs!
You're generally better off with random internet tests than the Benchmarks Game, since they at least don't have the kind of cheating-but-not-cheating that ruins the Benchmarks Game.
That said, it's totally possible to do assembly-level tuning in Rust and, sans-SIMD, there's no real difficulty with getting Rust to go fast on these workflows.
[1] https://alioth.debian.org/tracker/?func=browse&group_id=1008...; See ones with "Joshua Landau" in title. All (but thread_ring, which is literally the silliest benchmark I've ever seen) beat C++ (sometimes by a lot). The disclaimer would be that none of my submissions have been submitted (or rejected) yet.
Make your mind-up!
Is it cheating or not cheating?
http://benchmarksgame.alioth.debian.org/sometimes-people-jus...
>>"realistic" programs<<
For example?
"The performance of a benchmark, even if it is derived from a real program, may not help to predict the performance of similar programs that have different hot spots."
http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus...
Please let Veedrac speak for himself.
Veedrac's saying a bunch of stuff -- saying it doesn't make trash-talk into The Truth(TM).
>>fastest benchmarks game programs cheat, relative to the slowest ones<<
So programs that are compiled cheat, relative to programs that are interpreted?
Seems like "cheat" needs to be in scare quotes.
> So programs that are compiled cheat, relative to programs
> that are interpreted?
> Seems like "cheat" needs to be in scare quotes.
Which takes us back to the unanswered question --
Veedrac, are you saying that the benchmarks game programs you wrote "cheat" ?
"I say this as someone who spent a fair while optimizing the Rust benchmarks to use all of the hacks the other languages use"
Do you think the C, C++, Haskell, etc. implementations "cheat"? :P
You seem to think programs not written to your notion of "idiomatic" code "cheat".
I think that's just name-calling.
It would be completely disingenuous to suggest that Haskell is on top for any reason other than it doing a different thing to the other benchmarks. Haskell is just not that fast. So, in a sense, it cheats.
Of course, I copied these techniques in my Rust implementation. So, if Haskell cheated, I cheated. The same can be said for almost all other benchmarks - but not normally quite to this extent.
But it's not cheating in a literal sense, since (AFAIK) the Haskell program has been accepted on the basis that it follows the rules. So it doesn't cheat in that it follows the rules, but it cheats in that it should be breaking the rules.
I can be more specific if needed.
Please do! Can you give some specific examples, preferably in order of those with the most effect on performance.
This reminds me I haven't written up stuff for chameneos-redux, which is probably the most fun and over-engineered of them all.
The fasta/Haskell thing I was mentioning are the parts at "The Haskell code does a quite clever alternative" and "the Haskell code just caches them all in an array".
The pre-compacting I mention in k_nucleotide was already done in the Scala implementation.
How parallelism is added also varies. The Scala code does each of the test cases in parallel whilst the Rust parallelizes each internally. (My new Rust code parallelizes even better.)
Several implementation make their own specialized hash table, and there isn't one particular hash table to implement. This is very important since hash table lookups take a substantial fraction of the time.
Note that I pick on Haskell and Scala here only because they have the best hacks, not because they're the only ones doing hacks. (I'd imagine listing all of the differences would take much longer than I have time for.)
thread-ring has lots of different implementations. Some are just scheduled coroutines restricted to a single process, then you have stuff like [1] which uses mutexes, real threads and sets thread affinity. I'm not totally sure what [2] is but I think it's something else entirely. I'm not being too specific here because honestly this benchmark confuses me.
[1] http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
[2] http://benchmarksgame.alioth.debian.org/u64/program.php?test...
If the Haskell fasta programs do not cheat, my fasta program does not cheat.
These statements are made to the best of my knowledge, although someone with a definitive understanding of the rules (you, perhaps) is in a better position to make such decisions. I can make similar comments about my other submissions, referring in each case to different competitors.
I'm being ambiguous because you're forcing me to.
I've told you what I'm unsure about. I've linked to write-ups of what I've done. I've pointed to parts in other programs that seem not to clearly match against the rules.
Yet you, who knows the rules, have opted not to make one clarification about this. Not a single "yes, that technique is legal" or "no, this is against the rules". If you did this, I would happily give a clearer response. If my programs do break the rules, I would happily retract them until they don't, and point out which other submitted programs should also be retracted. This would not bother me.
Instead, you're demanding me to make definitive statements on information you're keeping hidden from me.
This is not making me feel pleasant, and this does bother me.
"Haskell is just not that fast. So, in a sense, it cheats."
: that is not "pleasant", it's name-calling.
When you say other programs are "cheating-but-not-cheating"
: that is not "pleasant", it's name-calling.
Firstly, I didn't accuse them of cheating. "X-but-not-X", under my understanding of idiomatic English, refers to something that's X in spirit but not technically X, or vice versa. The former was what I was going for.
Secondly, I have said what I think they were doing wrong, at least for some of them. Since you seem to have missed it (it's literally the only chain in this subthread you haven't replied to), see [1].
To be doubly clear, for Haskell I am talking primarily about the fact that the Haskell code caches the result of the linear searches done on each RNG output. (Not the RNG output itself, but the transformation applied to the values produced.)
You do write that a linear or binary search must be made, and on more careful reading this probably does make the Haskell code illegal. (FWIW, that's more of a problem for the Haskell code than the Rust, which IIRC isn't so bottlenecked there - and technically Rust does do a linear search.)
The second point with regards to Haskell is the copying of slices from a buffer of doubled length. As far as I remember, no other implementation (bar mine) does this. But the rules say
> generate DNA sequences, by copying from a given sequence
so don't really give much guidance. I'd assume this one is legal, but I'd encourage you to ban it anyway or get all of the implementations that don't do this updated, because it's extremely unfair on those that don't.
Then, still for fasta, you've given no guidance on how the code is legally parallelizable. The Rust code (prior to mine) parallelizes reading input with adding line breaks. I'm fairly sure other languages do different things, but it's taking too long to remind myself what they do actually do. I, as I've stated, parallelize the RNG by implementing skip-ahead to allow working on separate blocks at the same time. Is that legal? It's certainly not explicitly banned.
For knucleotide, is Scala allowed to precompact the input?
What are the rules about the hash-table? You do give one ("grow the hashtable from a small default size"), but it's not clear what a small default size is. Some of the code seems to violate my intuition of small.
How about pre-lowercasing the input. Is that legal?
For thread-ring, the top Haskell competitor uses pseudo-preemptive threads, but they don't allow the compilers to make any premption points (if that makes sense). This means the runtime is actually unable to preempt at all. The internet tells me Haskell can only preempt at allocations, which aren't being made. Is that legal? (It doesn't help that implementations have moved from the original general-purpose hashes to generally-poorer specialized ones.)
For chameneos-redux, you say
> don't use arithmetic to complement the colour, use if-else or switch/case or pattern-match
Go does a lookup in an array with colname[complement[c0|c1<<2]]. Is that legal?
Most of chameneos-redux is underspecified. There are more differences between the C and C++ implementations than there are similarities. I'm running out of motivation to list them all, though, so I'll continue once the points above are clarified.
Note that these are only the things that are ambiguous with regards to the rules, not the ones that are merely unfair.
>>>Not a single "yes, that technique is legal" or "no, this is against the rules".<<
> The fasta description has stated "don't cache the random number sequence" since 2011.
Perhaps one should notice that this is one of the few hacks that people aren't doing.
You told us -- "Since I'm lazy, I'll point out that most of the stuff is indirectly mentioned in this write-up: https://llogiq.github.io/2015/10/03/fast.html"
Here's what people will read if they look there --
"However, since the generator only produces 139968 separate numbers, the Haskell code just caches them all in an array."
Now you're telling us that is not being done.
A more specific phrasing would be "the Haskell code just caches the results from the linear search in an array for each possible input."
The Haskell code does not cache the RNG output itself.
I've asked Llogiq to update the wording.
Given your stated understanding, you still accuse them of cheating in spirit.
In ordinary English you equivocate, to have your cake and eat it too.
Indeed I do. I wouldn't mean that as a criticism of the author, though, lest I'd have not done the same.
Those programs are not intelligent agents, they cannot cheat.
When you say the program cheats, it is a direct criticism of the author's intentions.
> You can use cocoa powder to make the cake rather than chocolate - it's a bit of a cheat, but nobody notices the difference.
If you're trying to produce cheap cakes and your competition also uses cocoa powder, it's a mark of intelligence to follow along. It's still a bit of a cheat, though.
---
If you don't accept this reasoning, maybe I'm wrong and the word is unfairly used here. But please at least accept that I meant no insult, regardless of whether I have to vocabulary to express myself correctly.
So sheep are markedly intelligent?
>a bit of a cheat<
It is not the cake that cheats.
>I meant no insult<
I don't think you're confused about how all these comments you've made about cheating will be read.
> I don't want to get into a semantic quibble
If you'd rather the semantic quibble than actually address the concerns I've raised, on your head be it.
And I see no reason to trust your statements of my opinion over my own knowledge of my opinion.
The fasta description has stated "don't cache the random number sequence" since 2011.
The original C implementation also had the rendering loop done in pure asm.
Are things at the point where you could reasonable write a modern 3D engine using Rust 1.0? [1] What would be the tradeoffs at this point vs. C++?
[1] Ignoring all the tooling involved with art, music, etc.
Also, you'd likely want to be using Rust 1.4. It just came out this week.
Meanwhile, Rust also has the Glium library for OpenGL programming, which is apparently so good that it makes OpenGL programmers froth at the mouth (I've never done OpenGL programming myself, but the praise is universal among all my graphics-programming friends).
If you look at things like Ogre and Box2D, they can likely be sped up by a factor of 10x in either language. (not total fps in ogre, just the engine part).
Using 1.4 since 3pm PDT yesterday.
That said, I want safety more than speed. Speed I can get by writing an unsafe extension in C/C++. Safety is very hard to achieve in C/C++. Getting it right is the first priority. That's why I see Rust being used instead of where traditionally C++ is the default. Or some other new language that can fill C++'s shoes. Rust does seem like the strongest candidate at this point.
I'm confident Rust and Golang performance will improve as the projects get more mature. The generated code is full of optimization opportunities.
[1]: Autovectorization optimizations by C++ compilers tend to be fragile, small changes will undo them. So far easier to use intrinsics than figure out the black magic for autovectorization optimizations.
Also, microbenchmarks are usually not very good at representing the real world characteristics. But then again, please look at the current version of the linked benchmark.[3] Rust now positions itself in the second place, thanks to the recent improvements given to the Rust programs used in the benchmarks. I believe Rust isn't as fast as highly optimized C, but I think it is safe to say that Rust is just as fast as C++, and both Rust and C++ are much faster than GC'ed languages such as Java, Haskell, and Go.
Edit: I would even say that if Rust's performance is comparable to GC'ed languages, there's no point in using Rust in the first place. Rust is paying a lot of complexity to achieve memory safety without relying on garbage collection.
[1] https://news.ycombinator.com/item?id=9531822
[2] https://www.reddit.com/r/cpp/comments/35pn6h/criticizing_the...
[3] http://benchmarksgame.alioth.debian.org/u64q/which-programs-...
Rust has a lot of nice safety features not related to memory management that I'd love to see even in a language with GC. On top of the obvious lack of data races, linear types and borrowing are great for ensuring other kinds of sanity like preventing you from mutating a collection while you iterate over it.
I'm not sure that's a fair assessment.
Even if you didn't gain anything in terms of raw performance, the gains in performance predictability would probably still be enough for many applications.
Even if you didn't gain anything in terms of performance at all, the ownership semantics also give you good safety characteristics around concurrency, which would still probably be worthwhile for at least some applications.