Are you saying that the benchmarks game programs you wrote "cheat" ?
Please let Veedrac speak for himself.
Veedrac's saying a bunch of stuff -- saying it doesn't make trash-talk into The Truth(TM).
>>fastest benchmarks game programs cheat, relative to the slowest ones<<
So programs that are compiled cheat, relative to programs that are interpreted?
Seems like "cheat" needs to be in scare quotes.
> So programs that are compiled cheat, relative to programs
> that are interpreted?
> Seems like "cheat" needs to be in scare quotes.
Which takes us back to the unanswered question --
Veedrac, are you saying that the benchmarks game programs you wrote "cheat" ?
"I say this as someone who spent a fair while optimizing the Rust benchmarks to use all of the hacks the other languages use"
Do you think the C, C++, Haskell, etc. implementations "cheat"? :P
You seem to think programs not written to your notion of "idiomatic" code "cheat".
I think that's just name-calling.
It would be completely disingenuous to suggest that Haskell is on top for any reason other than it doing a different thing to the other benchmarks. Haskell is just not that fast. So, in a sense, it cheats.
Of course, I copied these techniques in my Rust implementation. So, if Haskell cheated, I cheated. The same can be said for almost all other benchmarks - but not normally quite to this extent.
But it's not cheating in a literal sense, since (AFAIK) the Haskell program has been accepted on the basis that it follows the rules. So it doesn't cheat in that it follows the rules, but it cheats in that it should be breaking the rules.
I can be more specific if needed.
Please do! Can you give some specific examples, preferably in order of those with the most effect on performance.
This reminds me I haven't written up stuff for chameneos-redux, which is probably the most fun and over-engineered of them all.
The fasta/Haskell thing I was mentioning are the parts at "The Haskell code does a quite clever alternative" and "the Haskell code just caches them all in an array".
The pre-compacting I mention in k_nucleotide was already done in the Scala implementation.
How parallelism is added also varies. The Scala code does each of the test cases in parallel whilst the Rust parallelizes each internally. (My new Rust code parallelizes even better.)
Several implementation make their own specialized hash table, and there isn't one particular hash table to implement. This is very important since hash table lookups take a substantial fraction of the time.
Note that I pick on Haskell and Scala here only because they have the best hacks, not because they're the only ones doing hacks. (I'd imagine listing all of the differences would take much longer than I have time for.)
thread-ring has lots of different implementations. Some are just scheduled coroutines restricted to a single process, then you have stuff like [1] which uses mutexes, real threads and sets thread affinity. I'm not totally sure what [2] is but I think it's something else entirely. I'm not being too specific here because honestly this benchmark confuses me.
[1] http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
[2] http://benchmarksgame.alioth.debian.org/u64/program.php?test...
If the Haskell fasta programs do not cheat, my fasta program does not cheat.
These statements are made to the best of my knowledge, although someone with a definitive understanding of the rules (you, perhaps) is in a better position to make such decisions. I can make similar comments about my other submissions, referring in each case to different competitors.
I'm being ambiguous because you're forcing me to.
I've told you what I'm unsure about. I've linked to write-ups of what I've done. I've pointed to parts in other programs that seem not to clearly match against the rules.
Yet you, who knows the rules, have opted not to make one clarification about this. Not a single "yes, that technique is legal" or "no, this is against the rules". If you did this, I would happily give a clearer response. If my programs do break the rules, I would happily retract them until they don't, and point out which other submitted programs should also be retracted. This would not bother me.
Instead, you're demanding me to make definitive statements on information you're keeping hidden from me.
This is not making me feel pleasant, and this does bother me.
"Haskell is just not that fast. So, in a sense, it cheats."
: that is not "pleasant", it's name-calling.
When you say other programs are "cheating-but-not-cheating"
: that is not "pleasant", it's name-calling.
Firstly, I didn't accuse them of cheating. "X-but-not-X", under my understanding of idiomatic English, refers to something that's X in spirit but not technically X, or vice versa. The former was what I was going for.
Secondly, I have said what I think they were doing wrong, at least for some of them. Since you seem to have missed it (it's literally the only chain in this subthread you haven't replied to), see [1].
To be doubly clear, for Haskell I am talking primarily about the fact that the Haskell code caches the result of the linear searches done on each RNG output. (Not the RNG output itself, but the transformation applied to the values produced.)
You do write that a linear or binary search must be made, and on more careful reading this probably does make the Haskell code illegal. (FWIW, that's more of a problem for the Haskell code than the Rust, which IIRC isn't so bottlenecked there - and technically Rust does do a linear search.)
The second point with regards to Haskell is the copying of slices from a buffer of doubled length. As far as I remember, no other implementation (bar mine) does this. But the rules say
> generate DNA sequences, by copying from a given sequence
so don't really give much guidance. I'd assume this one is legal, but I'd encourage you to ban it anyway or get all of the implementations that don't do this updated, because it's extremely unfair on those that don't.
Then, still for fasta, you've given no guidance on how the code is legally parallelizable. The Rust code (prior to mine) parallelizes reading input with adding line breaks. I'm fairly sure other languages do different things, but it's taking too long to remind myself what they do actually do. I, as I've stated, parallelize the RNG by implementing skip-ahead to allow working on separate blocks at the same time. Is that legal? It's certainly not explicitly banned.
For knucleotide, is Scala allowed to precompact the input?
What are the rules about the hash-table? You do give one ("grow the hashtable from a small default size"), but it's not clear what a small default size is. Some of the code seems to violate my intuition of small.
How about pre-lowercasing the input. Is that legal?
For thread-ring, the top Haskell competitor uses pseudo-preemptive threads, but they don't allow the compilers to make any premption points (if that makes sense). This means the runtime is actually unable to preempt at all. The internet tells me Haskell can only preempt at allocations, which aren't being made. Is that legal? (It doesn't help that implementations have moved from the original general-purpose hashes to generally-poorer specialized ones.)
For chameneos-redux, you say
> don't use arithmetic to complement the colour, use if-else or switch/case or pattern-match
Go does a lookup in an array with colname[complement[c0|c1<<2]]. Is that legal?
Most of chameneos-redux is underspecified. There are more differences between the C and C++ implementations than there are similarities. I'm running out of motivation to list them all, though, so I'll continue once the points above are clarified.
Note that these are only the things that are ambiguous with regards to the rules, not the ones that are merely unfair.
>>>Not a single "yes, that technique is legal" or "no, this is against the rules".<<
> The fasta description has stated "don't cache the random number sequence" since 2011.
Perhaps one should notice that this is one of the few hacks that people aren't doing.
You told us -- "Since I'm lazy, I'll point out that most of the stuff is indirectly mentioned in this write-up: https://llogiq.github.io/2015/10/03/fast.html"
Here's what people will read if they look there --
"However, since the generator only produces 139968 separate numbers, the Haskell code just caches them all in an array."
Now you're telling us that is not being done.
A more specific phrasing would be "the Haskell code just caches the results from the linear search in an array for each possible input."
The Haskell code does not cache the RNG output itself.
I've asked Llogiq to update the wording.
Given your stated understanding, you still accuse them of cheating in spirit.
In ordinary English you equivocate, to have your cake and eat it too.
Indeed I do. I wouldn't mean that as a criticism of the author, though, lest I'd have not done the same.
Those programs are not intelligent agents, they cannot cheat.
When you say the program cheats, it is a direct criticism of the author's intentions.
> You can use cocoa powder to make the cake rather than chocolate - it's a bit of a cheat, but nobody notices the difference.
If you're trying to produce cheap cakes and your competition also uses cocoa powder, it's a mark of intelligence to follow along. It's still a bit of a cheat, though.
---
If you don't accept this reasoning, maybe I'm wrong and the word is unfairly used here. But please at least accept that I meant no insult, regardless of whether I have to vocabulary to express myself correctly.
So sheep are markedly intelligent?
>a bit of a cheat<
It is not the cake that cheats.
>I meant no insult<
I don't think you're confused about how all these comments you've made about cheating will be read.
> I don't want to get into a semantic quibble
If you'd rather the semantic quibble than actually address the concerns I've raised, on your head be it.
And I see no reason to trust your statements of my opinion over my own knowledge of my opinion.
The fasta description has stated "don't cache the random number sequence" since 2011.