But at the same time I feel sad for the state of science where working code is still not considered part of the paper, so the code is almost never published, let alone published as Free Software for other scienists (and hobbyists) to tinker with it, to fix it, to improve it, and to adapt it.
For example, why isn't it possible for everyone interested to let Leela Zero play against AlphaGo Zero, on their own hardware (or their university's hardware)?
Even worse is that a friend of mine who is a researcher in neuroscience/BCI has found that many of the papers couldn't be reproduced at all with flawed or incorrect conclusions. Since they typically don't publish their code, its very difficult for others to check the correctness.
Publishing spreadsheets is perhaps an even more urgent issue than publishing other types of code.
And of course the underlying data (anonymized) in case of statistical elicitation, but that's another topic. (Open Access, Open Data and Free Software are very closely related, though).
It might make some didactic sense, but it makes fewer and fewer sense as times go by when you can ship the actual code and shows actual results
Of course, there is a non-trivial performance trade-off with more cycles but more objects fits in cache, etc.
Some quick googling gave for example this source which seems to hint that under some situations it is not worth it at least https://stackoverflow.com/questions/4240974/when-is-it-worth...
Lastly, it seems like being quite harsh to call the code being of poor quality because of something like that. Unless it really is very important to get the extra memory, just the maintenance cost of packing things efficiently might make it not worth it. Even production code bases such as the JVM will happily use a single byte (or even more?) to store a boolean, and such things probably occur all over the place.
In this case the author is DeepMind/Google, so they know how to code and how to distribute it.
However we need not only the code but also the network. Leela Zero is code but more importantly an effort to spread the effort of training the network among many people. Each one volunteers some hw (CPU/GPU) and contributes a part of the training runs. The README states that the estimated training duration on commodity hardware would be 1,700 years. We're not that patient :-)
In any case, you'd still be running a competition between nodes for fastest/best solution, with lots of duplicated effort and consumed resources. That is the key part of crypto consensus afaiu, and regardless of "backend task", there's always going to be waste in this architecture.
Hard to make, easy to verify.
This may be a lot harder than you think.
How do you decide that one AI plays better than another? By playing lots of games. These games must be random, i.e. not known beforehand, otherwise the new AI could cheat.
But then, every cryptocurrency node would use a different set of game settings for validation. Which means that some (unlucky) nodes may reject the new AI while others accept it. The result are regular net splits, making it very hard to establish an keep the global consensus that is absolutely needed for a stable cryptocurrency system.
The core issue here is that validation must be 100% deterministic. Each node must run exactly the same calculations for validation, otherwise the consensus network is doomed.
Im assuming all games played against the incumbent; seeds are basically proof of wins. I guess you could still cheat though by copying the incumbent, get a 50-50 ratio of games, and then just find enough winners to make it 49-51.
That's the ideal situation when everyone plays by the rules. To make that scheme work as a "crypto"currency, you'll also have to make sure that breaking the rules doesn't bring a benefit. E.g. the validation examples can't be known beforehand, otherwise someone could just overfit to them to get a higher score with less work. But I think it might be possible to make it work.
Not all gradient updates improve performance