Repeatability: As Difficult as it is Important | Hacker News Reader

Repeatability: As Difficult as it is Important

systemsapproach.org

45 pointszdw15 comments

When I write up tutorials/explanations[0-5], or do presentations, I spend a lot of time (sometimes, weeks), testing and refining the artifacts.

I always provide gists, repos, and playgrounds, and they are as standalone as possible (no dependencies). I think that it is very important that readers be able to exactly reproduce what I’m saying, and that they are able to understand it completely.

I don’t think anyone actually cares (except me), but I’m anal. I do it for myself.

Most tutorials that I take are pretty slapdash, when it comes to supporting artifacts. People just throw them together. Sometimes, they are still working on them, as the class is starting, but I still learn what I need, so it probably isn’t mandatory that the class material be circumspect.

[UPDATE] I just noticed that HN formats link pairs, where I have a link, followed by another one in parentheses, in parallel columns, when displayed on small screens. Cool stuff.

[0] https://littlegreenviper.com/series/universal-links/ (https://github.com/LittleGreenViper/ImplementingUniversalLin...)

[1] https://littlegreenviper.com/series/bluetooth/ (https://github.com/ChrisMarshallNY/ITCB-master)

[2] https://littlegreenviper.com/miscellany/leaving-a-legacy/ (https://github.com/LittleGreenViper/Ozymandias)

[3] https://littlegreenviper.com/series/swiftwater/

[4] https://littlegreenviper.com/series/spm/

[5] https://littlegreenviper.com/miscellany/

The question is reproducibility, the answer is Nix.

It doesn't matter if you're using Python, Rust, C, Haskell, bash or some fresh hell written in PHP 4, perl and a fork of Netscape from 1999.

The answer is Nix. The Nix way _solves_ dependency management.

https://edolstra.github.io/pubs/phd-thesis.pdf

I keep meaning to try it. I love the idea.

Docker being good enough for many circumstances means there has been no pain motivation to try it for me. Docker vs. nothing is a huge leap. Deploying Python in particular would be untenable with out it (or a VM init script)

I heard Nix is a great way to build containers though.

> I heard Nix is a great way to build containers though.

It is, I'd recommend the fantastic https://www.youtube.com/watch?v=6Le0IbPRzOE to appreciate some major differences.

Agreed - although I came to the same sorts of realisations through Guix, the fundaments of the projects are the same (as I presume and hope you and other Nix people know). So the answer is Nix and its spiritual sibling, Guix, and any other project that attempts to implement the meat of that thesis there (which I hadn't seen actually - thank you!).

I feel there's a surprisingly strong standoffishness between the two communities of users, but that's a side point.

On topic though, for example, https://guix.gnu.org/en/blog/2020/reproducible-computations-..., and there are other excellent posts on the Guix blog going in to great detail on the topic of reproducibility and the efforts they are putting in there.

I tried using nix to build some old software, but failed miserably. Could you give me an example guide to someone packaging older software, that often also needs older compilers, with Nix?

My rant today on regression.[1]

WGPU.rs, Rust's cross-platform graphics library, just reduced performance by 21%. They didn't notice. While WGPU supports Windows, MacOS, Linux, and Android, it started as WebGPU support in browsers. So it comes from web dev land, the land of "move fast and break things", "agile", and not worrying too much about performance until things get really bad.

This is part of why so many game projects have abandoned Rust. The graphics stack, four years on, still isn't very good.

[1] https://www.reddit.com/r/rust_gamedev/comments/1giujbk/wgpu_...

The maintainers/developers do explicitly deprioritize performance, but I agree that it feels wrong for a "suitable for general purpose graphics" engine with the implication of being "modern" to deprioritize it that much. Although I guess it's an unforgiving trade-off to have to make.

Did they actually write that somewhere?

WGPU, due to breadth of coverage. has succeeded in becoming pretty much the only game in town. There's Vulkano, a safe wrapper for Vulkan, but only a few projects use it and it hasn't stabilized yet. There's Ash, which is an unsafe wrapper for Vulkan. That's pretty raw. Some Rust projects still use OpenGL, which has better support, or talk to Direct-X directly. There just isn't a good fast modern Rust graphics stack that covers the main desktop platforms.

> Did they actually write that somewhere?

wgpu.rs for comments about "general purpose", "modern" is my own interjection, and [1] about the difference in priorities between the WGPU maintainers and (funnily enough) you.

I do believe a cross-platform and modern interface is difficult to square with high performance. I don't think either side is wrong to choose the priorities it has, but someone will be underserved. They probably need far more manpower/talent than they have to improve performance proportionately.

[1] https://github.com/gfx-rs/wgpu/discussions/5525

Most of the time,

   git clone URL
   cd PROJECTNAME

gets you most of the way there.

For Rust,

   cargo test

usually takes care of all the dependencies and building.

The old days of

   ./autoconf
   make

are over, but there are modern replacements.

Python and Javascript have too many build and install systems and much worse versioning problems.

And as someone mostly working with Python, I can tell you, while there are good solutions to have reproducible setups, the mindset of people is just not there yet. People don't care enough or are too shortsighted to get it.

The problem with out-of-band package systems is a lack of quality control, and accountability. Entire ecosystems emerged to handle Python Environment specific version issues, and often ignored a systemic underlying legacy library deprecation (same code may not work in 10 months).

Despite personal opinions on Rust use-cases/scope-creep (why not use a higher level llvm language if you want safer syntax.) Cargo like npm, php pear, and pip3 etc. traded convenience for long-term security rot, high maintenance cycles, and version instability within downstream source code.

i.e. the proponents will likely feel like they are in perpetual Beta, as dependent resources permute under conflicting use-cases.

People often confuse convenience with Workmanship Standards, and rotten code trees in any language have the same result. This is not a new phenomena ( https://en.wikipedia.org/wiki/Second-system_effect ) =3

Only recently had someone "update" a thing, and in the process throwing reproducibility and checksums all out of the window. Told that person, that they built a nice prototype, but that those things need to be reintroduced to the thingy. But that person doesn't see the need. I guess some people just build throwaways all day every day. When in 2 months someone else wants to build the thing, it might simply not build any longer, because that dude did not pay attention to reproducibility at all.

This attitude seems to be wide-spread in Python land, unfortunately. Even prominent packages don't properly lock their versions and hashes. Many people deliver, I dare say, shoddy work.

Go is pretty good, too.

Reply on news.ycombinator.com